Scrapy Course: Python Web Scraping & Crawling for Beginners
Master web scraping with Scrapy and Python 3. Includes databases, web crawling, creating spiders and scraping Amazon.
Updated on Jun, 2025
Language - English
Development ,Data Science,Python
Lectures -28
Duration -3.5 hours
Lifetime Access
Lifetime Access
30-days Money-Back Guarantee
Get your team access to 10000+ top Tutorials Point courses anytime, anywhere.
Course Description
In early 2008, Scrapy was released into this world and it soon became the #1 Web Scraping tool for beginners. Why? It's because it's simple enough for beginners yet advanced enough for the pros.
Here are some of the use cases -
- E-commerce ( Amazon ) - Scrape product names, pricing and reviews
- Data - Get a huge collection of data/images for Machine Learning
- Email Address - Big companies scrape it and use it for Lead Generation
Come learn with me and I'll show you how you can bend Scrapy to your will. This course is great for beginners in Python at any age and any level of computer literacy.
The goal is simple: learn Scrapy by working on real projects step-by-step while we explain every concept along the way.
What you will learn?
Scrape Data from nearly Any Website
Build your own Spiders from scratch for all types of Web Scraping purposes
Transfer the data that you have scraped into JSON, CSV, and XML
Store the data in databases - SQLite3, MySQL, and MongoDB
Create Web Crawlers and follow links on any web page
Logging in to websites
Bypassing restrictions & bans by using User-Agents and Proxies
Internalize the concepts by completely scraping Amazon and get ready to scrape more advanced websites.
Goals
- Scraping single or multiple websites with Scrapy
- Building powerful crawlers and spiders
- Creating a web crawler for Amazon from scratch
- Bypass restrictions using User-Agents and Proxies
- Logging into Websites with Scrapy
- Storing data extracted by Scrapy into SQLite3, MySQL, and MongoDB databases
- Exporting data extracted by Scrapy into CSV, XML, or JSON files
- Understand Xpath and CSS selectors to extract data
Prerequisites
- Python Level: Beginner. This Scrapy tutorial assumes that you already know the Python basics ( variables, functions, etc. ) No need for more as we cover Object Oriented Programming in the BONUS section of this course.
- Please watch the preview lectures and read the description of this course before enrolling.
Curriculum
Check out the detailed breakdown of what’s inside the course
Introduction to Scrapy and Web Scraping
3 Lectures
-
Web Scraping, Spiders and Crawling
03:18
03:18
-
How does Scrapy work?
05:29
05:29
-
Robots.txt
01:18
01:18
Installation Guide for Scrapy
2 Lectures
Creating your first Spider
3 Lectures
Extracting data with Scrapy
3 Lectures
Storing the scraped data
3 Lectures
Extracting data to Databases : SQLite3, MySQL & MongoDB
4 Lectures
Web Crawling and Pagination
2 Lectures
Logging into websites using Scrapy
1 Lectures
Scraping Amazon.com & Bypassing Restrictions
4 Lectures
BONUS : Classes, Objects and Inheritance
2 Lectures
Instructor Details
Attreya Bhatt
When I was a kid I saw this YouTube video on how to make a folder invisible on Windows. I have never looked back since then. My love for technology has only grown.
I started with security since that was one of the areas that fascinated me. Then i went on to win the award for designing using Photoshop at Cofas '2012. On the destructive side, I always made scripts that used to mess up the systems at my school. I was almost suspended. I learned my lesson and vowed to do only constructive things. To make people aware of security issues, along with 2 friends started a Facebook page and group called YAP
To make things more interesting Web development came into my life which helped me get into the most prestigious chapter at my college IEEE. This helped me master different things that got my attention
- Android Development
- Augmented Reality
- Machine learning
- Python Development
- Internet of things ( IOT )
I never really wanted to go to a college. Still against it but it made me realize that the joy of creating something with a team of people is unparalleled. I created a blog which helps people who are not so familiar with technology get familiar with it and benefit from it. It has more 5,00,000 views today.
Things I learned and am still learning from it are Writing,Wordpress CMS, SEO, Google Analytics and Adsense and how to market a product after creating it.
Course Certificate
Use your certificate to make a career change or to advance in your current career.
Our students work
with the Best
Related Video Courses
View MoreAnnual Membership
Become a valued member of Tutorials Point and enjoy unlimited access to our vast library of top-rated Video Courses
Subscribe now
Online Certifications
Master prominent technologies at full length and become a valued certified professional.
Explore Now