Skip to content
#

scraping

Star

Web scraping is the process of programmatically retrieving web pages and extracting structured data from them for analysis, storage, or reuse. It powers price monitoring, search indexing, market research, and training-data collection.

Scraping ranges from plain HTTP requests and HTML parsing to full browser automation for JavaScript-rendered and anti-bot-protected pages. The ecosystem includes request libraries, HTML parsers, headless browsers, and cloud extraction platforms.

Here are 8,707 public repositories matching this topic...

Language: All
Filter by language
All 8,707 Python 4,198 JavaScript 877 Jupyter Notebook 813 TypeScript 611 HTML 320 Go 236 Java 138 PHP 131 C# 99 Rust 86
Sort: Most stars
Sort options
Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated
crawlee

apify / crawlee

Star 25.7k

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

nodejs javascript npm crawler scraper automation typescript web-crawler headless scraping crawling web-scraping web-crawling headless-chrome apify puppeteer playwright
  • Updated Sep 8, 2026
  • TypeScript

apify / crawlee-python

Star 9.5k

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

python crawler scraper automation web-crawler headless scraping crawling selenium pip web-scraping beautifulsoup web-crawling headless-chrome apify parsel playwright
  • Updated Sep 8, 2026
  • Python
Followers
343 followers
Website
github.com/topics/web-scraping

Related topics

browser-automation headless-browser playwright puppeteer selenium