Web and PDF Scraper Refactoring
This repository contains the example code of the Web and PDF scraper code roast. Here are the links to the videos:
- Part 1: https://youtu.be/MXM6VEtf8SE
- Part 2: (coming soon)
Web and PDF Scraper Refactoring
Overview
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
Gerapy Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js. Documentation Documentation
2.9k Jan 03, 2023
This code will be able to scrape movies from a movie website and also provide download links to newly uploaded movies.
Movies-Scraper You are probably tired of navigating through a movie website to get the right movie you'd want to watch during the weekend. There may e
1 Jan 31, 2022
Web Scraping Instagram photos with Selenium by only using a hashtag.
Web-Scraping-Instagram This project is used to automatically obtain images by web scraping Instagram with Selenium in Python. The required input will
3 Nov 24, 2022
Using Python and Pushshift.io to Track stocks on the WallStreetBets subreddit
wallstreetbets-tracker Using Python and Pushshift.io to Track stocks on the WallStreetBets subreddit.
91 Dec 08, 2022
用python爬取江苏几大高校的就业网站，并提供3种方式通知给用户，分别是通过微信发送、命令行直接输出、windows气泡通知。
crawler_for_university 用python爬取江苏几大高校的就业网站，并提供3种方式通知给用户，分别是通过微信发送、命令行直接输出、windows气泡通知。 环境依赖 wxpy,requests,bs4等库 功能描述 该项目基于python，通过爬虫爬各高校的就业信息网，爬取招聘信
8 Aug 16, 2021
Async Python 3.6+ web scraping micro-framework based on asyncio
Ruia 🕸️ Async Python 3.6+ web scraping micro-framework based on asyncio. ⚡ Write less, run faster. Overview Ruia is an async web scraping micro-frame
1.6k Jan 01, 2023
Web-scraping - Program that scrapes a website for a collection of quotes, picks one at random and displays it
web-scraping Program that scrapes a website for a collection of quotes, picks on
1 Jan 07, 2022
Web Crawlers for Data Labelling of Malicious Domain Detection & IP Reputation Evaluation
Web Crawlers for Data Labelling of Malicious Domain Detection & IP Reputation Evaluation This repository provides two web crawlers to label domain nam
1 Nov 05, 2021
A simple python script to fetch the latest covid info
covid-tracker-script A simple python script to fetch the latest covid info How it works First, get the current date in MM-DD-YYYY format. Check if the
0 Dec 15, 2021
Demonstration on how to use async python to control multiple playwright browsers for web-scraping
Playwright Browser Pool This example illustrates how it's possible to use a pool of browsers to retrieve page urls in a single asynchronous process. i
8 Oct 27, 2022
A universal package of scraper scripts for humans
Scrapera is a completely Chromedriver free package that provides access to a variety of scraper scripts for most commonly used machine learning and data science domains.
299 Dec 15, 2022
京东茅台抢购最新优化版本，京东秒杀，添加误差时间调整，优化了茅台抢购进程队列
京东茅台抢购最新优化版本，京东秒杀，添加误差时间调整，优化了茅台抢购进程队列
776 Jul 28, 2021
A simple Discord scraper for discord bots
A simple Discord scraper for discord bots. That includes sending an guild members ids to an file, Mass inviter for joining servers your bot is in and Fetching all the servers of the bot (w/MemberCoun
1 Jan 06, 2022
WebScraper - A script that prints out a list of all EXTERNAL references in the HTML response to an HTTP/S request
Project A: WebScraper A script that prints out a list of all EXTERNAL references
2 Apr 26, 2022
A scalable frontier for web crawlers
Frontera Overview Frontera is a web crawling framework consisting of crawl frontier, and distribution/scaling primitives, allowing to build a large sc
1.2k Jan 02, 2023
Automated data scraper for Thailand COVID-19 data
The Researcher COVID data Automated data scraper for Thailand COVID-19 data Accessing the Data 1st Dose Provincial Vaccination Data 2nd Dose Provincia
31 Apr 17, 2022
爬虫案例合集。包括但不限于《淘宝、京东、天猫、豆瓣、抖音、快手、微博、微信、阿里、头条、pdd、优酷、爱奇艺、携程、12306、58、搜狐、百度指数、维普万方、Zlibraty、Oalib、小说、招标网、采购网、小红书》
lxSpider 爬虫案例合集。包括但不限于《淘宝、京东、天猫、豆瓣、抖音、快手、微博、微信、阿里、头条、pdd、优酷、爱奇艺、携程、12306、58、搜狐、百度指数、维普万方、Zlibraty、Oalib、小说网站、招标采购网》 简介： 时光荏苒，记不清写了多少案例了。
793 Jan 05, 2023
Script for scrape user data like "id,username,fullname,followers,tweets .. etc" by Twitter's search engine .
TwitterScraper Script for scrape user data like "id,username,fullname,followers,tweets .. etc" by Twitter's search engine . Screenshot Data Users Only
19 Nov 17, 2022
Screenhook is a script that captures an image of a web page and send it to a discord webhook.
screenshot from the web for discord webhooks screenhook is a script that captures an image of a web page and send it to a discord webhook.
3 Jun 04, 2022
This was supposed to be a web scraping project, but somehow I've turned it into a spamming project
Introduction This was supposed to be a web scraping project, but somehow I've turned it into a spamming project.
1 Jan 23, 2022