Web scraping - andrejIka/frontend-toolset GitHub Wiki

• https://github.com/lorien/awesome-web-scraping/blob/master/proxy_services.md

Python

• Scrapy
• beautiful soup

PHP

• Gouette

Java

• Selenium
• Selenide

JS

• Nightmare
• Puppeteer
• casperjs
• phantomjs
• https://github.com/medialab/artoo - for console log data while scraping
• https://slimerjs.org/
• set of apis for scraping https://phantombuster.com/
• http://nightwatchjs.org/
• https://nickjs.org/

Third party services

• https://www.scraperapi.com/?utm_source=qwiki
• https://smartproxy.io/
• https://elements.heroku.com/addons/mongolab