Thursday, February 11, 2016

Tor IP changing and web scraping

Anonymous web scraping

Many of us who scrape web pages, be it for fun, data, love or something else, are concerned about anonymity. Well, not anonymity per se, we just don't want our IP blacklisted.

Many of us choose Tor and its network to achieve our goal. In this post I am going to share my experience and observations using Tor + Privoxy for Python driven web scraping.