Posts Tagged ‘Variables’

Noobies Guide on How to Scrape: Part 2 – URLs, URL Variables, and using Live HTTP Headers

Wednesday, April 8, 2009 21:11 1 Comment

Understanding the fundamentals of how sites communicate with themselves, and how we communicate with them, is crucial in being able to reverse engineering a site for our scraper.   Luckily it’s pretty easy for the most part.
Anatomy of a URL

The protocol your using.
The website your trying to get to.  Although www is synonymous with the base [...]

This was posted under category: Automation, Blackhat, Noobie Scraping Guide Tags: , , , ,