MadPPC

Now that we are up to speed on the data we want to collect, and how cURL works, a basic scraper it’s really just a hop, skip, and a jump away.
Getting Data
The only other point we haven’t covered was how to effectively pull data from our page.   For example say we want to grab the [...]

This post is a re-write.  I didn’t think the last version was very good, was too long, and looks like it even went out of date. This version is shorter, up to date (as of right now), and easier to follow I think.
For the sake of teaching, I’m going to pick a fairly easy target.  [...]

Now we get the idea of POST and GET.  We found our target, we know it’s url structure, we know where the data is, but how do we use PHP to fetch the webpages?
Luckily we have what is call cURL (from PHP.net):
PHP supports libcurl, a library created by Daniel Stenberg, that allows [...]

Understanding the fundamentals of how sites communicate with themselves, and how we communicate with them, is crucial in being able to reverse engineering a site for our scraper.   Luckily it’s pretty easy for the most part.
Anatomy of a URL

The protocol your using.
The website your trying to get to.  Although www is synonymous with the base [...]

Welcome to the Noobies Guide to Scraping: Part 1.  In this installment we are only going to focus on a few very basic things that we are going to need to get started, and no code will be wrote.
What is scraping?  Scraping is the process of getting / gathering data from some web source, whether [...]