I Wonder If This Is Working…
May 31st, 2009
I’ve recently been doing some intense Visual Basic work for a new project, and because my VB is a little rusty, I keep having to check things online. (I threw away the best VB book a year or two ago, when I thought I’d done my last ever VB project. Silly me).
Anyway… Virtually every search I do in Google comes up with a page at bigresource.com in the top 10 results
BigResource.com scans a few programming forums, scrapes the first post in each thread, and lists them on a page. Each page then has links to a lot of keyword-related other threads beneath it, with the text of those threads as well. So each page in bigresource.com is keyword-rich, themed, and full of content.
Every bit of it scraped.
So why is it doing so well in Google.
And how much are they earning from the Adsense ads they’ve got?
And how could you replicate this?
Thing with bigresource.com is when you go to view any of the articles, it presents you with a captcha code. I don’t get why. I’m concerned it is just a vehicle for getting humans to solve a captcha so that they can bypass the captcha protection on other websites…
Oh I hope it isn’t the case, and its just my own little conspiracy theory.
Hi,
Good point. I never saw the point of the captchas myself. You’d think it would be to prevent spiders and web scraping bots from scraping their website, but the page after the captch is the source site, not bigresource.com’s page.
You could be onto something…
Thanks for sharing.
Andy
Ahh, I’ve been annoyed by bigresource in the past, but never thought about the possible evil implications of the CAPTCHA. Anyway, I now use the CustomizeGoogle Firefox plugin to filter them in my Google search results, so I don’t click on them even accidentally…
Hi Matt,
Ah, what a great plugin. Now, all I need to do is find a list of all those mass-produced software directory sites that turn up when I’m looking for something vaguely related to software, and I’ll be in heaven
Great suggestion.
For anyone else who’s not heard of this plugin, it’s at http://www.customizegoogle.com/
What did we do before Firefox???
Andy
Hi Andy,
Unfortunately, customizegoogle doesn’t seem to be supporting Firefox 3.5 yet. As Experts-exchange has already started to irritate me, I found an alternative, which is a Greasemonkey script called Google Search Filter Plus. This doesn’t do as much as customizegoogle, but the filtering bit works absolutely fine.
http://userscripts.org/scripts/show/39996
Cheers,
Matt
Hi Matt,
Ah, I didn’t even know 3.5 was out yet, so I’m still using 3.0something.
Andy