Scrape Google

TeachMe

Member
Joined
Jun 27, 2012
Messages
16
Programming Experience
Beginner



  • Here's what I'm trying to accomplish:


    I would like to make a simple program that can parse the google serp and return the results to a richtextbox1.text. Here is an example of a very simple program that I'm trying to create for my own personal use:
    Google Scraper Footprint Finder


    I would appreciate if someone could help me by show some simple example code.


    Thanks...

 
Add a WebBrowser control to a form and nagivate to the Google url. After document is completed (DocumentCompleted event), you can get the urls from the browsers Document.Links collection.
 
Add a WebBrowser control to a form and nagivate to the Google url. After document is completed (DocumentCompleted event), you can get the urls from the browsers Document.Links collection.


You're saying to work around the issue of the google redirect links, I should just navigate to each google redirect, let it redirect and then extract the link? I would like to get the links directly from the google search engine results page instead if possible (just like the simple tool that I left in the link above does). Someone should download the tool and see how it works. It's very easy to use and understand. I'm just trying to make a simple tool just like it.
 
You're saying to work around the issue of the google redirect links, I should just navigate to each google redirect, let it redirect and then extract the link?
No, that would be a stupid approach, you only need to navigate to the Google search page. You can read the redirect url from the url query of such links. With for example Uri class you can get the Query, split it, and get the 'url' name-value pair, and again use Uri class to unescape the value data string.
 
No, that would be a stupid approach, you only need to navigate to the Google search page. You can read the redirect url from the url query of such links. With for example Uri class you can get the Query, split it, and get the 'url' name-value pair, and again use Uri class to unescape the value data string.


Hey John, could you show me a working example? I'm new to coding, but I will try my best to study and learn from it. Thanks!
 
Why don't you give it a start and post what you have troubles with? If you haven't already, you should read the relevant help topics for String class and Uri class.

Google redirect links seems very uncommon, by the way. I did manage to find one in search results yesterday, but today I'm having a hard time producing a search that would return any such links.
 
Back
Top