Build the DownloadString : Web Crawler « Network « C# / C Sharp

Home
C# / C Sharp
1.2D Graphics
2.Class Interface
3.Collections Data Structure
4.Components
5.Data Types
6.Database ADO.net
7.Design Patterns
8.Development Class
9.Event
10.File Stream
11.Generics
12.GUI Windows Form
13.Language Basics
14.LINQ
15.Network
16.Office
17.Reflection
18.Regular Expressions
19.Security
20.Services Event
21.Thread
22.Web Services
23.Windows
24.Windows Presentation Foundation
25.XML
26.XML LINQ
C# / C Sharp by API
C# / CSharp Tutorial
C# / CSharp Open Source
C# / C Sharp » Network » Web CrawlerScreenshots 
Build the DownloadString





using System;
using System.IO;
using System.Net;
using System.Text.RegularExpressions;

class MainClass {
    private static void Main() {
        string remoteUri = "http://www.apress.com";
        WebClient client = new WebClient();
        string str = client.DownloadString(remoteUri);
        MatchCollection matches = Regex.Matches(str, @"http\S+[^-,;:?]\.gif");
        foreach (Match match in matches) {
            foreach (Group grp in match.Groups) {
                string file = grp.Value.Substring(grp.Value.LastIndexOf('/'1);
                try {
                    Console.WriteLine("Downloading {0} to file {1}", grp.Value, file);
                    client.DownloadFile(new Uri(grp.Value), file);
                catch {
                    Console.WriteLine("Failed to download {0}", grp.Value);
                }
            }
        }
    }
}
           
       
Related examples in the same category
1.Set the BaseAddress for WebClient
2.Download a web page in a thread
3.Output webpage content
4.Create GetResponse from WebRequest
5.Check the ContentType
6.MiniCrawler: A skeletal Web crawler
www.java2java.com | Contact Us
Copyright 2009 - 12 Demo Source and Support. All rights reserved.
All other trademarks are property of their respective owners.