Download HttpCrawler

Description

A simple, extensible and multi-threaded HTTP Crawler written in C# using Visual Studio 2010 and .NET 4.0.
Source Files

The download file httpcrawler.zip has the following entries.

BuildProcessTemplates/DefaultTemplate.11.1.xaml
BuildProcessTemplates/DefaultTemplate.xaml
BuildProcessTemplates/LabDefaultTemplate.11.xaml
BuildProcessTemplates/UpgradeTemplate.xaml
HttpCrawler.Demo/HttpCrawler.Demo.csproj
HttpCrawler.Demo/HttpCrawler.Demo.csproj.vspscc
HttpCrawler.Demo/Program.cs/*from   w  w w.j av a2 s .  c om*/
HttpCrawler.Demo/Properties/AssemblyInfo.cs
HttpCrawler.Demo/app.config
HttpCrawler.Engine.Test/CssHttpCrawlerContentProcessorTests.cs
HttpCrawler.Engine.Test/HttpCrawler.Engine.Test.csproj
HttpCrawler.Engine.Test/HttpCrawler.Engine.Test.csproj.vspscc
HttpCrawler.Engine.Test/Internal/StringHttpCrawlerContent.cs
HttpCrawler.Engine.Test/Properties/AssemblyInfo.cs
HttpCrawler.Engine/ConsoleHttpCrawlerEngineLogger.cs
HttpCrawler.Engine/CssHttpCrawlerContentProcessor.cs
HttpCrawler.Engine/DoNothingHttpCrawlerContentPersister.cs
HttpCrawler.Engine/DoNothingHttpCrawlerEngineLogger.cs
HttpCrawler.Engine/FileSystemHttpCrawlerContentPersister.cs
HttpCrawler.Engine/HtmlHttpCrawlerContent.cs
HttpCrawler.Engine/HtmlHttpCrawlerContentProcessor.cs
HttpCrawler.Engine/HttpCrawler.Engine.csproj
HttpCrawler.Engine/HttpCrawler.Engine.csproj.vspscc
HttpCrawler.Engine/HttpCrawlerContent.cs
HttpCrawler.Engine/HttpCrawlerContentFactory.cs
HttpCrawler.Engine/HttpCrawlerEngine.cs
HttpCrawler.Engine/IHttpCrawlerContentPersister.cs
HttpCrawler.Engine/IHttpCrawlerContentProcessor.cs
HttpCrawler.Engine/IHttpCrawlerEngine.cs
HttpCrawler.Engine/IHttpCrawlerEngineLogger.cs
HttpCrawler.Engine/IHttpCrawlerUriFollowDeterminer.cs
HttpCrawler.Engine/Properties/AssemblyInfo.cs
HttpCrawler.Engine/SimilarOriginHttpCrawlerUriFollowDeterminer.cs
HttpCrawler.Engine/XmlHttpCrawlerContent.cs
HttpCrawler.Extensions.Tests/HttpCrawler.Extensions.Tests.csproj
HttpCrawler.Extensions.Tests/HttpCrawler.Extensions.Tests.csproj.vspscc
HttpCrawler.Extensions.Tests/ParallelHelperTests.cs
HttpCrawler.Extensions.Tests/Properties/AssemblyInfo.cs
HttpCrawler.Extensions/ConcurrentQueueExtensions.cs
HttpCrawler.Extensions/HashSetExtensions.cs
HttpCrawler.Extensions/HttpCrawler.Extensions.csproj
HttpCrawler.Extensions/HttpCrawler.Extensions.csproj.vspscc
HttpCrawler.Extensions/ParallelHelper.cs
HttpCrawler.Extensions/PathHelper.cs
HttpCrawler.Extensions/Properties/AssemblyInfo.cs
HttpCrawler.Extensions/UriExtensions.cs
HttpCrawler.Extensions/UriHelper.cs
HttpCrawler.Extensions/WebResponseExtensions.cs
HttpCrawler.sln
HttpCrawler.vsmdi
HttpCrawler.vssscc
Libraries/HtmlAgilityPack.License.txt
Libraries/HtmlAgilityPack.dll
Local.testsettings
TraceAndTestImpact.testsettings
Download

Click the following link to download httpcrawler.zip.
httpcrawler.zip