Download NRobots.txt
Description
The Robots Exclusion Protocol or robots.txt protocol, is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a website which is otherwise publicly viewable.
Source Files
The download file nrobots.zip has the following entries.
BuildProcessTemplates/DefaultTemplate.11.1.xaml
BuildProcessTemplates/LabDefaultTemplate.11.xaml
BuildProcessTemplates/UpgradeTemplate.xaml
Robots/Lib/xunit.dll//from w w w . j av a 2 s . c o m
Robots/LocalTestRun.testrunconfig
Robots/Robots.sln
Robots/Robots.vsmdi
Robots/Robots.vssscc
Robots/Robots/Fluent/RobotsBuilder.cs
Robots/Robots/IRobots.cs
Robots/Robots/Models/AccessUrlEntry.cs
Robots/Robots/Models/AllowEntry.cs
Robots/Robots/Models/CommentEntry.cs
Robots/Robots/Models/CrawlDelayEntry.cs
Robots/Robots/Models/DisallowEntry.cs
Robots/Robots/Models/Entry.cs
Robots/Robots/Models/EntryType.cs
Robots/Robots/Models/SiteMapEntry.cs
Robots/Robots/Models/UrlEntry.cs
Robots/Robots/Models/UserAgentEntry.cs
Robots/Robots/Properties/AssemblyInfo.cs
Robots/Robots/Robots.cs
Robots/Robots/Robots.csproj
Robots/Robots/Robots.csproj.vspscc
Robots/Robots/UserAgents/BotsUserAgents.cs
Robots/Robots/UserAgents/UserAgents.cs
Robots/Robots/app.config
Robots/RobotsTests/AccessUrlEntryTest.cs
Robots/RobotsTests/AllowEntryTest.cs
Robots/RobotsTests/CommentEntryTest.cs
Robots/RobotsTests/CrawlDelayEntryTest.cs
Robots/RobotsTests/DisallowEntryTest.cs
Robots/RobotsTests/EntryTest.cs
Robots/RobotsTests/Properties/AssemblyInfo.cs
Robots/RobotsTests/RobotsBuilderTest.cs
Robots/RobotsTests/RobotsTest.cs
Robots/RobotsTests/RobotsTests.csproj
Robots/RobotsTests/RobotsTests.csproj.vspscc
Robots/RobotsTests/SiteMapEntryTest.cs
Robots/RobotsTests/Test References/Robots.accessor
Robots/RobotsTests/UrlEntryTest.cs
Robots/RobotsTests/UserAgentEntryTest.cs
Download
Click the following link to download nrobots.zip.
nrobots.zip