Free Software Downloads and Reviews at www.criticalfiles.com
Search Software
Software Reviews | Our Awards | Most Popular | New & Updated

Crawlers Software

Programs 1-3 of 3 Pages: 1
Reduce the need for user clients to recache your web pages and saving bandwidth.
Create Snazzy Google Sitemaps in Seconds!
Identify which spiders visited your website, when, and what pages were indexed
ETagFix 1.00
ETagFix is an ISAPI plugin for Internet Information Server (IIS) that changes the etag produced by IIS so that your etag does not change every time your computer reboots. This can result in web crawlers and user clients wanting to recache your web page when they don't need to.
10 / 10
CoffeeCup Google SiteMapper 4.5
By placing a Sitemap-formatted file on your webserver, you enable the Google crawlers to find out what pages are present and which have recently changed, and to crawl your site accordingly. Using Sitemaps to inform and direct the Google crawlers, they hope to expand their coverage of the web and improve the time of inclusion in their index.
9 / 10
Robot-Manager 3.1
Manage and track visits by robots, spiders and crawlers to your website. Identify which spiders visited, when they visited, and what pages they indexed. Direct spiders to your most important pages, and exclude them from non-public pages. Maximizes effectiveness with search engines by avoiding over/under submitting. Excellent to improve search engine placements for users of automatic submission software such as WebPosition, TopDog, AddWeb, etc.
7 / 10
Programs 1-3 of 3 Pages: 1