ADmantX AIBOT ALittle Client ASPSeek Aboundex AhrefsBot AllSubmitter Anarchie Anarchy99 Apexoo Asterias Atomseobot AwarioRssBot BDFetch Bandit Barkrowler Battleztar Bazinga Bigfoot Black Hole Blackboard Brandwatch BuiltBotTough CATExplorador CODE87 Cegbfeieh CensysInspect CheTeam CheeseBot Chlooe Citoid Collector Copier CopyRightCheck Copyscape CrunchBot DBLBot DIIbot Dirbuster DnBCrawler-Analytics DomCopBot Ecxi Evil Exabot ExtLinksBot Extractor ExtractorPro Extreme Picture Finder Ezooms FDM FHscan FemtosearchBot Firefox/7.0 Foobot FrontPage Fuzz FyberSpider G-i-g-a-b-o-t GalaxyBot Go!Zilla Go-Ahead-Got-It GoZilla Gotit Grabber GrapeFX GrapeshotCrawler HEADMasterSEO HTMLparser Hloader Humanlinks HybridBot IDBTE4M Iblog IndeedBot Intelliseek InternetSeer JOC Web Spider JennyBot JikeSpider Jyxobot Keyword Density Kozmosbot Larbin LeechFTP LeechGet LexiBot Lftp Likse LinqiaMetadataDownloaderBot LinqiaRSSBot LinqiaScrapeBot MQQBrowser MTRobot MarkMonitor Mass Downloader MauiBot MeanPath Bot Meanpathbot MegaIndex.ru Metauri Microsoft Data Access Moblie Safari Mojeek Mojolicious NICErsPRO NPbot Navroad NearSite Needle NetLyzer NetSpider Netcraft NextGenSearchBot Nibbler Nikto Nimbostratus Ninja Nmap Nutch Offline Explorer OnCrawl OpenLinkProfiler Openvas OutclicksBot PECL::HTTP Page Analyzer PageAnalyzer PageGrabber PageThing.com Pandalytics Panscient Pavuk Petalbot PictureFinder PleaseCrawl Pockey Pu_iN Pump PxBroker Quick-Crawler RankFlex RankingBot Rankivabot RankurBot Reaper RepoMonkey Ripper Rogerbot SBIder SEOlyticsCrawler ScanAlert Scanbot ScoutJet Scrapy Searchestate SearchmetricsBot Seekport Semrush Siphon SiteCheckerBotCrawler SiteLockSpider SiteSnagger SiteSucker Siteimprove SmartDownload Snake Snapbot Snoopy SocialRankIOBot Sogou web spider SpankBot Spanner Spinn3r SputnikBot Sqlmap Sqlworm Sqworm SuperBot Surfbot SurveyBot Swiftbot Szukacz T8Abot Teleport Telesphoreo The Intraformant Toata Trendiction True_Robot Typhoeus URLy Warning Upflow VB Project VelenPublicWebCrawler Virusdie VoidEYE WWW-Collector-E WWW-Mechanize WWW::Mechanize Wallpapers Wallpapers/3.0 WeSEE Web Collage Web Fetch WebBandit WebSauger WebStripper WebZIP WebmasterWorldForumBot Webshag Whack Whacker Whatweb Who.is Bot Widow Wonderbot Woobot Wprecon Xaldon WebSpider Xenu Zauba ZoomBot ZyBorg adscanner arquivo-web-crawler arquivo.pt autoemailspider coccocbot-web com.plumanalytics demandbase-bot domainsproject.org facebookscraper heritrix internetVista monitor lwp-request lwp-trivial magpie-crawler muhstik-scan plumanalytics polaris version scan.lol sexsearcher sitechecker.pro sp_auditbot spyfu trendiction.com trendiction.de webpros.com webprosbot x22Mozilla xpymep1.exe