360Spider 404checker 80legs AIBOT ALittle Client ASPSeek Acunetix AhrefsBot Aipbot Alexibot Anarchie Ankit Apexoo Aspiegel Asterias Atomseobot Attach BDFetch BLEXBot BacklinkCrawler Bandit Barkrowler Battleztar Bazinga BetaBot Bigfoot Bitacle Blackboard Bolt BotALot Brandprotect Buck BuiltBotTough BunnySlippers CATExplorador CCBot CODE87 Calculon Cegbfeieh CensysInspect CheeseBot ChinaClaw Chlooe Claritybot Cocolyzebot Cogentbot Cosmos Craftbot Crescent CrunchBot Custo CyotekWebCopy DSearch Dirbuster Discoverybot Dispatch DomCopBot DomainSigmaCrawler DomainStatsBot Dotbot Download Wonder ECCP/1.0 Exabot ExtLinksBot Extractor EyeNetIE FlashGet Flunky Freeuploader FrontPage FyberSpider Fyrebot GT::WWW GalaxyBot Genieo GetWeb Go!Zilla GoZilla GrabNet GrapeFX HMView HaosouSpider Harvest HonoluluBot Humanlinks IDBTE4M Image Sucker InternetSeer Iria Iskanie JetCar Jorgee JustView Keyword Density Kozmosbot LNSpiderguy Lanshanbot Lftp LinkScan Linkbot LinqiaMetadataDownloaderBot LinqiaScrapeBot Lipperhey Spider MIIxpc MJ12bot MSIECrawler MTRobot Magnet Mail.RU_Bot MarkMonitor MarkWatch Mass Downloader MauiBot Mb2345Browser Meanpathbot Mediatoolkitbot Mozlila Mr.4x3 Musobot NPbot Nameprotect Navroad NearSite NetAnts NetZIP Netcraft Nettrack Netvibes Nikto Not Octopus OpenVAS Openfind OrangeBot OrangeSpider Page Analyzer PageScorer Papa Foto Petalbot Picscout Picsearch PictureFinder Piepmatz Pimonster Pixray Psbot Quick-Crawler RSSingBot RankFlex Rankivabot RankurBot RebelMouse RepoMonkey Ripper RocketCrawler SBIder SEOkicks SEOlyticsCrawler SEOstats SISTRIX SalesIntelligent Scanbot Screaming ScrepyBot SearchmetricsBot Seekport SemanticJuice Semrush SeobilityBot Siteimprove Sociscraper SpaceBison Spbot Sqlworm Sqworm Sucuri SuperBot SuperHTTP Surfbot TeleportPro Telesoft Telesphoreo Thumbor TightTwatBot Trendictionbot Turnitin Twice Upflow VB Project VCI VeriCiteCrawler Virusdie Voil Voltron WISENutbot WWW-Collector-E WWW::Mechanize WWWOFFLE Wallpapers/3.0 Web Auto Web Collage Web Enhancer Web Fuck WebAuto WebCollage WebCopier WebImageCollector WebSauger WebWhacker WebZIP Webalta Webshag WebsiteExtractor Webster Who.is Bot WiseGuys Robot Wotbox Xaldon WebSpider Xaldon_WebSpider YoudaoBot Zade Zeus Zitebot ZumBot archive.org_bot arquivo-web-crawler arquivo.pt autoemailspider coccocbot com.plumanalytics crawler.feedback crawler4j domainsproject.org gopher heritrix isitwp.com iubenda-radar lwp-trivial mediawords muhstik-scan page scorer plumanalytics seocompany.store sexsearcher spyfu sysscan tAkeOut trendiction.com ubermetrics-technologies.com webprosbot x09Mozilla