360Spider 404enemy ALittle Client Aboundex Acunetix AfD-Verbotsverfahren AiHitBot Aipbot AllSubmitter Anarchie Ankit Apexoo Atomseobot AwarioRssBot BBBike BDCbot BDFetch BacklinkCrawler BetaBot Black Hole BlackWidow BotALot Brandwatch BuiltBotTough BunnySlippers CazoodleBot CensysInspect Citoid Cloud mapping Collector Copier CopyRightCheck Copyscape CrazyWebCrawler Crescent CrunchBot DTS Agent DatabaseDriverMysqli Devil Digincore Discobot Discoverybot Dispatch DnyzBot DomainAppender DomainCrawler DomainSigmaCrawler Dotbot Download Wonder EMail Wolf EasyDL Ecxi EirGrabber Evil Ezooms FHscan FemtosearchBot FlashGet Fuzz FyberSpider Fyrebot G-i-g-a-b-o-t GT::WWW GalaxyBot Go-Ahead-Got-It Gotit GrapeshotCrawler HEADMasterSEO HTTP::Lite Haansoft Havij Hloader HonoluluBot HybridBot Id-search Indy Library Intelliseek Internet Ninja JennyBot JetCar Jetty JikeSpider JustView Jyxobot Keyword Density Kinza Kozmosbot Lanshanbot Larbin Leap Lftp Libwhisker Lightspeedsystems LinkScan LinkWalker Linkbot LinksManager LinqiaMetadataDownloaderBot LinqiaRSSBot LinqiaScrapeBot Litemage_walker Lmspider Ltx71 MFC_Tear_Sample MIDown tool MIIxpc MJ12bot MSFrontPage MSIECrawler MTRobot Magnet Majestic12 Mass Downloader Mata Hari MeanPath Bot MegaIndex.ru MicroMessenger Microsoft URL Control Mojeek Mojolicious Mr.4x3 NICErsPRO Needle NetAnts NetMechanic NetSpider NextGenSearchBot Nimbostratus Nutch OrangeBot PECL::HTTP PHPCrawl Page Analyzer PageGrabber PageScorer Pandalytics Panscient Papa Foto Pavuk Picscout Piepmatz Pimonster PleaseCrawl Pu_iN Pump Quick-Crawler RSSingBot Rankivabot ReGet RebelMouse SEOkicks SEOlyticsCrawler SISTRIX ScoutJet Scrapy Screaming ScreenerBot SearchmetricsBot SentiBot SeobilityBot Seomoz Siphon SiteCheckerBotCrawler Sitevigil SmartDownload Snake Sociscraper Sogou web spider Sosospider Sottopop Spanner SputnikBot Sqworm Stripper SuperHTTP SurveyBot Swiftbot T0PHackTeam TeleportPro Toweyabot Trendiction Trendictionbot True_Robot TurnitinBot TwengaBot URLy Warning URLy.Warning V-BOT Vagabondo VelenPublicWebCrawler VidibleScraper Virusdie VoidEYE WBSearchBot WWW::Mechanize WWWOFFLE Web Enhancer Web Pix Web Sauger WebCopier WebEnhancer WebFuck WebPix WebSauger WebSucker Webalta Website Quester WebsiteQuester Whack Whacker Who.is Bot Widow Woobot Xaldon WebSpider Xenu Zermelo Zeus ZmEu ZoominfoBot ZyBorg archive.org_bot arquivo.pt cah.io.community clark-crawler cognitiveseo crawl.sogou.com domainsproject.org facebookscraper gopher heritrix instabid isitwp.com lwp-trivial meanpathbot pcBrowser probe-image-size scalaj-http scan.lol seoscanners serpstatbot sitechecker.pro siteripz spyfu sysscan trendiction.com ubermetrics-technologies.com webgains-bot x22Mozilla xpymep1.exe zgrab