360Spider Acunetix AiHitBot Aipbot AllSubmitter Anarchie Anarchy Ankit Aspiegel Asterias AwarioSmartBot BDFetch BLEXBot BackStreet Backlink-Ceck BatchFTP BetaBot Black Hole BlackWidow Blow Boardreader Bolt BuiltBotTough BunnySlippers BuzzSumo CATExplorador CSHttp Calculon Chlooe Claritybot Cliqzbot Cloud mapping Collector CopyRightCheck Cosmos Craftbot CrazyWebCrawler DIIbot DSearch DTS Agent DataCha0s Deusu Digincore DigitalPebble Disco Discobot Dispatch DomCopBot DomainAppender DomainCrawler DomainSigmaCrawler Dotbot Download Wonder Drip Ecxi EirGrabber ExtLinksBot ExtractorPro Ezooms Firefox/7.0 FlashGet Flunky Fyrebot G-i-g-a-b-o-t GT::WWW GalaxyBot GetRight Gigabot Go!Zilla Gotit Grafula GrapeFX GrapeshotCrawler GridBot HEADMasterSEO HMView HTMLparser HTTrack Haansoft Havij Heritrix Hloader IDBTE4M Id-search InfoTekies InterGET InternetSeer JamesBOT JennyBot Jetty JustView LNSpiderguy LWP::Simple LeechGet LexiBot Lightspeedsystems Likse LinkScan Linkbot LinkpadBot LinksManager LinqiaMetadataDownloaderBot Lipperhey Lipperhey Spider Litemage_walker Lmspider MFC_Tear_Sample MIIxpc MQQBrowser MSFrontPage MSIECrawler Mag-Net Mail.RU_Bot Majestic SEO Majestic12 Masscan Mata Hari MegaIndex.ru Metauri MicroMessenger Microsoft Data Access Mister PiX Moblie Safari Mojolicious Morfeus Fucking Scanner Musobot NPbot Nameprotect Needle Net Vampire NetAnts Netcraft Nettrack NimbleCrawler Nimbostratus Nutch Offline Explorer OnCrawl OpenLinkProfiler OpenVAS OrangeBot OutfoxBot PECL::HTTP POE-Component-Client-HTTP Page Analyzer Panscient Pavuk Pi-Monster Picscout ProWebWalker Pu_iN Pump QueryN Metasearch Quick-Crawler RankFlex RebelMouse RepoMonkey RocketCrawler Rogerbot SBIder SEOkicks SEOkicks-Robot SEOprofiler Scanbot ScreenerBot Searchestate SemrushBot SentiBot SeoSiteCheckup SeobilityBot Shodan SiteCheckerBotCrawler Sitebeam SmartDownload Snapbot Snoopy Sogou web spider SpaceBison Spbot SputnikBot Sqlmap Stripper Sucuri SurveyBot Swiftbot Szukacz TeleportPro Telesphoreo Telesphorep TheNomad TightTwatBot Tracemyfile Trendictionbot True_Robot Twice URLy Warning Upflow V-BOT Vacuum Virusdie VoidEYE WASALive-Bot WISENutbot WWW-Collector-E WWW::Mechanize WallpapersHD WeSEE Web Collage Web Sauger Web Sucker WebAuto WebFetch WebLeacher WebPix WebReaper WebmasterWorldForumBot Webshag WebsiteExtractor Whacker Whatweb Widow WinHTTrack WiseGuys Robot Woobot Xaldon WebSpider Xenu Zade Zermelo Zitebot ZumBot ZyBorg arquivo-web-crawler check1.exe cognitiveseo demandbase-bot instabid lwp-request meanpathbot mediawords netEstate NE Crawler page scorer plumanalytics seostar serpstatbot sexsearcher sogouspider x09Mozilla xpymep1.exe zauba.io zgrab