80legs ADmantX ASPSeek Abonti Aboundex Aboundexbot Acunetix AfD-Verbotsverfahren AhrefsBot AiHitBot AllSubmitter AlphaBot Anarchie Anthill Asterias BDCbot BDFetch Badass BatchFTP Battleztar Bazinga Bigfoot Bitacle Black Hole BlackWidow Boardreader Bolt Brandprotect Brandwatch Buck Buddy BuzzSumo CCBot CODE87 CSHttp Calculon CazoodleBot Cegbfeieh CensysInspect CherryPicker Chlooe Claritybot Cogentbot Collector CopyRightCheck Cosmos CrazyWebCrawler Curious Custo DIIbot DSearch DTS Agent Deusu Devil Digincore DigitalPebble Disco Discobot DnyzBot DomCopBot DomainCrawler Download Wonder Drip EMail Wolf Evil Extractor ExtractorPro Extreme Picture Finder Firefox/7.0 Freeuploader FrontPage GT::WWW GetWeb Getintent Gigabot GrabNet Grabber HEADMasterSEO HTTrack HaosouSpider Havij HonoluluBot IDBTE4M IRLbot Iblog Indy Library Iskanie IstellaBot JOC Web Spider Jetty Jyxobot Keyword Density LWP::Simple LeechGet LexiBot LibWeb LinkextractorPro LinqiaRSSBot Lipperhey MFC_Tear_Sample MQQBrowser MSFrontPage MSIECrawler MTRobot Mag-Net Magnet Mass Downloader Masscan Mata Hari Mb2345Browser MeanPath Bot Meanpathbot Microsoft URL Control Mojeek Musobot NICErsPRO Nameprotect Navroad NearSite Nessus Net Vampire NetAnts NetSpider NetZIP Nettrack Netvibes NextGenSearchBot Nibbler Niki-bot Nikto Ninja Nmap OnCrawl OpenLinkProfiler OpenVAS OutclicksBot PECL::HTTP PHPCrawl Page Analyzer Papa Foto Picscout PictureFinder Piepmatz Pixray ProPowerBot Pu_iN PxBroker PyCurl QueryN Metasearch RSSingBot RankActive RankActiveLinkBot RankFlex RankingBot RankingBot2 Rankivabot Re-re ReGet RealDownload Reaper RocketCrawler Rogerbot SBIder SEOkicks Scanbot Scrapy SearchmetricsBot Seekport SeoSiteCheckup Shodan Siphon SiteLockSpider SiteSucker Siteimprove SmartDownload Snapbot Sogou web spider Spanner Spinn3r Steeler Suzuran T0PHackTeam T8Abot Teleport TheNomad Thumbor TightTwatBot Tracemyfile Trendiction True_Robot TurnitinBot TwengaBot VCI Vagabondo VelenPublicWebCrawler VidibleScraper WASALive-Bot WBSearchBot WEBDAV WPScan WWW-Collector-E WWW-Mechanize WWWOFFLE Wallpapers WeSEE Web Enhancer Web Fetch Web Sauger WebBandit WebEnhancer WebFetch WebImageCollector WebLeacher WebSauger WebStripper WebWhacker WebZIP WebmasterWorldForumBot Webster Whack Whacker Widow WinHTTrack Wonderbot Wprecon Xaldon WebSpider Xaldon_WebSpider Zauba ZumBot adscanner archive.org_bot arquivo-web-crawler arquivo.pt autoemailspider com.plumanalytics eCatch instabid isitwp.com linkdexbot meanpathbot netEstate NE Crawler page scorer polaris version probe-image-size seostar sysscan webgains-bot webprosbot x09Mozilla xpymep1.exe zauba.io