80legs Aboundex AfD-Verbotsverfahren Anthill BacklinkCrawler BlowFish Cloud mapping Copier CopyRightCheck Crescent CyotekWebCopy Digincore DomCopBot DomainAppender ECCP/1.0 Ecxi EirGrabber Extractor FHscan Fimap Firefox/7.0 Freeuploader FyberSpider Go-Ahead-Got-It Heritrix IlseBot IndeedBot Joomla Kinza LWP::Simple Lanshanbot Leap Likse LinkScan LinqiaMetadataDownloaderBot Lipperhey MIDown tool MQQBrowser Mail.RU_Bot Majestic12 MarkWatch Metauri Microsoft URL Control Mojeek Mr.4x3 NPbot Nameprotect NimbleCrawler Offline Navigator Openfind OrangeBot PECL::HTTP Picsearch Piepmatz PleaseCrawl Psbot RankingBot2 RepoMonkey SBIder SEOkicks-Robot SalesIntelligent ScoutJet SearchmetricsBot SemanticJuice Semrush SeoSiteCheckup SeobilityBot Spanner Suzuran TightTwatBot Titan Twice Typhoeus UnisterBot Vagabondo VeriCiteCrawler VidibleScraper Virusdie WWW-Collector-E WWW-Mechanize Web Fuck WebAuto WebBandit WebGo IS WebLeacher Webalta WebsiteExtractor arquivo-web-crawler com.plumanalytics domainsproject.org linkdexbot plumanalytics polaris version ripz tAkeOut trendiction.com trendiction.de webmeup-crawler x09Mozilla zauba.io