V4.2021.09.2622 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
-360Spider
 404enemy
-AIBOT
+ASPSeek
 Abonti
 Aboundex
-Acunetix
-AhrefsBot
 AiHitBot
+AllSubmitter
+Anarchie
 Anarchy
 Anarchy99
-Anthill
-Apexoo
-Asterias
 Attach
+AwarioRssBot
+BBBike
+BDCbot
 BDFetch
-BLEXBot
-BackWeb
-Badass
+BackDoorBot
+Backlink-Ceck
+BacklinkCrawler
 Barkrowler
-Battleztar Bazinga
+BatchFTP
+BetaBot
 Black Hole
 BlackWidow
+Blackboard
+Blow
 BlowFish
-BotALot
-Brandwatch
+Boardreader
+Brandprotect
 Buck
 Buddy
 BuiltBotTough
 BuiltWith
-BunnySlippers
-CazoodleBot
-CherryPicker
+BuzzSumo
+CATExplorador
+Cegbfeieh
+CensysInspect
+ChinaClaw
+Cliqzbot
 Cloud mapping
-Collector
-Cosmos
-Crawling at Home Project
-CrazyWebCrawler
-CyotekWebCopy
+Copier
+Crescent
+CrunchBot
+Curious
+Custo
 DBLBot
 DIIbot
+DSearch
 DTS Agent
 DatabaseDriverMysqli
-Devil
+Deusu
 Digincore
-DigitalPebble
 Dirbuster
-Disco
-Discobot
-Discoverybot
+Dispatch
+DittoSpyder
 DomCopBot
-Download Wonder
-Drip
-ECCP/1.0
-EMail Siphon
-EasyDL
-Ebingbong
-Ecxi
+DomainCrawler
+DomainStatsBot
+Dotbot
+Dragonfly
+EMail Wolf
 EroCrawler
 Exabot
-Ezooms
-FDM
-FemtosearchBot
-FlashGet
+ExtLinksBot
+ExtractorPro
+Fimap
 Flunky
 FyberSpider
-GT::WWW
-GalaxyBot
-GetRight
-GetWeb
-Getintent
-Gigabot
-Gotit
-Grafula
-GridBot
-HTMLparser
-HTTP::Lite
+Genieo
+Go-Ahead-Got-It
+GrapeFX
+HMView
 Haansoft
 HaosouSpider
 Harvest
 Heritrix
-HybridBot
+Hloader
+Humanlinks
+IDBTE4M
 IDBot
-IRLbot
-Id-search
+Iblog
 IlseBot
-IndeedBot
+Indy Library
 InfoNaviRobot
-JamesBOT
-Joomla
-Kozmosbot
-LNSpiderguy
+InterGET
+Jbrofuzz
+Jetty
+JustView
+Jyxobot
+Keyword Density
 Lanshanbot
-Larbin
 Leap
+LeechFTP
+LeechGet
 LexiBot
+Lftp
+LibWeb
 Lightspeedsystems
-Likse
-LinkScan
 LinkWalker
-Lipperhey Spider
+Linkbot
+LinqiaRSSBot
 Litemage_walker
-Ltx71
-MFC_Tear_Sample
+MIDown tool
+MIIxpc
 MJ12bot
 MSIECrawler
-Majestic-SEO
-Majestic12
-Masscan
+Magnet
+Mail.RU_Bot
+MarkMonitor
+MarkWatch
+MauiBot
 Mb2345Browser
-MeanPath Bot
-Metauri
+Meanpathbot
 MicroMessenger
-Microsoft Data Access
+Mister PiX
+Moblie Safari
+Mojeek
+Mojolicious
 Msrabot
-NICErsPRO
-NPbot
+Name Intelligence
 Navroad
 NearSite
-NetAnts
-Netvibes
+Needle
+NetLyzer
+Netcraft
 NextGenSearchBot
+Nibbler
 Niki-bot
-Nimbostratus
-Ninja
+Nikto
+NimbleCrawler
+Nutch
+Offline Navigator
 OnCrawl
-OpenVAS
-PageAnalyzer
-PageGrabber
-PageThing.com
+Openfind
+POE-Component-Client-HTTP
+Page Analyzer
+PageScorer
 Pandalytics
 Pavuk
-PeoplePal
 Pi-Monster
+PictureFinder
+Piepmatz
 Pimonster
-ProPowerBot
-Pu_iN
+ProWebWalker
+Probethenet
+Psbot
+Pump
+PxBroker
+PyCurl
 QueryN Metasearch
-Quick-Crawler
 RankActive
-RankActiveLinkBot
-RankingBot
-RankingBot2
-Rankivabot
 RankurBot
-ReGet
-Recorder
-RepoMonkey
+RebelMouse
 Ripper
-SEOlyticsCrawler
+RocketCrawler
+Rogerbot
+SEOkicks
+SEOkicks-Robot
 SEOprofiler
 Scanbot
+Scrapy
 Screaming
 ScrepyBot
-SentiBot
 SeobilityBot
-Seomoz
-Shodan
+Siphon
 Site Sucker
-SiteCheckerBotCrawler
-SiteExplorer
-SiteSnagger
 SiteSucker
 Sitebeam
-Siteimprove
 SlySearch
-SmartDownload
-Snapbot
 Snoopy
-SocialRankIOBot
 Sociscraper
 Sottopop
+SpaceBison
 Spammen
 Spanner
-Spinn3r
-SputnikBot
-Sqlmap
+Spbot
 Sqlworm
+Sqworm
 Steeler
 Stripper
-SuperBot
+Sucker
+Sucuri
 SuperHTTP
 Suzuran
-Szukacz
 T0PHackTeam
-Teleport
 The Intraformant
+TheNomad
+TightTwatBot
 Trendiction
-Turingos
+TwengaBot
 Twice
-Typhoeus
-URLy.Warning
 UnisterBot
 Upflow
-Vagabondo
-VelenPublicWebCrawler
-VeriCiteCrawler
-VidibleScraper
-Virusdie
+VCI
 Voil
-WEBDAV
-WWW-Mechanize
+Voltron
+WBSearchBot
+WWWOFFLE
 Wallpapers
 Wallpapers/3.0
-WallpapersHD
 Web Collage
-Web Enhancer
-Web Pix
-WebCopier
-WebFuck
+Web Fuck
+Web Sucker
 WebPix
-WebReaper
+WebSauger
 WebSucker
 WebWhacker
 WebZIP
-Webshag
+Webalta
+WebmasterWorldForumBot
+Website Quester
 WebsiteExtractor
-Whacker
+Whatweb
+Who.is Bot
+Widow
 Woobot
-Wprecon
+Xaldon WebSpider
+Xaldon_WebSpider
+YoudaoBot
 Zauba
 ZumBot
 ZyBorg
+archive.org_bot
+autoemailspider
 backlink-check
-check1.exe
+cah.io.community
+clark-crawler
+cognitiveseo
 crawl.sogou.com
+crawler.feedback
 dataforseo.com
 domainsproject.org
 eCatch
-facebookscraper
-gopher
+evc-batch
+instabid
 ips-agent
 isitwp.com
 lwp-request
-mediawords
-oBot
+lwp-trivial
+muhstik-scan
 page scorer
-polaris version
-scan.lol
-seostar
-serpstatbot
-sitechecker.pro
+plumanalytics
+probe-image-size
+scalaj-http
+seobility
 sp_auditbot
-spyfu
 sysscan
 trendiction.com
-ubermetrics-technologies.com
-voyagerx.com
 webpros.com
-x09Mozilla
+x22Mozilla
 xpymep1.exe
+zauba.io