V4.2021.08.2507 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
-01h4x.com
 360Spider
-404enemy
 ADmantX
+ALittle Client
 ASPSeek
 Abonti
-AfD-Verbotsverfahren
-AiHitBot
+Acunetix
+AhrefsBot
 AllSubmitter
+AlphaBot
+Anarchie
 Anarchy99
 Anthill
 Apexoo
-Aspiegel
 Asterias
-Attach
-AwarioRssBot
-BDCbot
-BacklinkCrawler
+AwarioSmartBot
+BLEXBot
+BackDoorBot
+BackWeb
 Badass
-BatchFTP
+Bandit
+Barkrowler
+BetaBot
+Bigfoot
 Bitacle
 Black Hole
-BlackWidow
+Blackboard
+Blow
 BlowFish
 Bolt
-BotALot
-Buck
-Buddy
-BuiltWith
-BunnySlippers
-CATExplorador
+Brandprotect
+Brandwatch
+Bullseye
 CCBot
 CODE87
-CSHttp
+CazoodleBot
 Cegbfeieh
-CensysInspect
-CherryPicker
 Chlooe
-Cloud mapping
-Copier
+Cogentbot
+Copyscape
 CrazyWebCrawler
-CrunchBot
-Curious
-CyotekWebCopy
-DTS Agent
-Demon
-Dirbuster
-Disco
-Discoverybot
+DSearch
+DataCha0s
+DatabaseDriverMysqli
+DigitalPebble
+Discobot
+DittoSpyder
 DnyzBot
 DomainCrawler
 DomainSigmaCrawler
+DomainStatsBot
 Domains Project
+Dotbot
+Download Wonder
 Drip
-ECCP/1.0
+EasyDL
+Ecxi
 EroCrawler
 Evil
 Express WebPictures
 ExtLinksBot
+Extractor
 EyeNetIE
-Ezooms
 FDM
+Fimap
+Firefox/7.0
 Flunky
+Foobot
 Freeuploader
 FrontPage
 Fuzz
-GT::WWW
+GalaxyBot
+Gigabot
 Go!Zilla
-Go-Ahead-Got-It
+GoZilla
+Gotit
 Grafula
+GrapeshotCrawler
 HEADMasterSEO
+HTTrack
 HaosouSpider
 Heritrix
 Hloader
-IDBTE4M
-IRLbot
+IDBot
 Iblog
-Image Sucker
-IndeedBot
-InfoNaviRobot
+Id-search
+IlseBot
 InfoTekies
 Intelliseek
-InterGET
-Internet Ninja
+InternetSeer
 JamesBOT
-Jbrofuzz
-Jetty
-Jyxobot
+JennyBot
+JetCar
+Joomla
+Jorgee
 Kenjin Spider
+Keyword Density
+Kozmosbot
 LNSpiderguy
 LWP::Simple
-LeechFTP
+Larbin
 LeechGet
 LexiBot
-LieBaoFast
+Lftp
 Lightspeedsystems
-LinkScan
+Likse
 LinkWalker
-Linkbot
-LinkpadBot
-LinqiaMetadataDownloaderBot
+Linkdexbot
+LinkextractorPro
+LinksManager
 LinqiaRSSBot
-Lipperhey
+Lipperhey Spider
 Litemage_walker
-MSIECrawler
-MTRobot
+MFC_Tear_Sample
+MIDown tool
+MIIxpc
 Mag-Net
-Majestic-SEO
-MarkWatch
-Masscan
+Majestic SEO
 Mata Hari
-Meanpathbot
+MeanPath Bot
+Mediatoolkitbot
 Metauri
-MicroMessenger
+Microsoft Data Access
 Microsoft URL Control
 Mister PiX
-Mojolicious
+Mojeek
 Morfeus Fucking Scanner
-Mozlila
-Mr.4x3
+Msrabot
 Musobot
-NPbot
-Navroad
-Nessus
-NetAnts
+Nameprotect
+Net Vampire
 NetLyzer
-NetMechanic
-NetSpider
 NetZIP
+Netvibes
 Nibbler
-Niki-bot
-NimbleCrawler
-Nmap
+Nikto
 Nutch
-Openvas
-OutfoxBot
-PECL::HTTP
-Page Analyzer
-PageAnalyzer
-PeoplePal
+Openfind
+OrangeBot
+PageScorer
+PageThing.com
+Pavuk
 Petalbot
-Pi-Monster
-Picsearch
-PictureFinder
-Pimonster
-Pixray
+Pu_iN
 Pump
-PyCurl
-QueryN Metasearch
-RankActiveLinkBot
-RankFlex
+RankActive
 RankingBot
-RankingBot2
-RankurBot
+Rankivabot
 Re-re
 ReGet
+RealDownload
 Reaper
-Recorder
 RepoMonkey
-RocketCrawler
+Ripper
 Rogerbot
-SEOkicks-Robot
-SEOprofiler
-SEOstats
-SISTRIX
 SMTBot
 SalesIntelligent
-ScreenerBot
-ScrepyBot
-Searchestate
-SemrushBot
+ScanAlert
+ScoutJet
+Scrapy
+Screaming
+Semrush
+SentiBot
 SeoSiteCheckup
-Sitebeam
+Site Sucker
+SiteCheckerBotCrawler
+SiteSnagger
+SlySearch
 SmartDownload
+Snake
+Snoopy
+Sociscraper
+SpaceBison
 SpankBot
-Spbot
-SuperHTTP
+Sqlmap
+Steeler
+SuperBot
 SurveyBot
-Szukacz
-T0PHackTeam
-T8Abot
-Telesoft
-Telesphoreo
+TeleportPro
 The Intraformant
 Thumbor
-Toata
+Tracemyfile
 Trendiction
-Trendictionbot
-Turnitin
+True_Robot
+Turingos
 TurnitinBot
+TwengaBot
 Twice
-Typhoeus
-URLy.Warning
 UnisterBot
-V-BOT
-VB Project
 VCI
 Vacuum
+Vagabondo
 VelenPublicWebCrawler
+VoidEYE
 Voil
-WEBDAV
+Voltron
+WBSearchBot
+WISENutbot
+WPScan
 WWW-Collector-E
 WWW-Mechanize
-WWW::Mechanize
 WWWOFFLE
-Wallpapers
-WeSEE
+Wallpapers/3.0
+Web Auto
 Web Enhancer
-Web Fuck
-Web Pix
-WebBandit
+WebAuto
 WebCollage
+WebCopier
 WebEnhancer
 WebFetch
-WebFuck
+WebGo IS
 WebLeacher
-Webalta
-WebmasterWorldForumBot
-Webshag
+WebReaper
+WebSauger
+WebWhacker
 Website Quester
-WebsiteExtractor
 Whack
-Whacker
-Whatweb
-Who.is Bot
 WinHTTrack
-Woobot
-Xaldon_WebSpider
-YoudaoBot
-ZoomBot
-ZyBorg
-crawler4j
+Wotbox
+Wprecon
+Xaldon WebSpider
+Zermelo
+Zitebot
+ZmEu
+archive.org_bot
+arquivo.pt
+coccocbot-web
 demandbase-bot
 eCatch
 evc-batch
 facebookscraper
-gopher
 heritrix
-mediawords
-muhstik-scan
-netEstate NE Crawler
+instabid
+ips-agent
+isitwp.com
+lwp-request
+lwp-trivial
+magpie-crawler
+oBot
+page scorer
 pcBrowser
-seoscanners
-siteripz
-spyfu
-tAkeOut
-trendiction.com
+scan.lol
+seobility
+seostar
+serpstatbot
+sexsearcher
+sogouspider
+sysscan
 trendiction.de
-voyagerx.com
-webmeup-crawler
+x22Mozilla
+zgrab