V4.2024.10.4837 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-404enemy
-AIBOT
-Abonti
-AfD-Verbotsverfahren
-Atomseobot
-AwarioSmartBot
-BDFetch
-BLEXBot
-BackDoorBot
-Backlink-Ceck
-Badass
-Blackboard
-Blow
+ADmantX
+ASPSeek
+AhrefsBot
+Alligator
+Bandit
+BatchFTP
+Battleztar Bazinga
+BetaBot
+Bigfoot
+BlowFish
 BuiltWith
 CATExplorador
-CensysInspect
-CheTeam
+CODE87
+CSHttp
 CheeseBot
-Cocolyzebot
-Copyscape
-Crawling at Home Project
-Crescent
-Curious
-Deusu
-Digincore
+ClaudeBot
+Craftbot
+CyotekWebCopy
+Devil
+Disco
 DomainCrawler
+EMail Siphon
+EroCrawler
 FHscan
-FemtosearchBot
-FrontPage
+Flunky
+Foobot
 FyberSpider
-G-i-g-a-b-o-t
 GPTBot
-GetWeb
-GridBot
-HTTrack
-Humanlinks
-IRLbot
-Iblog
+HMView
+Id-search
+InfoNaviRobot
+Internet Ninja
 JOC Web Spider
-JustView
-Jyxobot
-LNSpiderguy
-LWP::Simple
-Ltx71
-Masscan
-Morfeus Fucking Scanner
-Mr.4x3
-Net Vampire
-Nikto
-NimbleCrawler
-Offline Explorer
-OrangeSpider
-OutclicksBot
-Pandalytics
-Pi-Monster
-ProWebWalker
-Pu_iN
-PxBroker
-PyCurl
-Rankivabot
+JennyBot
+MFC_Tear_Sample
+MJ12bot
+Mail.RU_Bot
+MeanPath Bot
+Meanpathbot
+MegaIndex.ru
+MolokaiBot
+Mozlila
+NPbot
+NetSpider
+NextGenSearchBot
+Nimbostratus
+Octopus
+Picscout
+Pixray
+Probethenet
+Pump
+RankActiveLinkBot
 RankurBot
-SMTBot
-ScoutJet
-Scrapy
-SeobilityBot
+Re-re
+Reaper
+SBIder
+SEOkicks-Robot
+ScanAlert
+Screaming
+Searchestate
+SemanticJuice
+Semrush
+SenutoBot
+Siphon
 SiteExplorer
-Sosospider
-SpaceBison
-Spinn3r
-SuperBot
-T8Abot
-Thumbor
-TinyTestBot
+Sociscraper
+Steeler
+Suzuran
+Szukacz
+Teleport
+TeleportPro
+TheNomad
+Tracemyfile
+Trendiction
+True_Robot
+Turnitin
+TurnitinBot
+Voil
+WISENutbot
+WWW::Mechanize
+Wallpapers
+WallpapersHD
 Web Fuck
-WebCopier
-WebLeacher
-WebSauger
-WebZIP
-Whack
-Whatweb
+Web Sauger
+WebPix
+Webalta
+WebsiteExtractor
+Who.is Bot
 Widow
-Wotbox
-Xaldon WebSpider
-Xenu
-YoudaoBot
-Zeus
-anthropic-ai
-archive.org_bot
-instabid
-ips-agent
-isitwp.com
-lwp-request
-omgili
-s1z.ru
-satoristudio.net
-seocompany.store
-sogouspider
-trendiction.de
-webgains-bot
-webpros.com
+Xaldon_WebSpider
+Zauba
+ZyBorg
+crawler.feedback
+dataforseo.com
+demandbase-bot
+muhstik-scan
+polaris version
+ubermetrics-technologies.com
+webprosbot
+zgrab