V4.2024.09.4768 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
 AIBOT
-AfD-Verbotsverfahren
-Asterias
-AwarioRssBot
-Backlink-Ceck
-Badass
-BetaBot
-Blackboard
+Abonti
+AdsTxtCrawlerTP
+AllSubmitter
+Anarchy
+BDFetch
+BacklinksExtendedBot
+Bigfoot
+BotALot
 Buck
-BunnySlippers
+Bytespider
 CATExplorador
 Cegbfeieh
-Chlooe
-Cocolyzebot
-DTS Agent
-DataCha0s
-Devil
-Digincore
+ClaudeBot
+CopyRightCheck
+Crawling at Home Project
+DIIbot
+Demon
+DigitalPebble
 Discoverybot
-DnyzBot
-Domains Project
-Ebingbong
-Extractor
-Fimap
-Foobot
-Genieo
-GermCrawler
-Grafula
-HTMLparser
-HTTP::Lite
-Harvest
-IRLbot
-Image Sucker
-InfoTekies
+DomainAppender
+Dragonfly
+Ecxi
+ExtLinksBot
+Extreme Picture Finder
+Flunky
+FrontPage
+Getintent
+Go!Zilla
+GoZilla
+GridBot
+HEADMasterSEO
+Haansoft
+Heritrix
+ImagesiftBot
+IndeedBot
+InfraSec Scanner
 Internet Ninja
-InternetSeer
-Joomla
-LeechGet
-LinkpadBot
-LinqiaScrapeBot
-Litemage_walker
-MIIxpc
-MQQBrowser
-MarkMonitor
-MegaIndex.ru
-Microsoft Data Access
-Name Intelligence
-Nessus
-Netvibes
-PageThing.com
-Pandalytics
-Panscient
-PleaseCrawl
-PyCurl
-RankFlex
-RedesScrapy
-SISTRIX
+JamesBOT
+Jbrofuzz
+JennyBot
+Keybot Translation-Search-Machine
+Keyword Density
+Libwhisker
+LinkWalker
+Linkbot
+LinkextractorPro
+Lipperhey
+MSIECrawler
+Majestic-SEO
+MarkWatch
+MicroMessenger
+Mojolicious
+MolokaiBot
+NetAnts
+NetLyzer
+Nmap
+Nuclei
+OrangeBot
+POE-Component-Client-HTTP
+Picsearch
+ProPowerBot
+Proximic
+RankActiveLinkBot
+RankingBot2
+RepoMonkey
+Rogerbot
+SBIder
 SalesIntelligent
-Scanbot
-SearchmetricsBot
-SeekportBot
-SeobilityBot
-Seomoz
-Sosospider
-SpankBot
-Spbot
+SemrushBot
+SiteExplorer
+Siteimprove
+Snoopy
 Spinn3r
-Sqlworm
-Szukacz
-T8Abot
-Turnitin
-TurnitinBot
+Telesoft
+Titan
 TwengaBot
-Upflow
-VCI
-Vacuum
-WEBDAV
-WWW-Mechanize
-Web Enhancer
+Vagabondo
+Voil
+WASALive-Bot
+WeSEE
+Web Auto
 WebCollage
-WebGo IS
-WebWhacker
-Whacker
-WinHTTrack
-YaK
-Zitebot
-adscanner
-autoemailspider
-backlink-check
-cohere-ai
-dataforseobot
-evc-batch
-facebookscraper
+WebFuck
+WebmasterWorldForumBot
+WebsiteExtractor
+Whack
+Whatweb
+Widow
+anthropic-ai
+clark-crawler
+cognitiveseo
+crawler.feedback
+crawler4j
+demandbase-bot
 internetVista monitor
-lwp-request
-omgili
+lwp-trivial
 openai
-page scorer
-scalaj-http
-trendiction.de
+sp_auditbot
+zgrab