V4.2024.05.4534 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-Abonti
+01h4x.com
 Aboundexbot
 AwarioRssBot
-BackStreet
+BLEXBot
+Backlink-Ceck
+BatchFTP
 Bitacle
-Blow
-BuiltBotTough
-CherryPicker
-CyotekWebCopy
+Blackboard
+BlowFish
+BotALot
+Buck
+BuiltWith
+Calculon
+Copier
+Crawling at Home Project
+Custo
+DSearch
 DTS Agent
-Demon
-DigitalPebble
-Dragonfly
-EirGrabber
-EroCrawler
-Express WebPictures
-FacebookBot
-FlashGet
+ECCP/1.0
+Fuzz
+G-i-g-a-b-o-t
 GT::WWW
-Grabber
-GridBot
-HTTP::Lite
-HTTrack
-Heritrix
-Indy Library
-InfoTekies
-Internet Ninja
-InternetMeasurement
-JamesBOT
-Jetty
-Jorgee
-Kinza
+GrapeshotCrawler
+HaosouSpider
+Hloader
+HonoluluBot
+LNSpiderguy
+Larbin
 LeechFTP
-Libwhisker
-LinksManager
-Lmspider
-MJ12bot
-Mag-Net
-MarkWatch
-Mata Hari
-Mediatoolkitbot
-Morfeus Fucking Scanner
-Mr.4x3
-Msrabot
-Musobot
-Nessus
-Net Vampire
+LibWeb
+LieBaoFast
+LinkScan
+Lipperhey Spider
+Ltx71
+MIIxpc
+MSIECrawler
+MauiBot
+Minefield
+Moblie Safari
 NetLyzer
-Octopus
-OutclicksBot
-PageGrabber
-Picsearch
+Netcraft
+Nimbostratus
+Nutch
+Offline Explorer
+PHPCrawl
+Pandalytics
+Petalbot
 ProPowerBot
-ProWebWalker
-Probethenet
-Pump
-Rankivabot
-Recorder
-SBIder
-SMTBot
-Semrush
-SiteExplorer
-SiteSucker
-Siteimprove
-SlySearch
+RankingBot
+RankingBot2
+Re-re
+RebelMouse
+RedesScrapy
+RocketCrawler
+SalesIntelligent
+Seomoz
+SiteCheckerBotCrawler
+Sitebeam
+SmartDownload
 Snoopy
-SpaceBison
-Sqlworm
-Sucker
-Sucuri
+Sociscraper
+Sottopop
+Spammen
+SpankBot
+Spanner
 SuperHTTP
 Suzuran
-The Intraformant
-TightTwatBot
-Turingos
-Typhoeus
-VB Project
-VidibleScraper
-WWW-Collector-E
-WWW::Mechanize
-Web Pix
-WebEnhancer
-WebGo IS
-Webster
-Xenu
+Telesoft
+TinyTestBot
+VCI
+VelenPublicWebCrawler
+WBSearchBot
+Web Collage
+WebBandit
+WebPix
+Webshag
+WebsiteExtractor
+Widow
+WinHTTrack
+Wprecon
+Xaldon WebSpider
+Xaldon_WebSpider
+YaK
 YoudaoBot
-ZoominfoBot
-ZyBorg
-anthropic-ai
-isitwp.com
+ZmEu
+ZoomBot
+arquivo.pt
+evc-batch
+heritrix
+instabid
+internetVista monitor
+ips-agent
+lwp-trivial
+meanpathbot
 mediawords
-omgili
-plumanalytics
-polaris version
+openai.com
+probe-image-size
 scalaj-http
-sysscan
-trendiction.de
-voyagerx.com
-webprosbot
-zauba.io
+sitechecker.pro
+tAkeOut