V4.2024.10.4811 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-AwarioRssBot
+ASPSeek
+Acunetix
+AfD-Verbotsverfahren
+AlphaBot
+Apexoo
 BBBike
-BDCbot
-BackStreet
-Bigfoot
-BlackWidow
-Blow
-BunnySlippers
-CODE87
-CensysInspect
-Copyscape
-Crawling at Home Project
-Curious
-CyotekWebCopy
+BatchFTP
+Bolt
+ChatGPT-User
+Copier
 DBLBot
 Demon
-Devil
-Digincore
-Discobot
-DomCopBot
-DomainSigmaCrawler
-Drip
+Deusu
+Dispatch
 ECCP/1.0
-Ecxi
-Evil
-Fimap
-GPTBot
-Gigabot
-Go!Zilla
-Grafula
-Haansoft
-IRLbot
+Extractor
+Extreme Picture Finder
+FemtosearchBot
+FrontPage
+Getintent
+Go-Ahead-Got-It
+Grabber
+GridBot
+HEADMasterSEO
+HTMLparser
+HybridBot
+IDBot
 Iblog
-IlseBot
-InfraSec Scanner
-Intelliseek
-JikeSpider
-Jorgee
-LWP::Simple
+Internet Ninja
+JennyBot
+JetCar
+Joomla
+Keybot Translation-Search-Machine
+Lanshanbot
 LeechFTP
-Libwhisker
-MTRobot
+LibWeb
 Masscan
-MegaIndex.ru
-Mister PiX
-Morfeus Fucking Scanner
-Mr.4x3
-Nessus
+MolokaiBot
+Net Vampire
+NetAnts
 NetSpider
-NextGenSearchBot
-Nimbostratus
-Offline Navigator
-OrangeSpider
-PECL::HTTP
-Papa Foto
-Picsearch
+Openvas
+PageThing.com
+Petalbot
+Picscout
+PictureFinder
 PleaseCrawl
-Pockey
 Proximic
+Psbot
+Pump
+QueryN Metasearch
 Quick-Crawler
-RankActiveLinkBot
-RankingBot2
-Ripper
-Scanbot
-ScoutJet
-SeoSiteCheckup
-Shodan
-SiteLockSpider
-Sottopop
-T0PHackTeam
+Rainbot
+Rogerbot
+SEOkicks-Robot
+SEOlyt
+SISTRIX
+SalesIntelligent
+ScanAlert
+SemanticJuice
+Semrush
+SenutoBot
+SiteSucker
+Siteimprove
+SocialRankIOBot
+Spinn3r
+Sqlmap
+Steeler
 TeleportPro
-Telesphorep
-Toata
-Trendiction
-Turnitin
-WASALive-Bot
+TightTwatBot
+URLy Warning
+UnisterBot
 WEBDAV
 Wallpapers/3.0
-WallpapersHD
-Web Sucker
-WebBandit
-WebGo IS
-Whacker
+Web Collage
+WebCollage
+WebCopier
+WebStripper
+WebWhacker
+WebmasterWorldForumBot
+Webshag
+Whack
 WinHTTrack
-Wonderbot
-crawler.feedback
-facebookscraper
-instabid
-magpie-crawler
-meanpathbot
-openai
-openai.com
-plumanalytics
-satoristudio.net
+Wotbox
+Xaldon_WebSpider
+arquivo-web-crawler
+awario.com
+cah.io.community
+linkfluence
+probe-image-size
+s1z.ru
 scalaj-http
-seocompany.store
-sogouspider
+seobility
+spyfu
+sysscan
 trendiction.com
+trendiction.de
+webmeup-crawler
 webpros.com
-webprosbot
+zgrab