V4.2023.10.4028 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-AIBOT
-Alexibot
-AlphaBot
-Anarchy99
-Atomseobot
-BLEXBot
-Bandit
-Bigfoot
-Black Hole
-BotALot
-Brandwatch
+01h4x.com
+Alligator
+Anthill
+Attach
+AwarioSmartBot
+BacklinkCrawler
+BatchFTP
+Blow
+Boardreader
+Bullseye
+BuzzSumo
+Calculon
 CensysInspect
-CheTeam
-Citoid
-Crawling at Home Project
-Crescent
-CrunchBot
+Claritybot
+Cliqzbot
+Cogentbot
+Collector
+Copier
 Curious
 DataCha0s
-Dirbuster
+Deusu
+Devil
 DittoSpyder
-Dotbot
-Extractor
-FDM
-FemtosearchBot
-Firefox/7.0
-FlashGet
-G-i-g-a-b-o-t
-Genieo
-GetWeb
-GrapeshotCrawler
+Freeuploader
+Fyrebot
+GalaxyBot
+Go-Ahead-Got-It
+GrapeFX
+HMView
 Harvest
-HonoluluBot
-IlseBot
-Image Fetch
-Image Sucker
-InternetSeer
+Havij
+IDBot
+Internet Ninja
 Iskanie
-Jbrofuzz
-Jorgee
-Kozmosbot
-LNSpiderguy
-LibWeb
-Lipperhey Spider
-MIIxpc
-Mb2345Browser
-Microsoft Data Access
-Navroad
-NearSite
-Net Vampire
-PHPCrawl
+IstellaBot
+JustView
+Kinza
+Leap
+LeechGet
+LexiBot
+Ltx71
+MSIECrawler
+Majestic12
+MicroMessenger
+Mojolicious
+Morfeus Fucking Scanner
+NICErsPRO
+Netvibes
+Nikto
+OutclicksBot
 PageScorer
-Pandalytics
-Psbot
-RSSingBot
-RankActive
+Pimonster
+PleaseCrawl
+Quick-Crawler
 RankFlex
-ReGet
-SEOlyticsCrawler
-SEOprofiler
-ScoutJet
-SiteLockSpider
-Siteimprove
-SpankBot
-Sqlworm
-Sqworm
-SuperBot
-Telesoft
-Telesphoreo
-Thumbor
-Titan
-Tracemyfile
+Re-re
+RepoMonkey
+Scanbot
+Screaming
+SearchmetricsBot
+Shodan
+Siphon
+Snake
+Sogou web spider
+SuperHTTP
+Suzuran
+Toata
+Toweyabot
 Trendiction
-VidibleScraper
-WWW::Mechanize
-Web Collage
-Web Enhancer
-WebBandit
+Typhoeus
+VCI
+WWWOFFLE
+Wallpapers
+WallpapersHD
+Web Auto
+Web Pix
+WebAuto
+WebCollage
 WebImageCollector
-WebSucker
-Webster
-Widow
-ZmEu
-ZyBorg
-archive.org_bot
-crawler.feedback
-dataforseo.com
-evc-batch
-lwp-request
+Whatweb
+autoemailspider
+dataforseobot
+facebookscraper
+gopher
+internetVista monitor
+isitwp.com
+lwp-trivial
 magpie-crawler
-meanpathbot
-openai
-page scorer
-pcBrowser
-satoristudio.net
+muhstik-scan
+openai.com
+ripz
 scalaj-http
 scan.lol
-seocompany.store
+seobility
+seostar
 serpstatbot
-siteripz
+sitechecker.pro
+spyfu
+webgains-bot
+zgrab