V4.2023.04.3676 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-404checker
-Ankit
-Atomseobot
-AwarioSmartBot
-BBBike
-Bigfoot
-Brandwatch
-CODE87
-Calculon
-CensysInspect
+ADmantX
+Alexibot
+BackWeb
+Badass
+Bandit
+Black Hole
+Blow
+CherryPicker
 Chlooe
-Claritybot
-Cliqzbot
-Cocolyzebot
-Copier
-Craftbot
+Cloud mapping
+Cogentbot
+Collector
+CopyRightCheck
+Copyscape
+Crawling at Home Project
+Curious
 Custo
-Digincore
-Extractor
-EyeNetIE
-FHscan
-Fimap
+DTS Agent
+Deusu
+DigitalPebble
+DomCopBot
+Evil
+FlashGet
 FrontPage
-Fuzz
-GT::WWW
-Go-Ahead-Got-It
-GoZilla
 GrabNet
-Havij
-IndeedBot
-Intelliseek
-InterGET
-JamesBOT
+Grafula
+Heritrix
+HonoluluBot
+HybridBot
+Indy Library
+InfoNaviRobot
+JOC Web Spider
 JennyBot
-JustView
-Jyxobot
-Larbin
-LexiBot
-Lightspeedsystems
-LinksManager
-LinqiaMetadataDownloaderBot
+Jorgee
+Keyword Density
+Kinza
+Kozmosbot
+Lftp
+LieBaoFast
+Likse
+Litemage_walker
+MSFrontPage
 Majestic-SEO
 Mass Downloader
-Mata Hari
-Microsoft Data Access
-Morfeus Fucking Scanner
-NearSite
-Ninja
-Not
-OnCrawl
-OrangeSpider
+MicroMessenger
+Minefield
+MolokaiBot
+Mr.4x3
+Navroad
+Netvibes
+NextGenSearchBot
+Nibbler
+Openvas
 OutfoxBot
-Papa Foto
-Pump
+PHPCrawl
+Page Analyzer
+Pandalytics
+Pavuk
+PictureFinder
 QueryN Metasearch
-ReGet
-RebelMouse
-RepoMonkey
-Ripper
+Reaper
+Recorder
+SBIder
 SEOkicks
-Scanbot
-Scrapy
-ScreenerBot
-Searchestate
-Seekport
-SemrushBot
-Shodan
-Site Sucker
-SiteExplorer
-SlySearch
+SEOstats
+Screaming
+Snake
+Sogou web spider
+Sosospider
+Spammen
+Sqlmap
 Surfbot
-TheNomad
-Toweyabot
-Tracemyfile
-Turnitin
-URLy Warning
-VelenPublicWebCrawler
-WASALive-Bot
-WWW-Collector-E
-Wallpapers
-Web Pix
-Web Sauger
-WebAuto
-WebFuck
-WebSucker
-WebZIP
+Teleport
+Titan
+Trendictionbot
+True_Robot
+TurnitinBot
+WEBDAV
+WWW-Mechanize
+WebFetch
+WebLeacher
+Website Quester
 WebsiteQuester
 Widow
 WiseGuys Robot
+Xaldon WebSpider
+YoudaoBot
+ZumBot
+ZyBorg
+adscanner
+archive.org_bot
 backlink-check
-evc-batch
+dataforseo.com
+internetVista monitor
+lwp-request
 meanpathbot
-mediawords
-muhstik-scan
 oBot
 page scorer
-satoristudio.net
-scalaj-http
-webmeup-crawler
-x09Mozilla
+pcBrowser
+x22Mozilla