V4.2024.07.4641 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,100 +1,100 @@
-AIBOT
-Abonti
-AfD-Verbotsverfahren
-AllSubmitter
-BackStreet
-Barkrowler
-Bitacle
-Blow
-BlowFish
+01h4x.com
+360Spider
+ADmantX
+ASPSeek
+AhrefsBot
+Atomseobot
+Buck
+BuiltBotTough
 Bullseye
-CheTeam
-Chlooe
+BunnySlippers
+Calculon
+CazoodleBot
 ClaudeBot
-Cloud mapping
-Collector
-Crawling at Home Project
-CrunchBot
 DTS Agent
-Demon
-DigitalPebble
-Dirbuster
-DnyzBot
-DomCopBot
-ExtLinksBot
-Extractor
-FlashGet
-Fyrebot
-GT::WWW
-Getintent
-GrabNet
-Hloader
-IDBTE4M
-Indy Library
-Internet Ninja
-IstellaBot
-JOC Web Spider
+DatabaseDriverMysqli
+Devil
+Dispatch
+Download Wonder
+EasyDL
+EroCrawler
+FacebookBot
+FemtosearchBot
+Fimap
+FrontPage
+Google-Extended
+GrapeshotCrawler
+HMView
+Humanlinks
+HybridBot
+Image Fetch
+Information Security Team InfraSec Scanner
+Jbrofuzz
+Jyxobot
+LWP::Simple
 Lftp
-Likse
-MSIECrawler
-Mag-Net
-Metauri
-Microsoft URL Control
-MolokaiBot
-Mozlila
-NPbot
-Navroad
-NetLyzer
-NetZIP
-Nettrack
-Nikto
-NimbleCrawler
-Nimbostratus
-Nmap
-Openfind
-PHPCrawl
-PageAnalyzer
-QueryN Metasearch
-Re-re
-Reaper
-Ripper
-Rogerbot
-SEOprofiler
+LieBaoFast
+LinkWalker
+MSFrontPage
+MarkWatch
+Microsoft Data Access
+Mojeek
+Mr.4x3
+Msrabot
+Needle
+Netvibes
+Nibbler
+Offline Explorer
+OnCrawl
+OpenVAS
+Openvas
+PageGrabber
+PageThing.com
+ProWebWalker
+PyCurl
+Quick-Crawler
+RSSingBot
+RankurBot
+RebelMouse
+SEOkicks-Robot
+SEOlyticsCrawler
 ScanAlert
+Scanbot
+ScreenerBot
 Seekport
-SemrushBot
-SeoSiteCheckup
-Seomoz
-SpaceBison
-Sqworm
-SurveyBot
-Telesoft
-Telesphoreo
-Thumbor
-Toata
+Semrush
+SiteExplorer
+SiteSucker
+Sitevigil
+Steeler
+Teleport
+Turnitin
+Typhoeus
+URLy Warning
+VB Project
 Vagabondo
-VeriCiteCrawler
-VidibleScraper
-Wallpapers
-WallpapersHD
-Web Auto
-Web Sauger
-WebEnhancer
-WebImageCollector
-WebmasterWorldForumBot
-Webster
-Whacker
-Whatweb
-Zauba
-arquivo-web-crawler
-autoemailspider
-cognitiveseo
-crawl.sogou.com
-dataforseo.com
-gopher
-heritrix
-netEstate NE Crawler
-probe-image-size
-scalaj-http
-webmeup-crawler
-zgrab
+Voil
+Voltron
+WWW-Mechanize
+WWW::Mechanize
+Web Enhancer
+Web Pix
+WebAuto
+WebStripper
+WebWhacker
+WebsiteQuester
+Widow
+ZyBorg
+anthropic-ai
+clark-crawler
+crawler4j
+dataforseobot
+eCatch
+ips-agent
+oBot
+pcBrowser
+plumanalytics
+scan.lol
+sexsearcher
+sitechecker.pro
+trendiction.com