V4.2022.08.3279 [ci skip]

dev-tools/test_units/random-bots-for-whitelist-test.list

@@ -1,250 +1,250 @@
 01h4x.com
-360Spider
 404checker
+404enemy
 AIBOT
 ASPSeek
 Aboundex
 Aboundexbot
 Acunetix
-AhrefsBot
 AiHitBot
-AllSubmitter
 Alligator
-Ankit
-Apexoo
-Attach
+Anarchy
+Anarchy99
+Anthill
+Asterias
+AwarioRssBot
 AwarioSmartBot
 BDFetch
-BLEXBot
-BackDoorBot
-BackWeb
 Backlink-Ceck
-BacklinkCrawler
-Barkrowler
-BatchFTP
+Bandit
+Battleztar Bazinga
 Bigfoot
-BlackWidow
+Black Hole
 Blackboard
-Blow
 BlowFish
 Boardreader
 Brandprotect
-Brandwatch
-Buddy
-Bullseye
-BunnySlippers
 BuzzSumo
-CATExplorador
-CCBot
+CODE87
 CSHttp
 Cegbfeieh
-CherryPicker
+CensysInspect
 ChinaClaw
-Citoid
 Claritybot
-Cliqzbot
-Collector
-Copier
-Craftbot
+Cloud mapping
+Cogentbot
+CopyRightCheck
+Copyscape
+Cosmos
 Crawling at Home Project
 Custo
-DSearch
-DTS Agent
-DataCha0s
-DatabaseDriverMysqli
-Demon
-Deusu
-Dirbuster
+DBLBot
+Devil
 Disco
-Dispatch
-DomCopBot
-DomainStatsBot
-Domains Project
+DittoSpyder
+DnBCrawler-Analytics
+DomainCrawler
 Dotbot
-Download Wonder
 Dragonfly
+ECCP/1.0
+EMail Siphon
 EasyDL
-Ebingbong
 EroCrawler
-Evil
 Exabot
+Express WebPictures
+ExtLinksBot
 Extractor
+ExtractorPro
 Extreme Picture Finder
-Foobot
+EyeNetIE
+Ezooms
+FHscan
+FemtosearchBot
+Firefox/7.0
+FlashGet
 FyberSpider
-G-i-g-a-b-o-t
+Fyrebot
 GT::WWW
-GalaxyBot
-Genieo
-GermCrawler
-Getintent
-Go!Zilla
-Go-Ahead-Got-It
-GrapeshotCrawler
-HTMLparser
+GrabNet
+Grabber
+HEADMasterSEO
 Haansoft
-HaosouSpider
 Heritrix
-HonoluluBot
+Humanlinks
+IDBTE4M
 IRLbot
+IlseBot
 InfoNaviRobot
-InfoTekies
-Intelliseek
-Iria
+InterGET
 Iskanie
+IstellaBot
+JOC Web Spider
+JamesBOT
 Jbrofuzz
 JetCar
+JikeSpider
 Jyxobot
 Kenjin Spider
+Keybot Translation-Search-Machine
+Keyword Density
 Kinza
 Lanshanbot
 Larbin
 LeechFTP
 LeechGet
-Libwhisker
-LieBaoFast
+LexiBot
+Lightspeedsystems
 LinkWalker
-LinkextractorPro
+Linkbot
+LinksManager
+LinqiaMetadataDownloaderBot
 LinqiaRSSBot
-Ltx71
+Litemage_walker
 MFC_Tear_Sample
-MIIxpc
-Majestic12
-MarkMonitor
-Mass Downloader
+MIDown tool
+MQQBrowser
+MSFrontPage
+Mag-Net
+MarkWatch
 MauiBot
-Mediatoolkitbot
+MegaIndex.ru
 MicroMessenger
-Microsoft URL Control
+Microsoft Data Access
 Mister PiX
 Moblie Safari
-Mr.4x3
-Musobot
+Mojeek
+Mojolicious
+Morfeus Fucking Scanner
 NICErsPRO
+NPbot
 NearSite
 Net Vampire
 NetAnts
-NetSpider
-Netcraft
-Nettrack
+NetLyzer
+NetMechanic
+Netvibes
 Nibbler
-Niki-bot
-Nikto
-NimbleCrawler
 Ninja
-Nmap
-Not
-Nutch
-Octopus
+Nuclei
 Offline Navigator
-OpenVAS
+OpenLinkProfiler
 Openfind
-OutfoxBot
-PHPCrawl
-POE-Component-Client-HTTP
-PageAnalyzer
-PageScorer
-PageThing.com
-Pandalytics
-Pavuk
-PeoplePal
+Openvas
+OrangeSpider
+OutclicksBot
+PECL::HTTP
+Page Analyzer
+Panscient
+Pi-Monster
 PictureFinder
-Piepmatz
-Pixray
-Pockey
-ProPowerBot
-Probethenet
-PxBroker
+PleaseCrawl
+ProWebWalker
+Pump
+PyCurl
+QueryN Metasearch
 RSSingBot
-RankActive
+RankActiveLinkBot
 RankingBot2
+Rankivabot
+RankurBot
 Re-re
-ReGet
-RealDownload
-Reaper
-RedesScrapy
-RepoMonkey
-Ripper
-SBIder
+RebelMouse
 SEOkicks
-SEOlyticsCrawler
-Screaming
+SEOkicks-Robot
+Scanbot
 ScreenerBot
-Searchestate
-SemanticJuice
-Semrush
-SeobilityBot
-Seomoz
-Shodan
+SearchmetricsBot
 Site Sucker
-SiteCheckerBotCrawler
 SiteSnagger
-SlySearch
+Siteimprove
+Snake
 Snoopy
-Spammen
-SpankBot
-Sqlmap
+SocialRankIOBot
+SpaceBison
+Spanner
 Sqlworm
 Sucuri
-SuperHTTP
+SurveyBot
 Suzuran
-TheNomad
+T0PHackTeam
+Teleport
+The Intraformant
 Thumbor
-Trendiction
+TightTwatBot
+Titan
+Toata
+Toweyabot
 True_Robot
-Turingos
-URLy Warning
+Typhoeus
 URLy.Warning
-UnisterBot
-Upflow
-VidibleScraper
-Voltron
-WBSearchBot
+VCI
+VelenPublicWebCrawler
+Voil
+WISENutbot
+WWW-Collector-E
+WWW-Mechanize
 WWW::Mechanize
+WWWOFFLE
 Wallpapers/3.0
 WallpapersHD
-Web Fuck
+WeSEE
+Web Auto
+Web Collage
 Web Sauger
 Web Sucker
-WebAuto
+WebBandit
+WebCollage
 WebEnhancer
+WebFetch
+WebFuck
+WebGo IS
 WebImageCollector
-WebReaper
-WebsiteExtractor
+WebPix
+WebStripper
+WebSucker
+WebZIP
+Webalta
+Webshag
+WebsiteQuester
 Webster
 Whack
+Whacker
 Whatweb
 Who.is Bot
 Widow
-WiseGuys Robot
 Wonderbot
-Wprecon
-Xaldon WebSpider
+Zauba
 Zeus
 ZumBot
-ZyBorg
-archive.org_bot
-autoemailspider
-backlink-check
-com.plumanalytics
-crawl.sogou.com
+adscanner
+coccocbot
+crawler.feedback
+crawler4j
+dataforseo.com
+domainsproject.org
+eCatch
 evc-batch
 facebookscraper
 heritrix
 ips-agent
-isitwp.com
 iubenda-radar
-magpie-crawler
 meanpathbot
-oBot
-s1z.ru
-satoristudio.net
-scalaj-http
+mediawords
+netEstate NE Crawler
+page scorer
+polaris version
+probe-image-size
 scan.lol
 seobility
+seostar
 serpstatbot
-sitechecker.pro
+siteripz
+sp_auditbot
 spyfu
-tAkeOut
-voyagerx.com
+sysscan
+webpros.com
+webprosbot
 x09Mozilla
+x22Mozilla
+zauba.io
+zgrab