V4.2021.12.2802 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
-01h4x.com
+404checker
 404enemy
-ADmantX
-ASPSeek
+ALittle Client
 Aboundex
-Aboundexbot
-Acunetix
-Alexibot
-AlphaBot
-Anarchy99
-Aspiegel
+AhrefsBot
+AllSubmitter
+Ankit
+Apexoo
 Asterias
-AwarioRssBot
-BDCbot
-Backlink-Ceck
-Bandit
+BBBike
+BackDoorBot
+BacklinkCrawler
+Badass
+Barkrowler
 BatchFTP
-BetaBot
-BlackWidow
-Blow
+Black Hole
+Blackboard
+BlowFish
 Boardreader
-BuiltBotTough
-CATExplorador
-CCBot
+BotALot
+Brandprotect
+Brandwatch
+Buck
 CODE87
-Calculon
 Cegbfeieh
-CheTeam
+CensysInspect
+CheeseBot
 CherryPicker
-ChinaClaw
-Chlooe
-Citoid
-Cocolyzebot
+Claritybot
 Cogentbot
-Collector
-CopyRightCheck
-Craftbot
 CrazyWebCrawler
+Curious
+Custo
+CyotekWebCopy
 DTS Agent
 DataCha0s
-Deusu
-Devil
+Demon
+Digincore
 DigitalPebble
 Dirbuster
-Dispatch
-DnBCrawler-Analytics
+Discoverybot
+DittoSpyder
 DnyzBot
-DomCopBot
+DomainAppender
+DomainCrawler
 DomainSigmaCrawler
-Dragonfly
+Dotbot
 ECCP/1.0
 EMail Siphon
+EMail Wolf
 EasyDL
-Ebingbong
-Ecxi
+EirGrabber
 EroCrawler
-Exabot
-Extractor
-FemtosearchBot
-Firefox/7.0
-FlashGet
-Flunky
-Freeuploader
-Fyrebot
-GT::WWW
+Evil
+Foobot
+FrontPage
 Genieo
+GermCrawler
 GetWeb
-Getintent
-Go-Ahead-Got-It
-GoZilla
-Gotit
+Gigabot
 Grabber
+Grafula
 GrapeFX
-HEADMasterSEO
-HMView
-HTMLparser
-HTTP::Lite
-Haansoft
-HaosouSpider
-Havij
+GrapeshotCrawler
 Heritrix
-Hloader
-HybridBot
 IDBTE4M
-IlseBot
+IRLbot
+Id-search
 Image Fetch
 IndeedBot
 Indy Library
-InfoTekies
-InterGET
-Internet Ninja
-InternetSeer
-Iskanie
-IstellaBot
-JennyBot
-Jetty
-JikeSpider
+Intelliseek
+Iria
+Jbrofuzz
+Joomla
 Jorgee
-Jyxobot
-Larbin
+JustView
+Kenjin Spider
+LNSpiderguy
 Leap
-LeechFTP
+LeechGet
 LexiBot
-Lftp
-LieBaoFast
+LibWeb
+Libwhisker
 Lightspeedsystems
-LinkScan
-Linkbot
-LinqiaRSSBot
-Lipperhey Spider
-MIIxpc
-MJ12bot
+LinkpadBot
+LinqiaMetadataDownloaderBot
+LinqiaScrapeBot
+Lmspider
+Ltx71
+MIDown tool
 MSFrontPage
-MSIECrawler
-Mag-Net
+MTRobot
 Magnet
+Majestic SEO
 MarkMonitor
-Mata Hari
+Mass Downloader
+Masscan
 MauiBot
-Meanpathbot
 Mediatoolkitbot
-MegaIndex.ru
-Metauri
-Microsoft URL Control
-Minefield
+MicroMessenger
+Mister PiX
 Mojeek
-Msrabot
-Nameprotect
+Morfeus Fucking Scanner
+Mr.4x3
+Musobot
+NPbot
+Name Intelligence
+Needle
 Net Vampire
 NetLyzer
 NetZIP
+Nettrack
+Netvibes
+NextGenSearchBot
 Nibbler
+Niki-bot
 Nuclei
-Offline Navigator
+Octopus
+Offline Explorer
 Openvas
-PHPCrawl
+OrangeBot
+OutclicksBot
 Page Analyzer
-PageGrabber
-PageScorer
-Pandalytics
+PageAnalyzer
+PageThing.com
+Panscient
+Pavuk
 PeoplePal
-Petalbot
-Pi-Monster
-Pimonster
-Pixray
+Picsearch
+Piepmatz
+PleaseCrawl
+Pockey
+ProPowerBot
 Probethenet
+Psbot
+Pu_iN
 PxBroker
-QueryN Metasearch
+Quick-Crawler
+RSSingBot
 RankActive
-RankingBot2
-Re-re
-ReGet
+RankingBot
+RealDownload
+RebelMouse
+Recorder
 RepoMonkey
-RocketCrawler
+Ripper
+Rogerbot
+SEOkicks
 SEOkicks-Robot
-SEOprofiler
+SEOlyticsCrawler
 SEOstats
-SMTBot
-Scanbot
-Screaming
+SISTRIX
+Scrapy
 ScreenerBot
 ScrepyBot
-SemanticJuice
-SeobilityBot
+Searchestate
+Semrush
+SemrushBot
 Siphon
-SiteSnagger
-SiteSucker
+SiteExplorer
+SiteLockSpider
 Sitebeam
-SocialRankIOBot
-Spammen
+Siteimprove
+SlySearch
+Snapbot
+Snoopy
+SpankBot
 Spanner
-Sqlworm
+SputnikBot
 Sqworm
 Steeler
 Stripper
 Sucuri
-Surfbot
+SuperBot
+SuperHTTP
 SurveyBot
-Suzuran
-TeleportPro
+Szukacz
 Telesoft
-Telesphoreo
-Thumbor
-Tracemyfile
+TightTwatBot
+Toata
+Toweyabot
 Trendictionbot
-Turnitin
-Twice
-Typhoeus
+True_Robot
 URLy Warning
 UnisterBot
-Upflow
-VCI
-Vagabondo
-VoidEYE
-Voltron
-WBSearchBot
-WPScan
+VB Project
+VeriCiteCrawler
+VidibleScraper
+Voil
+WASALive-Bot
+WISENutbot
 WWW-Collector-E
 WWW::Mechanize
 WWWOFFLE
-Web Enhancer
-Web Fetch
+Wallpapers
+Wallpapers/3.0
 Web Fuck
-Web Pix
-Web Sucker
+Web Sauger
+WebBandit
 WebCollage
-WebFetch
+WebCopier
+WebFuck
 WebGo IS
+WebImageCollector
 WebLeacher
-WebReaper
 WebSauger
-WebStripper
-Webalta
-WebmasterWorldForumBot
-Website Quester
+WebWhacker
 WebsiteExtractor
-Webster
-Whack
-Widow
+WebsiteQuester
+Whacker
 WinHTTrack
-Woobot
+Wonderbot
 Wotbox
-Xaldon WebSpider
-Xaldon_WebSpider
-Zauba
-Zermelo
-Zeus
 ZmEu
 ZoomBot
-ZoominfoBot
+adscanner
+archive.org_bot
 arquivo-web-crawler
-arquivo.pt
 autoemailspider
-crawler.feedback
+coccocbot-web
+com.plumanalytics
 dataforseo.com
+demandbase-bot
+eCatch
+evc-batch
+gopher
 heritrix
-instabid
+ips-agent
 lwp-request
-meanpathbot
-mediawords
-oBot
-pcBrowser
-plumanalytics
+lwp-trivial
+muhstik-scan
 polaris version
-probe-image-size
+s1z.ru
+scalaj-http
+scan.lol
+seocompany.store
 seoscanners
-serpstatbot
-sogouspider
-trendiction.com
+seostar
+sitechecker.pro
+siteripz
+sp_auditbot
+trendiction.de
+ubermetrics-technologies.com
+voyagerx.com
+webgains-bot
+x22Mozilla
+xpymep1.exe
 zauba.io
+zgrab