V4.2022.03.2989 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
 01h4x.com
-360Spider
-404checker
 404enemy
-ADmantX
-Aboundex
-Aboundexbot
-AiHitBot
-AllSubmitter
+Abonti
+Acunetix
+Alexibot
 AlphaBot
-Anarchie
-Anarchy
+Anarchy99
+Anthill
 Apexoo
 Aspiegel
-AwarioSmartBot
+Attach
 BDFetch
+BackStreet
+BackWeb
 Backlink-Ceck
-BacklinkCrawler
 Badass
+Bandit
 Barkrowler
-Black Hole
+Bigfoot
+Bitacle
+Blackboard
 BlowFish
-Boardreader
-Buck
+Bolt
+Brandprotect
+BuiltWith
 BunnySlippers
-BuzzSumo
-CCBot
-CSHttp
-Calculon
-CensysInspect
+CATExplorador
+CODE87
+CazoodleBot
+Cegbfeieh
+CheTeam
+CheeseBot
 CherryPicker
-ChinaClaw
+Chlooe
 Citoid
-Claritybot
+Cloud mapping
 Cocolyzebot
-Collector
-CopyRightCheck
-Copyscape
-CrazyWebCrawler
-CrunchBot
+Cosmos
+Crawling at Home Project
+Curious
 Custo
-DIIbot
+CyotekWebCopy
+DTS Agent
+DataCha0s
 Demon
-Deusu
 Devil
-Digincore
 Discobot
-DomainCrawler
-DomainStatsBot
-Dragonfly
+DittoSpyder
+DnBCrawler-Analytics
+DomCopBot
+DomainAppender
+Domains Project
+EMail Siphon
 EMail Wolf
 EasyDL
+Ebingbong
 Ecxi
+EirGrabber
 Evil
 Extractor
+EyeNetIE
+FHscan
 FemtosearchBot
 Firefox/7.0
 FlashGet
-Flunky
 Foobot
-FrontPage
-Fuzz
-G-i-g-a-b-o-t
-GT::WWW
 GermCrawler
-Go!Zilla
-Go-Ahead-Got-It
-Grafula
+GetRight
+GetWeb
+GoZilla
+GrabNet
 GrapeshotCrawler
 GridBot
-HTMLparser
+HEADMasterSEO
+HMView
 HTTP::Lite
-HTTrack
+HaosouSpider
+Havij
+Heritrix
 Hloader
-Iblog
-Indy Library
-InfoNaviRobot
-Intelliseek
-Internet Ninja
-Iskanie
+Humanlinks
+HybridBot
+IDBTE4M
+IRLbot
+Image Sucker
+InfoTekies
+InternetSeer
+Iria
+JOC Web Spider
+JamesBOT
+Jbrofuzz
 JetCar
-Joomla
+Jetty
 Jorgee
+Keybot Translation-Search-Machine
+Kozmosbot
 LNSpiderguy
+Lanshanbot
+LeechFTP
 LeechGet
+LexiBot
+Lftp
+LibWeb
 Likse
-LinqiaRSSBot
+Linkdexbot
+LinkextractorPro
 LinqiaScrapeBot
-Lipperhey
-Litemage_walker
-Lmspider
-Ltx71
 MIDown tool
-MJ12bot
-MQQBrowser
-MSFrontPage
-Mag-Net
-Majestic SEO
-Majestic-SEO
+MSIECrawler
+Majestic12
 MarkMonitor
-Mass Downloader
+MarkWatch
 MauiBot
 MeanPath Bot
-MegaIndex.ru
-Microsoft Data Access
-Mojeek
+Meanpathbot
+MicroMessenger
+Moblie Safari
+MolokaiBot
+Morfeus Fucking Scanner
 Mozlila
-Msrabot
+Musobot
 Name Intelligence
 Nameprotect
-Navroad
+NearSite
+Nessus
 Net Vampire
-NetLyzer
-NetMechanic
-Nettrack
+NetAnts
+NetSpider
 Netvibes
-Nibbler
-NimbleCrawler
-Nimbostratus
-Ninja
+NextGenSearchBot
 Nmap
-Not
-Nutch
-Offline Navigator
-OpenVAS
-Openfind
-OrangeSpider
+Offline Explorer
+OpenLinkProfiler
 PECL::HTTP
-POE-Component-Client-HTTP
-PageAnalyzer
-Panscient
+PHPCrawl
+PageGrabber
+Pavuk
+PeoplePal
+Petalbot
+Pi-Monster
 Picscout
-PictureFinder
-Pimonster
+Picsearch
 ProPowerBot
-ProWebWalker
-PxBroker
+Probethenet
+Pump
 QueryN Metasearch
+Quick-Crawler
 RSSingBot
 RankActive
-RankFlex
-RankingBot2
+RankActiveLinkBot
+RankingBot
+Rankivabot
 RankurBot
-RealDownload
-Recorder
-RedesScrapy
-RepoMonkey
-Ripper
+Reaper
+RocketCrawler
 SEOkicks
+SEOkicks-Robot
+SISTRIX
+SMTBot
 Scanbot
-ScoutJet
-Screaming
 ScreenerBot
-SemanticJuice
+ScrepyBot
+Searchestate
 Semrush
-SemrushBot
 SentiBot
-SeoSiteCheckup
-SiteCheckerBotCrawler
-SiteExplorer
-SiteSucker
+Seomoz
+Siphon
+SiteLockSpider
 Sitebeam
-Siteimprove
-Sitevigil
-SlySearch
+SmartDownload
+Snake
 Snapbot
-SpaceBison
+Snoopy
+SocialRankIOBot
+Spammen
 SpankBot
-Spbot
-SputnikBot
-Sqlmap
-SurveyBot
+Sucuri
+SuperBot
+SuperHTTP
+Swiftbot
 Szukacz
-T0PHackTeam
 T8Abot
-Telesphorep
-The Intraformant
-Toweyabot
-Tracemyfile
-Trendiction
-Turnitin
-Twice
-URLy.Warning
-UnisterBot
-VB Project
+Telesoft
+TheNomad
+Thumbor
+Titan
+Trendictionbot
+Turingos
+TwengaBot
+V-BOT
 Vagabondo
-VelenPublicWebCrawler
-WASALive-Bot
-WBSearchBot
-WEBDAV
-WISENutbot
+Virusdie
+VoidEYE
+Voltron
 WWW-Mechanize
-Wallpapers/3.0
+WWW::Mechanize
 WallpapersHD
-Web Fuck
+Web Auto
+Web Enhancer
 Web Pix
-WebCollage
-WebEnhancer
-WebFetch
+Web Sucker
+WebAuto
 WebGo IS
 WebImageCollector
-WebLeacher
-WebSucker
+WebPix
+WebReaper
+WebSauger
 WebWhacker
-WebZIP
-WebmasterWorldForumBot
+Website Quester
+WebsiteQuester
 Webster
-Whack
-Whacker
-Whatweb
-Wonderbot
+Widow
+WiseGuys Robot
 Woobot
-Wotbox
-Wprecon
 Xaldon_WebSpider
 YoudaoBot
 Zade
-ZmEu
+Zeus
+Zitebot
 ZoomBot
-ZumBot
-ZyBorg
-arquivo.pt
-cah.io.community
+ZoominfoBot
+archive.org_bot
 clark-crawler
-coccocbot
-cognitiveseo
-com.plumanalytics
-demandbase-bot
+crawl.sogou.com
+crawler.feedback
 eCatch
-instabid
+evc-batch
 internetVista monitor
 iubenda-radar
+lwp-trivial
 magpie-crawler
 meanpathbot
+mediawords
 page scorer
-pcBrowser
+plumanalytics
+probe-image-size
 ripz
-s1z.ru
-scalaj-http
 scan.lol
-seocompany.store
-sitechecker.pro
+seobility
+seoscanners
+sexsearcher
 siteripz
-sp_auditbot
+spyfu
+trendiction.com
+trendiction.de
+ubermetrics-technologies.com
+voyagerx.com
 webgains-bot
-webmeup-crawler
-x09Mozilla
-x22Mozilla
+xpymep1.exe
 zauba.io
-zgrab