V4.2021.08.2529 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
 01h4x.com
+360Spider
 404checker
 AIBOT
-ASPSeek
-Abonti
-Aboundex
 Acunetix
+AhrefsBot
+AiHitBot
+Aipbot
 AllSubmitter
-Alligator
-Anarchie
+AlphaBot
+Anarchy
 Apexoo
-Attach
-AwarioRssBot
-BBBike
-BDCbot
+Asterias
+AwarioSmartBot
+BDFetch
 BLEXBot
 Backlink-Ceck
 Badass
 Bandit
 BatchFTP
-BetaBot
-Blackboard
-Blow
-BlowFish
-Brandprotect
-Buddy
+Battleztar Bazinga
+BlackWidow
 Bullseye
-BunnySlippers
-CCBot
-CODE87
-Cegbfeieh
-CherryPicker
-ChinaClaw
-Cocolyzebot
-Copier
+CSHttp
+Calculon
+CheTeam
+Chlooe
+Cliqzbot
+Cloud mapping
+Copyscape
+Cosmos
+Craftbot
+CrazyWebCrawler
+Crescent
+CrunchBot
 Curious
-CyotekWebCopy
-DBLBot
-DTS Agent
-DataCha0s
-DatabaseDriverMysqli
+DSearch
 Devil
-Digincore
+DigitalPebble
 Discobot
 Dispatch
-DittoSpyder
-DomainSigmaCrawler
-Dragonfly
-EMail Wolf
-EasyDL
-Ebingbong
+DomCopBot
+DomainCrawler
+Dotbot
+ECCP/1.0
+Ecxi
 EirGrabber
-EroCrawler
-Exabot
-Extractor
+Evil
 EyeNetIE
-Firefox/7.0
+Ezooms
+FHscan
 Flunky
-Foobot
-Freeuploader
-FrontPage
 Fuzz
-FyberSpider
 Fyrebot
 G-i-g-a-b-o-t
-GT::WWW
-GetWeb
+GalaxyBot
+GermCrawler
 Gigablast
-Gotit
-GrabNet
+Go!Zilla
+GoZilla
 Grabber
+GrapeFX
+GrapeshotCrawler
 GridBot
+HTTP::Lite
+HaosouSpider
 Harvest
-Havij
 Heritrix
+HybridBot
 IDBTE4M
 Id-search
-Image Sucker
-InfoNaviRobot
-InfoTekies
-Internet Ninja
+IndeedBot
+Indy Library
+Intelliseek
+InterGET
+Iria
 Iskanie
-JamesBOT
-Jbrofuzz
-JennyBot
+IstellaBot
+JOC Web Spider
 JikeSpider
 Joomla
-Kenjin Spider
-Keyword Density
-Lanshanbot
-Larbin
-Leap
-LeechFTP
-LibWeb
-LinqiaMetadataDownloaderBot
-Lipperhey
-Litemage_walker
+Jorgee
+JustView
+Kinza
+Kozmosbot
+LWP::Simple
+LexiBot
+LinkpadBot
+LinksManager
+LinqiaRSSBot
+LinqiaScrapeBot
 Ltx71
-MIDown tool
 MJ12bot
-MQQBrowser
 MSIECrawler
-Mag-Net
-Magnet
-Mail.RU_Bot
-MarkMonitor
-MarkWatch
+Majestic12
 Mass Downloader
+Mata Hari
 MauiBot
-Mb2345Browser
-MeanPath Bot
-Mediatoolkitbot
+Meanpathbot
 MegaIndex.ru
-Metauri
-Microsoft URL Control
+MicroMessenger
+Mister PiX
 Moblie Safari
-Mojeek
-NICErsPRO
-Name Intelligence
+Mojolicious
+Musobot
+NPbot
 Nameprotect
-NearSite
-Needle
+Navroad
+NetAnts
+NetMechanic
+NetSpider
+NetZIP
 Nettrack
-Netvibes
-Nibbler
 Nikto
+NimbleCrawler
 Nimbostratus
-Ninja
-Nmap
+Nuclei
 Octopus
-Offline Navigator
-OnCrawl
 OpenLinkProfiler
+OpenVAS
+Openfind
+Openvas
+OutfoxBot
 PHPCrawl
+POE-Component-Client-HTTP
 Page Analyzer
-Panscient
+Papa Foto
+Pi-Monster
+Picscout
 Picsearch
-Piepmatz
+Pixray
+Pockey
 ProPowerBot
-Pump
-PxBroker
+ProWebWalker
+Psbot
 PyCurl
+QueryN Metasearch
 Quick-Crawler
-RankActive
-RankActiveLinkBot
-RankFlex
+RankingBot
 RankingBot2
-Rankivabot
-RankurBot
-ReGet
+Re-re
+RealDownload
+RebelMouse
 RedesScrapy
 RepoMonkey
+Rogerbot
 SEOkicks-Robot
 SEOprofiler
 SMTBot
-ScanAlert
 Scanbot
-ScoutJet
+Scrapy
 ScreenerBot
 ScrepyBot
-SearchmetricsBot
-SemrushBot
-SentiBot
-Seomoz
-Site Sucker
-Siteimprove
-Sitevigil
+Semrush
+Shodan
+Siphon
+SiteExplorer
+SiteSnagger
+SiteSucker
+SmartDownload
+Snake
 Snapbot
-Snoopy
-Spbot
+Sociscraper
+Sosospider
+Sottopop
+SpaceBison
 Spinn3r
+SputnikBot
 Sqlmap
-Sqlworm
-Stripper
+Sqworm
+Steeler
+Sucker
 Sucuri
 SuperBot
-SuperHTTP
-T0PHackTeam
-Teleport
-TeleportPro
+Telesphorep
 The Intraformant
-Titan
+TheNomad
+TightTwatBot
+Toata
 Tracemyfile
-Trendiction
-True_Robot
-Turingos
 Turnitin
 TurnitinBot
+TwengaBot
+URLy Warning
 UnisterBot
-V-BOT
-VelenPublicWebCrawler
-Voil
+Upflow
+VB Project
+VCI
+Vacuum
+Virusdie
+VoidEYE
+Voltron
 WASALive-Bot
-WEBDAV
-WISENutbot
-WWW-Collector-E
 WWW-Mechanize
 WWW::Mechanize
+WWWOFFLE
 Wallpapers/3.0
 Web Collage
+Web Fetch
+Web Pix
+Web Sauger
 Web Sucker
-WebCollage
-WebFetch
-WebGo IS
+WebBandit
+WebCopier
 WebImageCollector
 WebLeacher
-WebStripper
-WebWhacker
+WebPix
+WebSucker
 WebZIP
-WebmasterWorldForumBot
+Webalta
 Webshag
+Website Quester
 WebsiteExtractor
-WebsiteQuester
 Whack
+Whatweb
+Who.is Bot
 Widow
 WinHTTrack
+WiseGuys Robot
 Wonderbot
-Wotbox
 Wprecon
 Xaldon_WebSpider
+Xenu
 YoudaoBot
-Zade
-Zermelo
-ZmEu
-ZoomBot
+Zauba
+Zeus
+Zitebot
+ZumBot
 arquivo.pt
+autoemailspider
 backlink-check
-cognitiveseo
-com.plumanalytics
-crawler.feedback
+clark-crawler
+crawl.sogou.com
 eCatch
-facebookscraper
-instabid
-lwp-request
+evc-batch
+isitwp.com
 lwp-trivial
+meanpathbot
+mediawords
 muhstik-scan
-netEstate NE Crawler
-page scorer
-satoristudio.net
-scalaj-http
-sexsearcher
-sitechecker.pro
-sogouspider
-spyfu
+pcBrowser
+scan.lol
+seoscanners
+serpstatbot
+sysscan
 trendiction.com
-trendiction.de
-webmeup-crawler
 webpros.com
-x09Mozilla
-zgrab
+webprosbot
+x22Mozilla
+xpymep1.exe