V4.2022.03.2987 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
+01h4x.com
 404enemy
+ADmantX
 ASPSeek
 Abonti
-Aboundex
 Acunetix
-AhrefsBot
-AiHitBot
-Aipbot
+AfD-Verbotsverfahren
 AllSubmitter
-AlphaBot
-Ankit
-Anthill
-Asterias
-Atomseobot
-AwarioRssBot
-AwarioSmartBot
-BBBike
+Anarchie
+Anarchy
+Attach
+BDCbot
 BLEXBot
-BackDoorBot
-BackWeb
+BackStreet
+BacklinkCrawler
 Badass
-BatchFTP
-Bigfoot
-Boardreader
+Barkrowler
+Bitacle
+BlackWidow
+Blackboard
+BlowFish
 Bolt
-Brandprotect
-BuiltWith
+Buddy
+BuiltBotTough
 Bullseye
-CATExplorador
-CCBot
 CODE87
-Calculon
+Cegbfeieh
+CensysInspect
+CheTeam
+CheeseBot
 ChinaClaw
-Citoid
-Cloud mapping
-Cocolyzebot
+Chlooe
+Claritybot
+Cliqzbot
+Collector
 Copier
+CopyRightCheck
 Craftbot
-Crawling at Home Project
-Crescent
-Curious
-DTS Agent
+Custo
+CyotekWebCopy
+DSearch
+DatabaseDriverMysqli
+Demon
 Deusu
-Devil
 Digincore
-Discobot
+DigitalPebble
+Disco
+Discoverybot
 DnBCrawler-Analytics
-DnyzBot
-DomainSigmaCrawler
 DomainStatsBot
-Download Wonder
+Domains Project
+Dragonfly
 Drip
-ECCP/1.0
-EasyDL
+EMail Wolf
+Ebingbong
 Ecxi
 EirGrabber
+Evil
 Exabot
+Extreme Picture Finder
 Ezooms
-FDM
-FHscan
-Firefox/7.0
-Freeuploader
-Fuzz
+FemtosearchBot
+Fimap
+Foobot
+FyberSpider
+Fyrebot
 G-i-g-a-b-o-t
-GetRight
-GetWeb
+GT::WWW
+Genieo
+GermCrawler
 Gigabot
-Go!Zilla
-Go-Ahead-Got-It
-GrabNet
+Gotit
+Grafula
+GrapeshotCrawler
+GridBot
 HTTP::Lite
-Haansoft
-Hloader
+HTTrack
+Heritrix
 HonoluluBot
-IlseBot
+HybridBot
+IRLbot
 Image Fetch
-Internet Ninja
-InternetSeer
+IndeedBot
+InterGET
 Iria
-JennyBot
+Iskanie
+IstellaBot
+JamesBOT
+JetCar
+Jyxobot
 Kenjin Spider
-Keybot Translation-Search-Machine
 Kinza
-LeechGet
-LieBaoFast
-Lightspeedsystems
-LinkWalker
+Lanshanbot
+Larbin
+LexiBot
+Libwhisker
+Likse
+LinkScan
 Linkbot
-LinqiaScrapeBot
-Lipperhey
+Linkdexbot
+LinkextractorPro
+LinqiaRSSBot
 Lipperhey Spider
 Litemage_walker
 Ltx71
-MIIxpc
-MJ12bot
-MQQBrowser
-MSFrontPage
+MFC_Tear_Sample
+MIDown tool
+MSIECrawler
+Mag-Net
+Magnet
 Mail.RU_Bot
-Majestic12
-Mata Hari
+Majestic SEO
+Majestic-SEO
+Mass Downloader
+MeanPath Bot
+Meanpathbot
 Mediatoolkitbot
-MegaIndex.ru
+Metauri
 Microsoft URL Control
-Mojolicious
-Musobot
-NICErsPRO
-Name Intelligence
+Minefield
+Mister PiX
+Mojeek
+MolokaiBot
+Mr.4x3
+Nameprotect
 NearSite
-Needle
-Net Vampire
-NetLyzer
-NextGenSearchBot
-Niki-bot
-NimbleCrawler
-Ninja
+Nessus
+NetSpider
+Netcraft
+Nettrack
+Nikto
+Nmap
 Not
-Nutch
-Offline Explorer
-OpenLinkProfiler
+Offline Navigator
 OpenVAS
 Openfind
+Openvas
+OutclicksBot
 OutfoxBot
 PHPCrawl
-POE-Component-Client-HTTP
-PageScorer
-Pandalytics
+Page Analyzer
+PageGrabber
+PageThing.com
 Panscient
-Pavuk
-PictureFinder
+PeoplePal
+Petalbot
+Pi-Monster
 Pimonster
-PleaseCrawl
-ProPowerBot
-Psbot
+Pu_iN
 Pump
 PyCurl
 QueryN Metasearch
+Quick-Crawler
+RSSingBot
 RankActive
 RankingBot
 Rankivabot
 Re-re
 Reaper
-Ripper
-RocketCrawler
+Recorder
+RedesScrapy
+SBIder
 SEOkicks
-SEOprofiler
-SEOstats
 SISTRIX
 SalesIntelligent
-Scanbot
-ScoutJet
-ScrepyBot
-SemanticJuice
-Semrush
-SemrushBot
+ScanAlert
+Scrapy
+Seekport
+SentiBot
 SeoSiteCheckup
+Seomoz
 Shodan
-Siphon
-Site Sucker
-SiteLockSpider
-SiteSnagger
-SiteSucker
+Sitebeam
+Sitevigil
+SlySearch
 SmartDownload
 Snapbot
-SocialRankIOBot
+Snoopy
 Sociscraper
-Sottopop
-SpankBot
-Spanner
-Spinn3r
-Steeler
-Stripper
-Sucker
-Surfbot
-SurveyBot
-Suzuran
+Sosospider
+Spbot
+SputnikBot
 Swiftbot
-Szukacz
 T8Abot
+TeleportPro
 Telesoft
 Telesphoreo
+Telesphorep
 The Intraformant
-Thumbor
-Toata
+TightTwatBot
+Titan
 Toweyabot
-Tracemyfile
+Trendiction
+True_Robot
 Turingos
 Twice
-Typhoeus
 URLy Warning
-V-BOT
-Vacuum
+VB Project
 Vagabondo
 VelenPublicWebCrawler
-Virusdie
-WASALive-Bot
+VidibleScraper
+Voil
+WBSearchBot
+WEBDAV
+WISENutbot
 WPScan
+WWW-Collector-E
+WWW::Mechanize
 WWWOFFLE
-Wallpapers
-WallpapersHD
-WeSEE
-Web Auto
-Web Collage
-Web Enhancer
+Wallpapers/3.0
+Web Fuck
+WebAuto
+WebBandit
+WebEnhancer
 WebFetch
-WebFuck
 WebImageCollector
 WebLeacher
 WebPix
-WebReaper
-WebSauger
-WebZIP
-WebmasterWorldForumBot
+WebSucker
 Webshag
-Website Quester
-WebsiteQuester
-Webster
-Whack
-Widow
+Whacker
+WinHTTrack
 Wonderbot
-Wotbox
-Xaldon_WebSpider
+Woobot
+Wprecon
+Xaldon WebSpider
+Xenu
+YoudaoBot
 Zauba
-Zeus
-ZoomBot
-ZoominfoBot
-arquivo-web-crawler
-arquivo.pt
-autoemailspider
-cah.io.community
-check1.exe
-coccocbot
-dataforseo.com
-demandbase-bot
-eCatch
-iubenda-radar
-s1z.ru
+Zitebot
+ZmEu
+archive.org_bot
+crawler4j
+domainsproject.org
+heritrix
+instabid
+lwp-request
+meanpathbot
+mediawords
+plumanalytics
+polaris version
 satoristudio.net
+scalaj-http
 scan.lol
-seostar
-sexsearcher
-siteripz
-sogouspider
+sitechecker.pro
 sp_auditbot
 sysscan
 tAkeOut
-voyagerx.com
-x09Mozilla
-x22Mozilla
+trendiction.com
 zauba.io
-zgrab