V4.2021.08.2511 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
-01h4x.com
-360Spider
+404enemy
 80legs
 ADmantX
+AIBOT
+ALittle Client
 Abonti
 Aboundex
-AhrefsBot
-AiHitBot
-Alligator
-AwarioRssBot
+AfD-Verbotsverfahren
+Aipbot
+Alexibot
+AllSubmitter
+Anarchie
+Aspiegel
+Asterias
+Attach
+AwarioSmartBot
 BackStreet
-BackWeb
 Backlink-Ceck
-BacklinkCrawler
+Badass
+Bandit
+Bigfoot
+Black Hole
+BlowFish
 Boardreader
-BotALot
-Brandprotect
 Buddy
-BuiltBotTough
-BuiltWith
-BuzzSumo
-CATExplorador
-CensysInspect
+Calculon
 CheTeam
+CherryPicker
+ChinaClaw
 Claritybot
-Cogentbot
-Copyscape
+Cliqzbot
+Cloud mapping
+Cocolyzebot
 Cosmos
-CrunchBot
-DIIbot
-Demon
-Deusu
-Devil
-DigitalPebble
-Dirbuster
-Discobot
+CrazyWebCrawler
+Crescent
+Curious
+DSearch
+DataCha0s
+Digincore
+Disco
+Discoverybot
 Dispatch
-DomCopBot
-DomainCrawler
-Dotbot
+DomainStatsBot
+Domains Project
 Download Wonder
-EMail Wolf
-Ebingbong
-EroCrawler
+Drip
+ECCP/1.0
+EMail Siphon
 Exabot
-Express WebPictures
+ExtLinksBot
 ExtractorPro
-Extreme Picture Finder
-Ezooms
-FHscan
-FemtosearchBot
-Fimap
-FlashGet
-Foobot
+Flunky
+Fuzz
+Fyrebot
+G-i-g-a-b-o-t
 GT::WWW
-GetRight
-Getintent
-Gigabot
-GoZilla
-GrapeFX
-HTMLparser
-Haansoft
-HaosouSpider
-Harvest
+GetWeb
+Go!Zilla
+Go-Ahead-Got-It
+Grabber
+GrapeshotCrawler
+GridBot
 Havij
-HybridBot
+Heritrix
 IDBot
 Image Fetch
-InfoNaviRobot
+Image Sucker
+Indy Library
+InfoTekies
+Intelliseek
 InterGET
 InternetSeer
-Iria
 Iskanie
 IstellaBot
 JOC Web Spider
 Jbrofuzz
 JetCar
-JustView
-Jyxobot
+Jorgee
 Kinza
 Kozmosbot
+LNSpiderguy
 LWP::Simple
-Lanshanbot
+Larbin
+Leap
 LeechFTP
-LexiBot
-LieBaoFast
+LibWeb
+Lightspeedsystems
+LinkScan
 LinkWalker
 Linkdexbot
-LinqiaMetadataDownloaderBot
-LinqiaScrapeBot
-Lmspider
-MFC_Tear_Sample
+LinkpadBot
+LinqiaRSSBot
+Ltx71
+MIDown tool
 MQQBrowser
-MSFrontPage
-MSIECrawler
+MTRobot
 Mag-Net
 Majestic-SEO
+Mass Downloader
 Masscan
-Mata Hari
 Mb2345Browser
-MeanPath Bot
+Meanpathbot
+Mediatoolkitbot
+MegaIndex.ru
 Metauri
-Mojeek
-Mozlila
+MicroMessenger
+Moblie Safari
+Morfeus Fucking Scanner
 Mr.4x3
-Msrabot
-Musobot
-Name Intelligence
+NPbot
 Nameprotect
-Needle
-NetAnts
-NetSpider
-NetZIP
+Nessus
+NetMechanic
 Netcraft
-Netvibes
+Nibbler
 Niki-bot
 NimbleCrawler
-Nimbostratus
+Nmap
+Nutch
 Octopus
 Offline Explorer
 Offline Navigator
-OnCrawl
-OpenVAS
-OrangeBot
-OutclicksBot
+OpenLinkProfiler
+Openfind
+OrangeSpider
+OutfoxBot
+PECL::HTTP
+PHPCrawl
+POE-Component-Client-HTTP
+Page Analyzer
+PageAnalyzer
 PageGrabber
-PageScorer
-Pavuk
-Pi-Monster
-Pixray
-Pockey
+Pandalytics
+Panscient
+Petalbot
+Picscout
+Picsearch
+Pimonster
 ProPowerBot
-ProWebWalker
 Psbot
-Pump
+PxBroker
+PyCurl
 QueryN Metasearch
-RSSingBot
-RankFlex
-RankingBot
-RankingBot2
+RankActive
+Rankivabot
 Re-re
-RealDownload
-Reaper
-Recorder
-RepoMonkey
-RocketCrawler
-SBIder
+RedesScrapy
 SEOkicks
-SISTRIX
-SalesIntelligent
+SMTBot
+ScanAlert
 Scanbot
+ScoutJet
 Scrapy
+ScreenerBot
+ScrepyBot
 SearchmetricsBot
+Semrush
 SeoSiteCheckup
 SeobilityBot
-Shodan
 SiteLockSpider
-SlySearch
-SmartDownload
-Snoopy
-SocialRankIOBot
-Sosospider
-Sottopop
+SiteSnagger
+Snake
+Snapbot
+Sociscraper
+Sogou web spider
+SpaceBison
 Spanner
-Spbot
-Spinn3r
-SputnikBot
-Sqlmap
-Sqlworm
-Steeler
-Stripper
+Sucker
 Sucuri
 SuperBot
 SuperHTTP
-Surfbot
 SurveyBot
-Suzuran
 Szukacz
 T0PHackTeam
-T8Abot
-TeleportPro
 Telesoft
 Telesphoreo
-Telesphorep
 The Intraformant
-Thumbor
 TightTwatBot
+Toata
 Tracemyfile
+Trendiction
+Trendictionbot
+True_Robot
 Turingos
-TwengaBot
-UnisterBot
+Turnitin
+URLy Warning
+URLy.Warning
 V-BOT
-VB Project
 VCI
-VeriCiteCrawler
+Vacuum
 Virusdie
-Voltron
+VoidEYE
+WEBDAV
 WISENutbot
-WWW-Collector-E
 WWWOFFLE
-Wallpapers
+Wallpapers/3.0
+WeSEE
+Web Auto
+Web Collage
+Web Enhancer
 Web Fetch
-Web Pix
+Web Sauger
 Web Sucker
-WebCollage
-WebFuck
-WebPix
-WebReaper
-WebSucker
-WebWhacker
+WebImageCollector
+WebSauger
+WebStripper
+WebZIP
+Webalta
+WebmasterWorldForumBot
+Webshag
 Website Quester
-Whatweb
+Whack
 Who.is Bot
+WiseGuys Robot
+Wonderbot
 Woobot
-Zermelo
-Zitebot
+Wotbox
+Xaldon WebSpider
+Xenu
+Zauba
+Zeus
 ZmEu
+ZoomBot
 ZumBot
-ZyBorg
-archive.org_bot
 arquivo-web-crawler
 arquivo.pt
+autoemailspider
+cah.io.community
 cognitiveseo
-com.plumanalytics
+crawl.sogou.com
 crawler4j
-facebookscraper
-gopher
+evc-batch
 instabid
-lwp-request
-lwp-trivial
+internetVista monitor
+ips-agent
+isitwp.com
 magpie-crawler
-meanpathbot
-muhstik-scan
-polaris version
-probe-image-size
-scalaj-http
+page scorer
+pcBrowser
+s1z.ru
 scan.lol
-seobility
 seoscanners
-siteripz
-sogouspider
-spyfu
-tAkeOut
-trendiction.de
-voyagerx.com
+seostar
+serpstatbot
+sysscan
+trendiction.com
 webmeup-crawler
 x09Mozilla
-x22Mozilla
 xpymep1.exe
-zauba.io