V4.2021.12.2801 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
 01h4x.com
-404checker
 404enemy
+ADmantX
 ASPSeek
-AhrefsBot
-AiHitBot
+Aboundex
+Aboundexbot
+Acunetix
 Alexibot
-Alligator
-Ankit
-AwarioSmartBot
-BBBike
-BLEXBot
-BackStreet
-BacklinkCrawler
+AlphaBot
+Anarchy99
+Aspiegel
+Asterias
+AwarioRssBot
+BDCbot
+Backlink-Ceck
 Bandit
-Barkrowler
-Bitacle
-Black Hole
+BatchFTP
+BetaBot
+BlackWidow
 Blow
-BotALot
-Brandprotect
+Boardreader
 BuiltBotTough
-CSHttp
+CATExplorador
+CCBot
+CODE87
 Calculon
+Cegbfeieh
+CheTeam
+CherryPicker
 ChinaClaw
+Chlooe
+Citoid
 Cocolyzebot
+Cogentbot
 Collector
-Copier
-Crawling at Home Project
-Crescent
-Custo
-DSearch
+CopyRightCheck
+Craftbot
+CrazyWebCrawler
 DTS Agent
 DataCha0s
-Demon
 Deusu
-DittoSpyder
+Devil
+DigitalPebble
+Dirbuster
+Dispatch
+DnBCrawler-Analytics
 DnyzBot
-DomainAppender
+DomCopBot
 DomainSigmaCrawler
-Domains Project
-Dotbot
-Download Wonder
+Dragonfly
 ECCP/1.0
-Evil
+EMail Siphon
+EasyDL
+Ebingbong
+Ecxi
+EroCrawler
 Exabot
-Extreme Picture Finder
-Ezooms
-Fimap
-Foobot
-Fuzz
-FyberSpider
+Extractor
+FemtosearchBot
+Firefox/7.0
+FlashGet
+Flunky
+Freeuploader
 Fyrebot
-G-i-g-a-b-o-t
-GetRight
+GT::WWW
+Genieo
 GetWeb
-Go!Zilla
+Getintent
+Go-Ahead-Got-It
 GoZilla
-Grafula
+Gotit
+Grabber
 GrapeFX
-GridBot
 HEADMasterSEO
+HMView
 HTMLparser
+HTTP::Lite
 Haansoft
-Harvest
-Humanlinks
-Iblog
-Id-search
+HaosouSpider
+Havij
+Heritrix
+Hloader
+HybridBot
+IDBTE4M
 IlseBot
 Image Fetch
+IndeedBot
 Indy Library
-InfoNaviRobot
-Intelliseek
+InfoTekies
 InterGET
+Internet Ninja
+InternetSeer
 Iskanie
-JOC Web Spider
-JetCar
+IstellaBot
+JennyBot
+Jetty
 JikeSpider
-JustView
-Keyword Density
-Kinza
-LWP::Simple
-Lanshanbot
+Jorgee
+Jyxobot
+Larbin
 Leap
-LeechGet
+LeechFTP
 LexiBot
 Lftp
+LieBaoFast
+Lightspeedsystems
 LinkScan
-LinkpadBot
-LinksManager
-LinqiaMetadataDownloaderBot
+Linkbot
 LinqiaRSSBot
-LinqiaScrapeBot
-Lipperhey
-Lmspider
-MFC_Tear_Sample
-MIDown tool
-MQQBrowser
-Mail.RU_Bot
-Majestic SEO
-Majestic-SEO
-Majestic12
-MarkWatch
-Mass Downloader
+Lipperhey Spider
+MIIxpc
+MJ12bot
+MSFrontPage
+MSIECrawler
+Mag-Net
+Magnet
+MarkMonitor
 Mata Hari
-Mb2345Browser
-MeanPath Bot
+MauiBot
+Meanpathbot
+Mediatoolkitbot
+MegaIndex.ru
 Metauri
-MicroMessenger
 Microsoft URL Control
 Minefield
-Moblie Safari
 Mojeek
-Mozlila
-Mr.4x3
-Navroad
-Needle
-NetAnts
+Msrabot
+Nameprotect
+Net Vampire
 NetLyzer
-NetSpider
-Netcraft
-Nettrack
-Netvibes
-NextGenSearchBot
-Nikto
-NimbleCrawler
-Offline Explorer
-OpenLinkProfiler
-Openfind
-OrangeBot
-OrangeSpider
-OutclicksBot
+NetZIP
+Nibbler
+Nuclei
+Offline Navigator
+Openvas
 PHPCrawl
-POE-Component-Client-HTTP
 Page Analyzer
 PageGrabber
+PageScorer
 Pandalytics
-Piepmatz
+PeoplePal
+Petalbot
+Pi-Monster
+Pimonster
 Pixray
-PleaseCrawl
-ProWebWalker
-PyCurl
-RSSingBot
-RankingBot
-Rankivabot
-RankurBot
-Recorder
+Probethenet
+PxBroker
+QueryN Metasearch
+RankActive
+RankingBot2
+Re-re
+ReGet
+RepoMonkey
 RocketCrawler
-SBIder
+SEOkicks-Robot
 SEOprofiler
-SalesIntelligent
-ScoutJet
+SEOstats
+SMTBot
+Scanbot
 Screaming
+ScreenerBot
 ScrepyBot
-SearchmetricsBot
 SemanticJuice
-SemrushBot
-SeoSiteCheckup
-Seomoz
-Site Sucker
-SiteExplorer
-SiteLockSpider
+SeobilityBot
+Siphon
+SiteSnagger
 SiteSucker
 Sitebeam
-SlySearch
-SmartDownload
-Snapbot
 SocialRankIOBot
-Sogou web spider
-SpankBot
-Spbot
+Spammen
+Spanner
 Sqlworm
-Sucker
-SuperBot
+Sqworm
+Steeler
+Stripper
+Sucuri
+Surfbot
+SurveyBot
 Suzuran
-Swiftbot
-T0PHackTeam
-Teleport
 TeleportPro
+Telesoft
 Telesphoreo
-Telesphorep
-Toweyabot
-Trendiction
+Thumbor
+Tracemyfile
+Trendictionbot
 Turnitin
 Twice
-URLy.Warning
+Typhoeus
+URLy Warning
 UnisterBot
+Upflow
+VCI
 Vagabondo
-VelenPublicWebCrawler
-Virusdie
-WEBDAV
+VoidEYE
+Voltron
+WBSearchBot
 WPScan
 WWW-Collector-E
-WWW-Mechanize
 WWW::Mechanize
-Wallpapers/3.0
-WallpapersHD
-WeSEE
-Web Auto
+WWWOFFLE
+Web Enhancer
+Web Fetch
 Web Fuck
-Web Sauger
+Web Pix
 Web Sucker
-WebAuto
 WebCollage
 WebFetch
-WebImageCollector
+WebGo IS
+WebLeacher
 WebReaper
 WebSauger
-WebSucker
-WebWhacker
-Webshag
-WebsiteQuester
-Who.is Bot
+WebStripper
+Webalta
+WebmasterWorldForumBot
+Website Quester
+WebsiteExtractor
+Webster
+Whack
+Widow
 WinHTTrack
-WiseGuys Robot
 Woobot
-Wprecon
+Wotbox
+Xaldon WebSpider
 Xaldon_WebSpider
-Zade
+Zauba
+Zermelo
+Zeus
+ZmEu
+ZoomBot
 ZoominfoBot
-clark-crawler
-crawl.sogou.com
+arquivo-web-crawler
+arquivo.pt
+autoemailspider
 crawler.feedback
-domainsproject.org
-facebookscraper
+dataforseo.com
 heritrix
-internetVista monitor
-isitwp.com
-lwp-trivial
-magpie-crawler
-muhstik-scan
+instabid
+lwp-request
+meanpathbot
+mediawords
 oBot
 pcBrowser
+plumanalytics
 polaris version
 probe-image-size
-s1z.ru
-satoristudio.net
 seoscanners
 serpstatbot
-sexsearcher
-sp_auditbot
+sogouspider
 trendiction.com
-ubermetrics-technologies.com
-voyagerx.com
-webgains-bot
-webmeup-crawler
-webpros.com
-webprosbot
-x09Mozilla
-xpymep1.exe
+zauba.io