V4.2021.07.2458 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
 01h4x.com
-AIBOT
+404checker
+404enemy
 ASPSeek
-Aboundexbot
+Abonti
+AhrefsBot
 Aipbot
-AllSubmitter
+Alexibot
 AlphaBot
 Anarchy
 Ankit
-Anthill
+Apexoo
+Aspiegel
 Asterias
-AwarioRssBot
-AwarioSmartBot
-BDFetch
-BackStreet
-BackWeb
-Backlink-Ceck
+Attach
+BLEXBot
+BackDoorBot
+BacklinkCrawler
 Badass
+Bandit
+BatchFTP
+Bigfoot
 BlackWidow
-Boardreader
-BotALot
+Bolt
 Brandprotect
 Brandwatch
-Buck
-Buddy
-BuiltBotTough
-CATExplorador
-CODE87
-CensysInspect
-CheTeam
-Chlooe
-Cliqzbot
+BuiltWith
+BuzzSumo
+Calculon
 Cogentbot
 Copier
-CopyRightCheck
-Cosmos
-CrazyWebCrawler
+Copyscape
+Craftbot
 CrunchBot
-Curious
 Custo
-CyotekWebCopy
 DBLBot
-DSearch
+DIIbot
+DTS Agent
 DatabaseDriverMysqli
-Discoverybot
-Dispatch
-DittoSpyder
-DomainStatsBot
+Devil
+Digincore
+DigitalPebble
+Discobot
+DomainAppender
+DomainCrawler
 Dotbot
-Download Wonder
-Drip
-EMail Wolf
-EasyDL
-EirGrabber
-EroCrawler
-Evil
-Express WebPictures
-Extractor
+ECCP/1.0
+EMail Siphon
+Ecxi
+ExtLinksBot
 ExtractorPro
 EyeNetIE
-FHscan
+FDM
+Fimap
 FlashGet
-Freeuploader
+Foobot
 FrontPage
-Fuzz
-FyberSpider
-Fyrebot
-G-i-g-a-b-o-t
 GermCrawler
 GetRight
-Gigablast
-Gigabot
+GetWeb
+Getintent
 Go!Zilla
-GoZilla
+Gotit
+GrabNet
+Grabber
 Grafula
-GrapeshotCrawler
 HMView
-HTTP::Lite
 Haansoft
-Hloader
-Humanlinks
-HybridBot
+HaosouSpider
+Harvest
+Havij
 IDBot
-IRLbot
-Iblog
 Id-search
-Image Sucker
 InfoTekies
 Intelliseek
-JOC Web Spider
+Internet Ninja
+Iskanie
+JamesBOT
 Jbrofuzz
-Jetty
-Joomla
-Kozmosbot
-LeechGet
-Lftp
+JustView
+Kenjin Spider
+Kinza
+Lanshanbot
+Larbin
+LeechFTP
 Libwhisker
-Lightspeedsystems
-Likse
+LinkScan
 LinkWalker
-Linkbot
 Linkdexbot
+LinksManager
 LinqiaMetadataDownloaderBot
+LinqiaScrapeBot
 Lipperhey
-Ltx71
+Lipperhey Spider
 MIDown tool
-MJ12bot
+MIIxpc
 MSFrontPage
-MTRobot
-Magnet
+Majestic-SEO
+MarkWatch
 Mass Downloader
+Masscan
 Mata Hari
 Mb2345Browser
 MeanPath Bot
-Mediatoolkitbot
+MegaIndex.ru
 Metauri
+MicroMessenger
 Microsoft Data Access
-Morfeus Fucking Scanner
-Msrabot
+Microsoft URL Control
+Moblie Safari
+Mojolicious
+Mr.4x3
+Musobot
 NICErsPRO
-NPbot
-Name Intelligence
+Nameprotect
 Navroad
 NearSite
 Needle
-Nessus
 NetLyzer
-NetMechanic
+NetSpider
 NetZIP
-Nibbler
+Netcraft
+Nettrack
 Nikto
-NimbleCrawler
+Nimbostratus
 Ninja
 Nuclei
 OPPO A33
-Octopus
 Offline Explorer
 Offline Navigator
-OpenLinkProfiler
+OnCrawl
 Openfind
+Openvas
 OrangeBot
 OrangeSpider
-OutfoxBot
-PECL::HTTP
-Page Analyzer
+OutclicksBot
+PageAnalyzer
+PageScorer
 PageThing.com
-Petalbot
-Pi-Monster
+Pandalytics
+Panscient
+Pavuk
+PeoplePal
 Picscout
-Piepmatz
-Pixray
-ProPowerBot
-Pu_iN
-PxBroker
-PyCurl
-RankActiveLinkBot
-RankingBot
-Rankivabot
+Picsearch
+PleaseCrawl
+ProWebWalker
+Probethenet
+QueryN Metasearch
+Quick-Crawler
+RankurBot
 RealDownload
-Recorder
-RepoMonkey
 Ripper
-Rogerbot
-SEOkicks
-SEOstats
+RocketCrawler
 SISTRIX
-SalesIntelligent
+SMTBot
 ScanAlert
-Scrapy
-Screaming
-ScreenerBot
+Scanbot
+ScoutJet
+ScrepyBot
 Searchestate
+SearchmetricsBot
+Semrush
 SentiBot
-SeoSiteCheckup
-SeobilityBot
-Site Sucker
-SiteSucker
-Sitebeam
-Sitevigil
-SmartDownload
+Shodan
+Siphon
+SiteExplorer
 Snapbot
 Snoopy
-SocialRankIOBot
 Sociscraper
+Sogou web spider
+SpaceBison
 Spanner
-Spbot
-Spinn3r
-SputnikBot
-Sqlmap
-Sqlworm
-Steeler
-Sucuri
-Surfbot
+Sucker
+Suzuran
+Szukacz
 T0PHackTeam
-T8Abot
-Telesoft
-Titan
+Teleport
+TeleportPro
+Telesphoreo
+Telesphorep
+Toata
 Toweyabot
 Tracemyfile
-Turnitin
+Trendictionbot
 Typhoeus
-URLy.Warning
 Upflow
-VeriCiteCrawler
+VB Project
+Vagabondo
+VidibleScraper
 Voil
-WBSearchBot
+Voltron
+WASALive-Bot
+WEBDAV
+WISENutbot
+WPScan
 WWW-Collector-E
-WWW-Mechanize
-Web Collage
-Web Enhancer
+WWW::Mechanize
+Wallpapers/3.0
+WallpapersHD
 Web Fuck
-Web Sauger
+Web Pix
+Web Sucker
 WebAuto
-WebBandit
 WebFetch
-WebFuck
-WebImageCollector
-WebSauger
+WebReaper
+WebStripper
+WebSucker
+WebWhacker
 Webalta
+WebmasterWorldForumBot
 Webshag
 Website Quester
-Whack
-Whatweb
-WinHTTrack
-Wotbox
-Zauba
-ZmEu
+WebsiteExtractor
+Webster
+Who.is Bot
+Wonderbot
+Woobot
+Xaldon WebSpider
+YoudaoBot
+Zade
+ZoomBot
 ZoominfoBot
-archive.org_bot
-arquivo-web-crawler
+ZyBorg
+autoemailspider
 backlink-check
-cognitiveseo
 com.plumanalytics
-crawl.sogou.com
-crawler.feedback
 crawler4j
 demandbase-bot
-facebookscraper
-gopher
-ips-agent
+domainsproject.org
+eCatch
 lwp-request
-meanpathbot
+lwp-trivial
+netEstate NE Crawler
+oBot
 page scorer
 pcBrowser
-scan.lol
+plumanalytics
+polaris version
+probe-image-size
+seobility
+seoscanners
 serpstatbot
+sexsearcher
 sitechecker.pro
-tAkeOut
+spyfu
+sysscan
+voyagerx.com
 webmeup-crawler
-x09Mozilla
+xpymep1.exe