V4.2021.10.2670 [ci skip]

dev-tools/test_units/random-bots-for-test.list

@@ -1,250 +1,250 @@
 01h4x.com
-360Spider
-404checker
-ASPSeek
-AfD-Verbotsverfahren
-AhrefsBot
+404enemy
+AIBOT
+ALittle Client
 AiHitBot
-Aipbot
 Alexibot
+Alligator
 AlphaBot
 Anarchy
-Anthill
-Aspiegel
-Asterias
-Attach
-AwarioRssBot
-BBBike
-BDFetch
-BLEXBot
+Anarchy99
+Ankit
 BackDoorBot
+BackWeb
 Backlink-Ceck
+BacklinkCrawler
 Badass
+BatchFTP
+Bitacle
 Black Hole
-Blow
+Blackboard
 Boardreader
-Brandwatch
+Bolt
+BotALot
+Buck
 Buddy
-BuzzSumo
 CATExplorador
 CODE87
 CazoodleBot
+Cegbfeieh
 CensysInspect
+Chlooe
 Claritybot
 Cloud mapping
 Cocolyzebot
-Cogentbot
-Copier
-Copyscape
+Collector
+CopyRightCheck
+Craftbot
 Crawling at Home Project
-CrazyWebCrawler
-Crescent
+CrunchBot
 Custo
-CyotekWebCopy
-DSearch
 DataCha0s
-DatabaseDriverMysqli
-Devil
-Digincore
-DigitalPebble
-Dirbuster
-Discoverybot
 Dispatch
-DomainAppender
+DnBCrawler-Analytics
+DnyzBot
+DomCopBot
+DomainSigmaCrawler
 Dotbot
-EasyDL
-Ecxi
-EroCrawler
+Drip
+Ebingbong
+EirGrabber
 Evil
-Express WebPictures
 Extractor
-ExtractorPro
+Extreme Picture Finder
 Ezooms
+FDM
 FHscan
+Fimap
+FlashGet
 Foobot
-Fyrebot
-G-i-g-a-b-o-t
-GT::WWW
+FrontPage
+FyberSpider
 GalaxyBot
-Genieo
 GetRight
+GetWeb
+Getintent
 Gigabot
-Go!Zilla
-Go-Ahead-Got-It
-Grabber
+GoZilla
+Gotit
+Grafula
 GrapeFX
 GrapeshotCrawler
+GridBot
+HEADMasterSEO
 HMView
-HTMLparser
-HTTrack
-HaosouSpider
+HTTP::Lite
 Harvest
-Heritrix
-Hloader
-IDBTE4M
+IRLbot
 Id-search
-Image Sucker
-IndeedBot
-InfoNaviRobot
+IlseBot
+Image Fetch
 InfoTekies
-Iria
 Iskanie
-JOC Web Spider
+Jbrofuzz
 JikeSpider
 Joomla
-JustView
-Jyxobot
+Kenjin Spider
+Kozmosbot
 LNSpiderguy
+LeechFTP
+Lftp
 LibWeb
 Libwhisker
 LieBaoFast
 Lightspeedsystems
 Likse
-LinkScan
+LinkWalker
 LinkextractorPro
 LinkpadBot
 LinksManager
 LinqiaMetadataDownloaderBot
 LinqiaRSSBot
+LinqiaScrapeBot
+Litemage_walker
+Ltx71
 MFC_Tear_Sample
-MIDown tool
-MIIxpc
 MJ12bot
-MQQBrowser
-MSFrontPage
-MSIECrawler
+Magnet
 Majestic SEO
-Majestic12
-MarkMonitor
+Masscan
+MauiBot
+Mb2345Browser
+MeanPath Bot
 Meanpathbot
-MegaIndex.ru
-Microsoft URL Control
+MicroMessenger
 Mister PiX
-Mojeek
-Mojolicious
 Mr.4x3
-NICErsPRO
-NPbot
-NearSite
-Netvibes
-Nimbostratus
-Ninja
-Nmap
-Nuclei
-Openvas
+Nameprotect
+NetMechanic
+Octopus
+OrangeBot
 OutclicksBot
-PECL::HTTP
+PHPCrawl
+POE-Component-Client-HTTP
 PageGrabber
-PageScorer
-PeoplePal
-Pi-Monster
-PictureFinder
-Piepmatz
+PageThing.com
+Papa Foto
+Petalbot
+Picscout
 Pimonster
+PleaseCrawl
+ProWebWalker
 Probethenet
-Psbot
+Pu_iN
 Pump
+Quick-Crawler
 RSSingBot
-RankActiveLinkBot
-RankFlex
-RankingBot2
+RankActive
 Rankivabot
-Re-re
+ReGet
 RealDownload
 Reaper
 RebelMouse
-RepoMonkey
-Ripper
 Rogerbot
-SISTRIX
-Scrapy
+SEOkicks
+SEOkicks-Robot
+SEOlyticsCrawler
+SEOprofiler
+SMTBot
+Screaming
 ScreenerBot
-Semrush
+SemrushBot
+SentiBot
 SeoSiteCheckup
-Siphon
-Site Sucker
-SiteLockSpider
-Siteimprove
+SiteExplorer
+SiteSnagger
 Sitevigil
-Snapbot
-Sociscraper
-Sosospider
+SlySearch
+SmartDownload
+Sogou web spider
 Sottopop
-Spammen
-Spinn3r
-SputnikBot
-Sqlmap
-Sqlworm
-Sqworm
-Sucuri
+SpaceBison
+SpankBot
+Spbot
 SuperBot
-SuperHTTP
 Swiftbot
-T0PHackTeam
-Teleport
+Telesphoreo
+Telesphorep
 The Intraformant
-Thumbor
-Turnitin
-TurnitinBot
+TightTwatBot
+Toata
+Toweyabot
+Tracemyfile
+Trendictionbot
+True_Robot
 TwengaBot
-URLy Warning
-UnisterBot
-V-BOT
-VidibleScraper
+URLy.Warning
+Upflow
+VB Project
+VCI
+VelenPublicWebCrawler
+Virusdie
+Voltron
 WASALive-Bot
-WISENutbot
+WEBDAV
+WPScan
 WWW-Collector-E
-Web Enhancer
+WWWOFFLE
+Wallpapers/3.0
+Web Auto
+Web Collage
 Web Fuck
+Web Pix
 Web Sauger
-WebAuto
+Web Sucker
 WebBandit
-WebCopier
-WebEnhancer
+WebCollage
 WebFetch
 WebGo IS
-WebImageCollector
 WebPix
-WebReaper
-WebSauger
-WebWhacker
-WebZIP
-Webalta
+WebStripper
+WebSucker
 WebmasterWorldForumBot
 Webshag
-WebsiteExtractor
 WebsiteQuester
-Whacker
+Webster
+Whack
 Whatweb
-Who.is Bot
-Wotbox
+WinHTTrack
+WiseGuys Robot
+Wonderbot
+Woobot
 Wprecon
-Xaldon WebSpider
 Xaldon_WebSpider
-Xenu
+YoudaoBot
 Zade
-Zauba
-Zermelo
 Zeus
 Zitebot
-check1.exe
-clark-crawler
+ZoomBot
+ZoominfoBot
+archive.org_bot
+arquivo.pt
+autoemailspider
+backlink-check
+cah.io.community
 coccocbot-web
-dataforseo.com
-domainsproject.org
+com.plumanalytics
+crawl.sogou.com
+crawler4j
 evc-batch
 facebookscraper
-isitwp.com
+heritrix
+instabid
+internetVista monitor
+lwp-trivial
 magpie-crawler
-meanpathbot
-page scorer
-pcBrowser
-probe-image-size
-seostar
+mediawords
+oBot
+plumanalytics
+polaris version
+satoristudio.net
+scalaj-http
+scan.lol
 sexsearcher
-sitechecker.pro
-siteripz
-spyfu
-ubermetrics-technologies.com
+sysscan
+tAkeOut
+trendiction.com
+trendiction.de
 voyagerx.com
-x22Mozilla
+webpros.com
+zauba.io