nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-10-18 10:11:12 +00:00

250 lines
2.7 KiB
Text

360Spider
404checker
404enemy
ADmantX
Acunetix
AfD-Verbotsverfahren
AhrefsBot
Aipbot
AlphaBot
Anarchy99
Apexoo
Atomseobot
BBBike
BackStreet
BackWeb
Backlink-Ceck
BacklinkCrawler
Black Hole
Blow
BlowFish
Boardreader
Brandwatch
Buck
Buddy
BuiltBotTough
BuiltWith
Bullseye
CODE87
CazoodleBot
CheTeam
CheeseBot
Citoid
Cliqzbot
Cloud mapping
Cocolyzebot
Crawling at Home Project
CrazyWebCrawler
Crescent
Curious
DIIbot
DTS Agent
DatabaseDriverMysqli
Demon
Devil
Digincore
DigitalPebble
Dirbuster
Disco
Discoverybot
Dotbot
Drip
EMail Siphon
EMail Wolf
EasyDL
Ecxi
ExtractorPro
EyeNetIE
Fimap
Firefox/7.0
Fuzz
Fyrebot
GermCrawler
Gigabot
Go!Zilla
Go-Ahead-Got-It
GrabNet
Grabber
Grafula
HTMLparser
HTTP::Lite
Haansoft
HaosouSpider
Havij
Heritrix
IDBTE4M
IDBot
Iblog
Image Fetch
IndeedBot
Indy Library
InterGET
Internet Ninja
Jbrofuzz
Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine
Keyword Density
Kinza
LNSpiderguy
Lanshanbot
Larbin
LeechGet
Lftp
Libwhisker
Lightspeedsystems
LinqiaRSSBot
Lipperhey Spider
Litemage_walker
MIIxpc
MJ12bot
MQQBrowser
Majestic SEO
Majestic-SEO
Majestic12
Masscan
Mata Hari
Microsoft Data Access
Microsoft URL Control
Minefield
Moblie Safari
MolokaiBot
Mozlila
NICErsPRO
Nameprotect
NearSite
NetMechanic
NetSpider
NimbleCrawler
Nmap
Octopus
Offline Explorer
OpenLinkProfiler
OpenVAS
Openfind
OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP
PageGrabber
PageThing.com
Pavuk
Picsearch
Piepmatz
PleaseCrawl
Pockey
ProPowerBot
Pu_iN
Pump
PxBroker
QueryN Metasearch
Quick-Crawler
RankActive
RankActiveLinkBot
RankingBot
RankingBot2
RankurBot
Re-re
Reaper
RedesScrapy
RocketCrawler
Rogerbot
SBIder
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SEOstats
SalesIntelligent
ScanAlert
Scrapy
Screaming
ScreenerBot
SearchmetricsBot
Seekport
SemanticJuice
SentiBot
SeoSiteCheckup
SeobilityBot
SiteLockSpider
SiteSnagger
SiteSucker
SmartDownload
Snapbot
Sogou web spider
Sosospider
Sottopop
SpaceBison
Spammen
Spanner
Spinn3r
Sqworm
Steeler
Sucuri
SuperHTTP
SurveyBot
Suzuran
Telesphoreo
Thumbor
Toata
Toweyabot
Trendiction
Trendictionbot
TurnitinBot
TwengaBot
Typhoeus
URLy Warning
V-BOT
VB Project
VCI
Vacuum
Vagabondo
VeriCiteCrawler
Virusdie
Voil
Voltron
WISENutbot
WPScan
WWW-Mechanize
Wallpapers
WallpapersHD
Web Fetch
Web Sauger
WebCollage
WebEnhancer
WebStripper
WebWhacker
WebmasterWorldForumBot
WebsiteQuester
Webster
Whatweb
Widow
WiseGuys Robot
Wprecon
Xenu
Zade
Zauba
Zermelo
ZoomBot
arquivo-web-crawler
arquivo.pt
backlink-check
coccocbot
crawl.sogou.com
crawler.feedback
crawler4j
demandbase-bot
heritrix
internetVista monitor
ips-agent
iubenda-radar
lwp-trivial
magpie-crawler
mediawords
seostar
sitechecker.pro
sp_auditbot
sysscan
tAkeOut
trendiction.de
voyagerx.com
webmeup-crawler
webpros.com