nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-01-14 10:06:51 +00:00

100 lines
1 KiB
Text

404checker
ADmantX
Aipbot
Anarchie
Anarchy
Apexoo
Barkrowler
Bitacle
Black Hole
Brandprotect
BuiltWith
CATExplorador
Crawling at Home Project
Crescent
Custo
CyotekWebCopy
DTS Agent
Devil
Discobot
DittoSpyder
DomainAppender
Drip
ECCP/1.0
ExtLinksBot
FlashGet
GPTBot
GermCrawler
GetRight
Getintent
Go-Ahead-Got-It
Gotit
GrapeshotCrawler
HTTP::Lite
Haansoft
Havij
IDBTE4M
Id-search
Image Sucker
Iskanie
IstellaBot
Joomla
Lftp
Lightspeedsystems
LinkpadBot
Lmspider
MQQBrowser
Mag-Net
Majestic SEO
Mass Downloader
Mata Hari
Microsoft Data Access
NetMechanic
Niki-bot
Nmap
Nutch
OpenLinkProfiler
PECL::HTTP
Pixray
Proximic
RankActiveLinkBot
Re-re
Ripper
Rogerbot
SBIder
SEOkicks-Robot
ScrepyBot
Searchestate
SemrushBot
SentiBot
Seomoz
Sitebeam
Sociscraper
SpaceBison
Sucuri
TinyTestBot
Tracemyfile
Twice
URLy.Warning
Vagabondo
Voil
Voltron
WISENutbot
WebCopier
WebFetch
WebLeacher
WebSucker
WebWhacker
Whack
WinHTTrack
WiseGuys Robot
Wotbox
ZoominfoBot
demandbase-bot
eCatch
gopher
magpie-crawler
probe-image-size
sogouspider
sp_auditbot
voyagerx.com