nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-07-05 10:09:15 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404enemy
AIBOT
Acunetix
AfD-Verbotsverfahren
AiHitBot
AlphaBot
Apexoo
AwarioSmartBot
BDCbot
BDFetch
Backlink-Ceck
Badass
BlackWidow
Boardreader
Brandwatch
Buck
Buddy
Bullseye
BunnySlippers
CCBot
CazoodleBot
CherryPicker
Chlooe
Citoid
Cliqzbot
Cosmos
Craftbot
CrazyWebCrawler
CrunchBot
Custo
CyotekWebCopy
DBLBot
DIIbot
Devil
Digincore
DigitalPebble
Dirbuster
Discobot
Discoverybot
DittoSpyder
DomainAppender
DomainCrawler
Domains Project
Dotbot
Dragonfly
ECCP/1.0
EasyDL
Ecxi
EirGrabber
EroCrawler
Express WebPictures
ExtLinksBot
ExtractorPro
Fimap
FlashGet
Flunky
Freeuploader
FrontPage
FyberSpider
Fyrebot
G-i-g-a-b-o-t
GT::WWW
Genieo
Go-Ahead-Got-It
Gotit
GrabNet
GrapeFX
GrapeshotCrawler
Haansoft
HaosouSpider
Heritrix
Hloader
HybridBot
IDBTE4M
Iblog
IlseBot
IndeedBot
Indy Library
Intelliseek
InterGET
InternetSeer
Iria
Iskanie
JamesBOT
Jbrofuzz
JetCar
Jetty
Jorgee
Kinza
Lanshanbot
Larbin
Leap
Lftp
LibWeb
LieBaoFast
Lightspeedsystems
LinkScan
LinkWalker
LinkextractorPro
LinkpadBot
LinqiaMetadataDownloaderBot
Ltx71
MIDown tool
MJ12bot
MQQBrowser
MSIECrawler
Mail.RU_Bot
Majestic SEO
MarkWatch
MeanPath Bot
Meanpathbot
MegaIndex.ru
Metauri
MicroMessenger
Microsoft URL Control
Moblie Safari
Morfeus Fucking Scanner
Mozlila
NetLyzer
Netvibes
Nibbler
NimbleCrawler
Nimbostratus
Ninja
Octopus
OrangeBot
OutfoxBot
PHPCrawl
PageAnalyzer
PageGrabber
PageScorer
PageThing.com
Pandalytics
Pavuk
Pi-Monster
Picscout
Piepmatz
ProPowerBot
ProWebWalker
Probethenet
RankingBot
RankingBot2
RankurBot
ReGet
RebelMouse
RedesScrapy
Ripper
RocketCrawler
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler
SISTRIX
Scrapy
Screaming
ScreenerBot
ScrepyBot
SearchmetricsBot
SentiBot
Shodan
Site Sucker
SiteExplorer
SiteLockSpider
SiteSucker
Sitebeam
Siteimprove
Snoopy
Sociscraper
SpaceBison
SpankBot
Spbot
Steeler
SuperBot
SuperHTTP
Surfbot
Swiftbot
Szukacz
TeleportPro
The Intraformant
TightTwatBot
Toweyabot
Trendiction
TurnitinBot
URLy Warning
URLy.Warning
UnisterBot
V-BOT
Vagabondo
Virusdie
Voil
WASALive-Bot
WPScan
WWW::Mechanize
WeSEE
Web Enhancer
Web Fetch
Web Pix
Web Sucker
WebCollage
WebEnhancer
WebFetch
WebFuck
WebImageCollector
WebLeacher
WebSucker
WebWhacker
WebZIP
Webalta
WebmasterWorldForumBot
WebsiteExtractor
WebsiteQuester
Whatweb
Who.is Bot
Widow
Wonderbot
Woobot
Wotbox
Xaldon WebSpider
Xenu
Zermelo
Zeus
ZmEu
ZumBot
arquivo-web-crawler
cognitiveseo
com.plumanalytics
crawl.sogou.com
crawler.feedback
dataforseo.com
demandbase-bot
domainsproject.org
eCatch
facebookscraper
gopher
heritrix
internetVista monitor
iubenda-radar
linkdexbot
meanpathbot
muhstik-scan
netEstate NE Crawler
polaris version
ripz
s1z.ru
scalaj-http
seoscanners
sp_auditbot
trendiction.de
webgains-bot
x09Mozilla