nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-11-18 10:09:17 +00:00

250 lines
2.6 KiB
Text

360Spider
ADmantX
ALittle Client
ASPSeek
Abonti
Aboundexbot
Acunetix
AfD-Verbotsverfahren
AiHitBot
Aipbot
AllSubmitter
Anthill
Apexoo
Aspiegel
Asterias
Attach
AwarioSmartBot
BBBike
BDFetch
BackWeb
Barkrowler
BatchFTP
BlackWidow
BlowFish
BotALot
Brandprotect
Brandwatch
Buck
Buddy
BuiltWith
CCBot
CensysInspect
CheTeam
CherryPicker
ChinaClaw
Cocolyzebot
Copier
Cosmos
Crawling at Home Project
CrazyWebCrawler
Crescent
Custo
DBLBot
DTS Agent
Demon
Digincore
Discobot
Discoverybot
Dispatch
DittoSpyder
DnyzBot
Download Wonder
ECCP/1.0
Exabot
Extreme Picture Finder
FHscan
FemtosearchBot
Fimap
FlashGet
FrontPage
Fuzz
GT::WWW
GalaxyBot
GetRight
Getintent
Go!Zilla
Grabber
Grafula
GrapeFX
GridBot
HTTP::Lite
HaosouSpider
Harvest
HybridBot
IDBTE4M
IDBot
Image Sucker
IndeedBot
InfoNaviRobot
Intelliseek
Iria
Jbrofuzz
Jetty
JustView
Kenjin Spider
LNSpiderguy
LieBaoFast
Lightspeedsystems
LinkScan
Linkbot
Linkdexbot
LinksManager
Lipperhey Spider
Lmspider
Ltx71
MFC_Tear_Sample
MIIxpc
MJ12bot
MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot
Masscan
Mata Hari
Meanpathbot
Mediatoolkitbot
Microsoft URL Control
Mister PiX
Mojeek
Mr.4x3
Msrabot
NICErsPRO
Name Intelligence
Net Vampire
NetAnts
Nettrack
Niki-bot
NimbleCrawler
Offline Explorer
OnCrawl
OpenLinkProfiler
OrangeBot
OrangeSpider
OutclicksBot
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
PageGrabber
Papa Foto
Pi-Monster
PleaseCrawl
ProWebWalker
Psbot
Pu_iN
Pump
PyCurl
QueryN Metasearch
RankActive
RankFlex
RankingBot
RankingBot2
Rankivabot
RankurBot
Re-re
RedesScrapy
RocketCrawler
SEOkicks-Robot
SEOprofiler
ScanAlert
Scanbot
ScoutJet
Scrapy
ScrepyBot
Searchestate
SemanticJuice
Semrush
SeobilityBot
Siphon
Site Sucker
SiteCheckerBotCrawler
Siteimprove
SlySearch
Snapbot
Snoopy
Sociscraper
Sogou web spider
Sosospider
Spanner
SputnikBot
Sqworm
Sucker
Sucuri
SuperBot
Szukacz
T8Abot
Telesphorep
The Intraformant
Thumbor
Titan
Toata
Trendiction
TurnitinBot
TwengaBot
Typhoeus
VB Project
Vacuum
Vagabondo
VelenPublicWebCrawler
Virusdie
Voil
WISENutbot
WPScan
WWW::Mechanize
WWWOFFLE
Wallpapers
Wallpapers/3.0
Web Collage
Web Pix
Web Sauger
WebAuto
WebBandit
WebCopier
WebGo IS
WebImageCollector
WebStripper
WebSucker
WebWhacker
WebZIP
WebmasterWorldForumBot
Webshag
Website Quester
WebsiteQuester
Webster
WinHTTrack
WiseGuys Robot
Wonderbot
Xenu
Zade
ZumBot
adscanner
archive.org_bot
autoemailspider
backlink-check
com.plumanalytics
dataforseo.com
evc-batch
heritrix
ips-agent
isitwp.com
meanpathbot
mediawords
netEstate NE Crawler
page scorer
pcBrowser
plumanalytics
scalaj-http
seoscanners
serpstatbot
sogouspider
spyfu
trendiction.com
trendiction.de
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler
webprosbot
x09Mozilla
x22Mozilla
zauba.io