nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-09-21 10:10:07 +00:00

250 lines
2.5 KiB
Text

01h4x.com
404enemy
AIBOT
ALittle Client
Aboundex
Acunetix
AfD-Verbotsverfahren
AhrefsBot
AiHitBot
AllSubmitter
AlphaBot
Ankit
Apexoo
Aspiegel
AwarioRssBot
BackStreet
BackWeb
BacklinkCrawler
Badass
Barkrowler
BatchFTP
Battleztar Bazinga
Black Hole
BlackWidow
BlowFish
Brandprotect
Buddy
BuiltBotTough
BunnySlippers
CCBot
CODE87
CazoodleBot
Cliqzbot
Collector
Crawling at Home Project
CrazyWebCrawler
Crescent
CyotekWebCopy
DBLBot
DSearch
DTS Agent
DatabaseDriverMysqli
Deusu
Disco
DittoSpyder
DnyzBot
DomainCrawler
Dotbot
Download Wonder
Drip
ECCP/1.0
EMail Siphon
EMail Wolf
Ebingbong
EirGrabber
Evil
Ezooms
FDM
FemtosearchBot
Fimap
FlashGet
Fuzz
FyberSpider
Fyrebot
G-i-g-a-b-o-t
GetWeb
Getintent
Go-Ahead-Got-It
Gotit
GrabNet
GrapeshotCrawler
GridBot
HTTP::Lite
HTTrack
Harvest
Havij
Humanlinks
HybridBot
IRLbot
Id-search
Image Sucker
IndeedBot
InfoTekies
Iskanie
JamesBOT
JetCar
Joomla
JustView
Jyxobot
Kenjin Spider
Keyword Density
Kozmosbot
Larbin
Leap
LexiBot
Linkdexbot
LinkextractorPro
LinkpadBot
Lipperhey
MJ12bot
MSIECrawler
MTRobot
Mail.RU_Bot
Majestic SEO
Masscan
Mb2345Browser
MeanPath Bot
MicroMessenger
Mister PiX
Moblie Safari
Musobot
Navroad
Needle
Nessus
Net Vampire
NetAnts
NetSpider
NetZIP
Netcraft
Netvibes
NextGenSearchBot
Nikto
Nimbostratus
Nuclei
Nutch
Octopus
Offline Explorer
OnCrawl
Openfind
OutclicksBot
OutfoxBot
PageAnalyzer
PageScorer
PageThing.com
Panscient
Pavuk
PeoplePal
Petalbot
Picscout
Picsearch
Pixray
ProPowerBot
ProWebWalker
Psbot
PxBroker
PyCurl
Quick-Crawler
RankActive
RankFlex
RankingBot
Rankivabot
RankurBot
RepoMonkey
SBIder
SEOkicks
SEOstats
SMTBot
SalesIntelligent
ScanAlert
Scanbot
ScoutJet
Scrapy
Screaming
Searchestate
SearchmetricsBot
SemrushBot
Seomoz
SiteCheckerBotCrawler
SiteSucker
Sitebeam
Siteimprove
Sitevigil
SmartDownload
Snapbot
Snoopy
Sogou web spider
Spammen
Spbot
Stripper
Sucker
SuperBot
Suzuran
Swiftbot
Telesoft
Thumbor
Toata
Trendiction
Turnitin
TwengaBot
UnisterBot
Vagabondo
VeriCiteCrawler
WASALive-Bot
WEBDAV
WPScan
WWW-Collector-E
WWW::Mechanize
Wallpapers
Web Collage
Web Enhancer
Web Fuck
Web Pix
Web Sauger
Web Sucker
WebCopier
WebFuck
WebGo IS
WebSauger
WebSucker
Webalta
WebmasterWorldForumBot
Website Quester
WebsiteExtractor
Whack
Whatweb
Who.is Bot
WinHTTrack
Wonderbot
Woobot
Wprecon
Zeus
ZoominfoBot
archive.org_bot
check1.exe
coccocbot-web
cognitiveseo
crawler4j
dataforseo.com
demandbase-bot
domainsproject.org
eCatch
facebookscraper
heritrix
instabid
ips-agent
isitwp.com
lwp-request
pcBrowser
ripz
scalaj-http
seostar
serpstatbot
sitechecker.pro
sp_auditbot
spyfu
sysscan
tAkeOut
trendiction.de
webpros.com
x09Mozilla