nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-01-07 10:09:33 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404enemy
80legs
ADmantX
AIBOT
ASPSeek
Aboundex
Aboundexbot
AfD-Verbotsverfahren
AiHitBot
AlphaBot
Asterias
Attach
AwarioRssBot
BBBike
BDCbot
BackWeb
Battleztar Bazinga
Bigfoot
Black Hole
BlackWidow
Brandprotect
Brandwatch
Buck
BuiltWith
CODE87
CSHttp
CazoodleBot
Cegbfeieh
CensysInspect
ChinaClaw
Claritybot
Cloud mapping
Cocolyzebot
Collector
CyotekWebCopy
DTS Agent
Demon
Devil
Digincore
Dirbuster
Disco
Discobot
Discoverybot
DnBCrawler-Analytics
DnyzBot
DomCopBot
DomainSigmaCrawler
Domains Project
Dragonfly
ECCP/1.0
EMail Siphon
Ebingbong
Ecxi
EirGrabber
Evil
Exabot
Extractor
Extreme Picture Finder
Ezooms
FDM
FemtosearchBot
Fimap
FlashGet
Foobot
Fuzz
FyberSpider
GT::WWW
GalaxyBot
GermCrawler
GetRight
Go-Ahead-Got-It
GoZilla
GrapeshotCrawler
HMView
HTMLparser
HTTrack
Haansoft
Havij
Heritrix
HybridBot
IDBTE4M
Iblog
Id-search
IlseBot
InfoNaviRobot
InfoTekies
Internet Ninja
InternetSeer
Iskanie
JamesBOT
JennyBot
JetCar
Joomla
Jorgee
Kozmosbot
Lanshanbot
Leap
LeechGet
Lftp
LinkScan
LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Lmspider
MFC_Tear_Sample
MIDown tool
MIIxpc
MQQBrowser
Mag-Net
Majestic SEO
Majestic-SEO
MarkMonitor
MarkWatch
MauiBot
Microsoft Data Access
Mr.4x3
NICErsPRO
NPbot
Navroad
NearSite
Needle
NetMechanic
Nettrack
Netvibes
NextGenSearchBot
Nibbler
Niki-bot
Nikto
NimbleCrawler
Nimbostratus
Nutch
Offline Explorer
OrangeSpider
PECL::HTTP
PHPCrawl
PageGrabber
PageScorer
Pandalytics
Panscient
Papa Foto
PeoplePal
Pi-Monster
Picscout
Picsearch
Piepmatz
Pixray
Probethenet
Pu_iN
PxBroker
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActive
RankurBot
RealDownload
RedesScrapy
RepoMonkey
RocketCrawler
SEOkicks-Robot
SISTRIX
ScanAlert
ScoutJet
ScreenerBot
ScrepyBot
Searchestate
SemrushBot
SeoSiteCheckup
Shodan
SiteExplorer
SiteSnagger
SiteSucker
Siteimprove
SlySearch
SmartDownload
Snake
Sociscraper
Sosospider
Spammen
Stripper
Sucuri
SuperBot
Suzuran
Swiftbot
Trendictionbot
TurnitinBot
Typhoeus
Upflow
VCI
Vagabondo
VelenPublicWebCrawler
Virusdie
Voil
Voltron
WBSearchBot
WWW-Mechanize
WWW::Mechanize
Wallpapers/3.0
WallpapersHD
WeSEE
Web Enhancer
Web Sucker
WebAuto
WebCollage
WebFuck
WebGo IS
WebPix
Webshag
WebsiteQuester
Whack
Whacker
Widow
WinHTTrack
WiseGuys Robot
Wonderbot
Wprecon
Xaldon WebSpider
Xaldon_WebSpider
Zade
Zeus
ZoominfoBot
backlink-check
cah.io.community
coccocbot-web
cognitiveseo
com.plumanalytics
crawl.sogou.com
dataforseo.com
demandbase-bot
facebookscraper
gopher
heritrix
internetVista monitor
ips-agent
mediawords
oBot
polaris version
satoristudio.net
scalaj-http
seobility
seoscanners
serpstatbot
sexsearcher
sitechecker.pro
spyfu
trendiction.de
webmeup-crawler
x22Mozilla
zauba.io
zgrab