nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-07-31 10:09:10 +00:00

250 lines
2.5 KiB
Text

01h4x.com
404checker
ASPSeek
Aboundex
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
AiHitBot
AllSubmitter
Alligator
AlphaBot
Anarchie
Anarchy99
Aspiegel
Asterias
Atomseobot
AwarioRssBot
BBBike
Bandit
BatchFTP
Battleztar Bazinga
Bitacle
BlackWidow
Boardreader
Bolt
BotALot
Brandwatch
Buck
BuiltWith
Bullseye
BuzzSumo
CATExplorador
CSHttp
Chlooe
Cocolyzebot
Cogentbot
Collector
Crawling at Home Project
CrunchBot
Curious
Custo
DBLBot
DSearch
DataCha0s
Discobot
DnyzBot
DomCopBot
DomainAppender
Domains Project
Dotbot
Download Wonder
Dragonfly
EMail Wolf
EasyDL
Ebingbong
Ecxi
EroCrawler
EyeNetIE
FemtosearchBot
G-i-g-a-b-o-t
Genieo
Getintent
Gigabot
GrabNet
GrapeFX
GrapeshotCrawler
GridBot
Havij
Heritrix
Hloader
Humanlinks
HybridBot
IDBTE4M
InterGET
Iskanie
JamesBOT
Jbrofuzz
JennyBot
Jetty
Joomla
Jorgee
Kenjin Spider
Kozmosbot
LWP::Simple
LibWeb
Libwhisker
Likse
LinkScan
Linkbot
LinkextractorPro
LinksManager
LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Litemage_walker
Ltx71
Mag-Net
Magnet
Mail.RU_Bot
MarkWatch
MegaIndex.ru
Metauri
Moblie Safari
Mr.4x3
Msrabot
Musobot
Nameprotect
Navroad
NearSite
NetAnts
NetLyzer
NetMechanic
Nettrack
NextGenSearchBot
Niki-bot
Ninja
Not
Nutch
Openvas
PECL::HTTP
PHPCrawl
Page Analyzer
PageAnalyzer
PageGrabber
PageScorer
PageThing.com
Pandalytics
Pavuk
PeoplePal
Picscout
Piepmatz
Pixray
PleaseCrawl
Probethenet
Pu_iN
Pump
PxBroker
QueryN Metasearch
RankActive
RankingBot
RankingBot2
RankurBot
ReGet
RealDownload
Recorder
RedesScrapy
Ripper
RocketCrawler
Rogerbot
SEOprofiler
SalesIntelligent
ScanAlert
Screaming
ScrepyBot
SentiBot
SiteExplorer
SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam
SmartDownload
Snapbot
Sociscraper
Sosospider
Sottopop
SpaceBison
Spanner
Spbot
Sqlworm
Sqworm
Steeler
Sucuri
SuperBot
SurveyBot
Swiftbot
TeleportPro
Telesphoreo
Telesphorep
Titan
Toweyabot
Trendictionbot
Turnitin
Twice
Typhoeus
Upflow
V-BOT
VCI
Vacuum
Vagabondo
VeriCiteCrawler
VoidEYE
Voil
WBSearchBot
WEBDAV
WPScan
WallpapersHD
WeSEE
Web Auto
Web Collage
Web Enhancer
Web Fuck
Web Sauger
WebCollage
WebEnhancer
WebPix
WebReaper
WebStripper
WebWhacker
WebZIP
Website Quester
WebsiteQuester
Webster
Whatweb
Who.is Bot
Wonderbot
Woobot
Xaldon_WebSpider
Xenu
Zermelo
Zeus
ZumBot
arquivo-web-crawler
arquivo.pt
autoemailspider
cognitiveseo
com.plumanalytics
crawler4j
dataforseo.com
domainsproject.org
eCatch
facebookscraper
heritrix
instabid
isitwp.com
lwp-request
lwp-trivial
magpie-crawler
meanpathbot
netEstate NE Crawler
page scorer
plumanalytics
ripz
satoristudio.net
scalaj-http
seocompany.store
sysscan
tAkeOut
webgains-bot
webmeup-crawler
zauba.io