nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-04-28 22:07:16 +00:00

250 lines
2.6 KiB
Text

01h4x.com
ASPSeek
AfD-Verbotsverfahren
AhrefsBot
AiHitBot
Alexibot
Alligator
AlphaBot
Anarchy
Aspiegel
AwarioRssBot
BBBike
BDCbot
BDFetch
BLEXBot
BackWeb
Backlink-Ceck
BatchFTP
Bitacle
Black Hole
Blackboard
BlowFish
Boardreader
Buck
Buddy
BuiltWith
CCBot
CensysInspect
CheeseBot
CherryPicker
Chlooe
Citoid
Cogentbot
Collector
Craftbot
Crawling at Home Project
CrunchBot
Curious
CyotekWebCopy
DBLBot
Demon
Devil
Dispatch
DittoSpyder
DnyzBot
Dotbot
Download Wonder
Dragonfly
EMail Wolf
EasyDL
Ebingbong
Ecxi
Exabot
ExtractorPro
Fimap
Flunky
Foobot
GT::WWW
Go!Zilla
Gotit
Grabber
Grafula
GrapeFX
HEADMasterSEO
HTTP::Lite
Havij
Heritrix
Hloader
Humanlinks
IRLbot
Id-search
Image Fetch
Indy Library
InfoNaviRobot
InfoTekies
Internet Ninja
InternetSeer
JOC Web Spider
Jbrofuzz
JennyBot
JikeSpider
Joomla
Keybot Translation-Search-Machine
Keyword Density
Kinza
Kozmosbot
Leap
LeechFTP
LexiBot
Lftp
Lightspeedsystems
Likse
LinkScan
LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey
MIIxpc
MJ12bot
MQQBrowser
MTRobot
Majestic-SEO
Majestic12
MarkMonitor
Mass Downloader
Masscan
Mb2345Browser
MeanPath Bot
Meanpathbot
MegaIndex.ru
Metauri
MicroMessenger
Microsoft Data Access
Microsoft URL Control
Moblie Safari
Mojeek
Mojolicious
MolokaiBot
Mozlila
Msrabot
Name Intelligence
Needle
Net Vampire
NetLyzer
NetSpider
Netcraft
NextGenSearchBot
NimbleCrawler
Nimbostratus
Nmap
Offline Navigator
OnCrawl
Openfind
OrangeBot
OrangeSpider
OutfoxBot
PageScorer
Pandalytics
Panscient
Papa Foto
Picsearch
PictureFinder
Pimonster
ProPowerBot
Pu_iN
PyCurl
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankFlex
RankingBot
RankingBot2
RealDownload
Reaper
RebelMouse
RedesScrapy
Ripper
Rogerbot
SBIder
SEOkicks-Robot
SISTRIX
ScanAlert
Scrapy
Screaming
Semrush
SemrushBot
SeobilityBot
SiteCheckerBotCrawler
SiteExplorer
SiteSnagger
SiteSucker
Sitebeam
Sitevigil
Snapbot
Snoopy
SocialRankIOBot
Sogou web spider
SpaceBison
Spbot
Sqlmap
Steeler
Stripper
Sucuri
Surfbot
Thumbor
Turingos
Turnitin
TurnitinBot
URLy.Warning
UnisterBot
V-BOT
VCI
Vacuum
Vagabondo
WPScan
WWW-Mechanize
Web Enhancer
Web Sauger
WebCollage
WebFetch
WebFuck
WebGo IS
WebImageCollector
WebPix
WebReaper
WebSauger
WebStripper
WebZIP
WebsiteQuester
Whatweb
WinHTTrack
Woobot
Xaldon WebSpider
Xaldon_WebSpider
Zauba
Zeus
ZoominfoBot
ZyBorg
archive.org_bot
arquivo.pt
autoemailspider
com.plumanalytics
crawler.feedback
demandbase-bot
domainsproject.org
evc-batch
facebookscraper
gopher
instabid
ips-agent
isitwp.com
lwp-trivial
meanpathbot
muhstik-scan
netEstate NE Crawler
pcBrowser
polaris version
scan.lol
seobility
serpstatbot
sexsearcher
sogouspider
spyfu
tAkeOut
trendiction.de
webgains-bot
x09Mozilla
x22Mozilla