nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-07-22 22:07:12 +00:00

250 lines
2.6 KiB
Text

360Spider
404enemy
ADmantX
AIBOT
Aboundexbot
AhrefsBot
AiHitBot
Aipbot
Alligator
Anarchie
Anarchy99
Anthill
Apexoo
Atomseobot
BDCbot
BLEXBot
Backlink-Ceck
BacklinkCrawler
Barkrowler
Battleztar Bazinga
BetaBot
Bigfoot
Bitacle
BlackWidow
BotALot
Buck
BuiltBotTough
Bullseye
BunnySlippers
BuzzSumo
CATExplorador
CCBot
CSHttp
CazoodleBot
Cegbfeieh
CensysInspect
CheTeam
CherryPicker
Claritybot
Cloud mapping
Collector
Copier
Cosmos
Crawling at Home Project
CrazyWebCrawler
DSearch
DataCha0s
Demon
Deusu
Digincore
Dirbuster
Discobot
DnBCrawler-Analytics
DomainCrawler
DomainStatsBot
Ecxi
EirGrabber
ExtractorPro
FemtosearchBot
Fimap
Firefox/7.0
Flunky
Foobot
Freeuploader
FyberSpider
GT::WWW
Genieo
GermCrawler
GetWeb
Gigabot
GoZilla
Grafula
GrapeshotCrawler
GridBot
HEADMasterSEO
HMView
HTMLparser
HTTrack
Heritrix
IDBTE4M
IDBot
IRLbot
Id-search
Intelliseek
Internet Ninja
InternetSeer
Iria
IstellaBot
Jbrofuzz
JennyBot
Jetty
Keybot Translation-Search-Machine
Keyword Density
LWP::Simple
Lanshanbot
Larbin
LeechGet
LibWeb
Libwhisker
Linkbot
LinksManager
Lipperhey Spider
Litemage_walker
MFC_Tear_Sample
MIIxpc
Mag-Net
MarkMonitor
Masscan
MauiBot
MegaIndex.ru
Mister PiX
Moblie Safari
Mojeek
Morfeus Fucking Scanner
Musobot
NPbot
Name Intelligence
Nameprotect
Navroad
Nessus
NetLyzer
NetZIP
Nibbler
Ninja
Octopus
Offline Explorer
OpenLinkProfiler
OpenVAS
Openvas
PageAnalyzer
PageThing.com
Papa Foto
Pavuk
Petalbot
Pixray
PleaseCrawl
Probethenet
PxBroker
PyCurl
RSSingBot
RankActiveLinkBot
Rankivabot
RealDownload
Reaper
RebelMouse
Recorder
RedesScrapy
RocketCrawler
Rogerbot
SBIder
SEOstats
SMTBot
SalesIntelligent
ScanAlert
Scanbot
Scrapy
ScrepyBot
SearchmetricsBot
Semrush
Seomoz
SiteLockSpider
SiteSucker
SmartDownload
Snapbot
Sociscraper
Sottopop
Spanner
Spinn3r
SputnikBot
Sqlmap
Sqlworm
Steeler
Swiftbot
T0PHackTeam
TeleportPro
Telesphoreo
Telesphorep
TightTwatBot
Titan
Toweyabot
Trendiction
Trendictionbot
Turnitin
TurnitinBot
URLy.Warning
V-BOT
VelenPublicWebCrawler
VeriCiteCrawler
VoidEYE
WASALive-Bot
WWW-Collector-E
WWW::Mechanize
WeSEE
Web Enhancer
Web Pix
Web Sauger
Web Sucker
WebCollage
WebEnhancer
WebFetch
WebPix
WebStripper
WebZIP
WebsiteExtractor
Webster
Whack
Whacker
Whatweb
WinHTTrack
Wonderbot
Woobot
Xenu
Zeus
Zitebot
ZmEu
ZoomBot
ZoominfoBot
arquivo-web-crawler
autoemailspider
backlink-check
cognitiveseo
crawl.sogou.com
dataforseo.com
eCatch
gopher
heritrix
instabid
internetVista monitor
lwp-request
magpie-crawler
meanpathbot
mediawords
netEstate NE Crawler
oBot
s1z.ru
satoristudio.net
scan.lol
seobility
seocompany.store
seoscanners
seostar
serpstatbot
sitechecker.pro
sogouspider
sp_auditbot
spyfu
tAkeOut
voyagerx.com
webmeup-crawler
x09Mozilla