nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-10-01 22:07:13 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404checker
80legs
ASPSeek
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
Anarchie
Anarchy
Aspiegel
Atomseobot
AwarioSmartBot
BDCbot
BDFetch
Backlink-Ceck
BatchFTP
Bigfoot
Black Hole
Brandwatch
Buck
Buddy
BuiltBotTough
BuiltWith
CCBot
CODE87
CSHttp
CensysInspect
CheTeam
ChinaClaw
Cliqzbot
Cloud mapping
Cogentbot
Craftbot
Crescent
DBLBot
Demon
Deusu
Devil
DigitalPebble
Dirbuster
DomainCrawler
DomainSigmaCrawler
Domains Project
Dragonfly
ECCP/1.0
EMail Wolf
EasyDL
EirGrabber
Evil
Exabot
Express WebPictures
EyeNetIE
FDM
Fimap
Firefox/7.0
Freeuploader
FrontPage
FyberSpider
G-i-g-a-b-o-t
GT::WWW
Genieo
Gigabot
Grabber
Grafula
GrapeshotCrawler
HEADMasterSEO
HTMLparser
Haansoft
Havij
HonoluluBot
IRLbot
Iblog
Image Fetch
IndeedBot
Indy Library
Iria
Iskanie
IstellaBot
JOC Web Spider
JamesBOT
Jbrofuzz
JennyBot
Jetty
JustView
Kenjin Spider
Keyword Density
Kozmosbot
LNSpiderguy
LWP::Simple
Lanshanbot
LeechFTP
LexiBot
LieBaoFast
LinkScan
LinkWalker
LinkextractorPro
LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Lmspider
Ltx71
MIIxpc
MSIECrawler
MTRobot
Magnet
Majestic SEO
Majestic-SEO
MarkMonitor
Microsoft URL Control
Minefield
Mister PiX
Mojeek
Mr.4x3
Msrabot
Musobot
NPbot
Nameprotect
NearSite
Needle
Net Vampire
NetAnts
NetLyzer
NetMechanic
NetZIP
NextGenSearchBot
Nikto
Not
Nuclei
Nutch
Offline Explorer
OrangeBot
OutfoxBot
PHPCrawl
Page Analyzer
PageGrabber
PageThing.com
Pandalytics
Petalbot
Pi-Monster
Picscout
Pimonster
ProPowerBot
ProWebWalker
Pu_iN
Pump
QueryN Metasearch
RankFlex
ReGet
RealDownload
RebelMouse
RepoMonkey
Ripper
SEOkicks
SEOlyticsCrawler
SalesIntelligent
ScoutJet
Scrapy
ScrepyBot
SearchmetricsBot
SemrushBot
SeoSiteCheckup
Shodan
Siphon
SiteExplorer
SiteLockSpider
SiteSucker
Siteimprove
Sitevigil
Sosospider
Sottopop
Spammen
Spanner
SputnikBot
Sqlworm
Sqworm
Stripper
Sucker
Surfbot
Suzuran
Teleport
TheNomad
TightTwatBot
Toata
Trendiction
Trendictionbot
True_Robot
Turingos
Turnitin
VB Project
Vacuum
VeriCiteCrawler
VidibleScraper
Virusdie
Voil
WASALive-Bot
WWW::Mechanize
Web Fetch
WebBandit
WebFetch
WebImageCollector
WebReaper
WebSauger
WebStripper
WebZIP
Webalta
Webshag
Website Quester
Widow
Wonderbot
Woobot
Wprecon
Zade
Zitebot
ZumBot
ZyBorg
arquivo.pt
autoemailspider
backlink-check
check1.exe
clark-crawler
crawler.feedback
dataforseo.com
domainsproject.org
eCatch
facebookscraper
heritrix
internetVista monitor
isitwp.com
iubenda-radar
mediawords
netEstate NE Crawler
page scorer
polaris version
probe-image-size
ripz
s1z.ru
scalaj-http
scan.lol
seoscanners
serpstatbot
sitechecker.pro
sogouspider
sp_auditbot
sysscan
trendiction.com
ubermetrics-technologies.com
voyagerx.com
x09Mozilla
zauba.io
zgrab