nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-06-14 22:07:15 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404checker
ADmantX
AIBOT
ASPSeek
Aboundex
AhrefsBot
Aipbot
Alexibot
Anarchie
Anarchy99
Ankit
Anthill
Apexoo
AwarioSmartBot
BLEXBot
BacklinkCrawler
Barkrowler
BetaBot
Bitacle
BlackWidow
Boardreader
Buddy
BuiltWith
BunnySlippers
CODE87
CSHttp
Calculon
CazoodleBot
CensysInspect
CheTeam
Chlooe
Cliqzbot
Cocolyzebot
Copier
Cosmos
Crawling at Home Project
CyotekWebCopy
DSearch
Dirbuster
Discobot
DnyzBot
DomainCrawler
Domains Project
Dotbot
Dragonfly
EMail Siphon
EasyDL
EroCrawler
Evil
ExtLinksBot
EyeNetIE
FHscan
FemtosearchBot
Foobot
FrontPage
Fuzz
GT::WWW
GalaxyBot
GermCrawler
GetWeb
Getintent
Go-Ahead-Got-It
GoZilla
Gotit
Grabber
Grafula
GrapeshotCrawler
HMView
HTTP::Lite
Haansoft
HaosouSpider
Harvest
Heritrix
Humanlinks
IRLbot
Image Sucker
IndeedBot
Indy Library
Iria
Iskanie
IstellaBot
JetCar
Jetty
Jyxobot
Keybot Translation-Search-Machine
Keyword Density
Kinza
LWP::Simple
Lanshanbot
Larbin
LeechFTP
LibWeb
LieBaoFast
Lightspeedsystems
LinkScan
LinkextractorPro
LinkpadBot
LinqiaScrapeBot
Litemage_walker
Lmspider
Ltx71
MJ12bot
MSIECrawler
MTRobot
Magnet
Majestic SEO
MarkMonitor
Mass Downloader
Mata Hari
Mb2345Browser
Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Metauri
Minefield
Mojeek
Morfeus Fucking Scanner
Mozlila
Msrabot
NICErsPRO
Name Intelligence
Nameprotect
Nessus
NetZIP
Netvibes
Nibbler
Niki-bot
NimbleCrawler
Nmap
Nuclei
OpenLinkProfiler
OpenVAS
Openfind
OrangeBot
PageAnalyzer
PageThing.com
Panscient
Papa Foto
Pavuk
Petalbot
Picsearch
PictureFinder
Pimonster
PleaseCrawl
Pockey
Psbot
Pump
PxBroker
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankingBot2
RankurBot
Rogerbot
SEOkicks
SEOkicks-Robot
SEOprofiler
SEOstats
SISTRIX
SalesIntelligent
ScanAlert
ScreenerBot
Semrush
Seomoz
Siphon
Site Sucker
SiteCheckerBotCrawler
Snapbot
SocialRankIOBot
Sogou web spider
Sosospider
SpaceBison
Spanner
Spinn3r
Sqlmap
Sucuri
Suzuran
T0PHackTeam
TeleportPro
Telesoft
Telesphoreo
Thumbor
Turingos
TurnitinBot
TwengaBot
Twice
URLy.Warning
VB Project
Vagabondo
VelenPublicWebCrawler
VidibleScraper
Virusdie
Voil
WBSearchBot
WEBDAV
WWW-Mechanize
WWWOFFLE
Wallpapers/3.0
WallpapersHD
Web Auto
Web Collage
Web Fuck
Web Pix
Web Sauger
WebCopier
WebFetch
WebImageCollector
WebStripper
WebSucker
WebWhacker
WebZIP
WebmasterWorldForumBot
Whack
Whacker
Wonderbot
Woobot
Xaldon WebSpider
Zeus
ZoominfoBot
ZumBot
cognitiveseo
crawl.sogou.com
crawler4j
dataforseo.com
demandbase-bot
domainsproject.org
evc-batch
facebookscraper
gopher
heritrix
instabid
linkdexbot
lwp-request
lwp-trivial
magpie-crawler
oBot
pcBrowser
probe-image-size
ripz
satoristudio.net
scalaj-http
sp_auditbot
sysscan
trendiction.com
trendiction.de
xpymep1.exe
zauba.io