nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-09-23 22:07:22 +00:00

250 lines
2.6 KiB
Text

360Spider
ASPSeek
Abonti
Aboundex
AiHitBot
Anarchie
Anarchy
Anarchy99
Ankit
BDFetch
BackDoorBot
Backlink-Ceck
Barkrowler
BatchFTP
BetaBot
Bigfoot
Black Hole
BlackWidow
Blackboard
BlowFish
Boardreader
Bolt
BotALot
Brandwatch
Buddy
BuiltBotTough
BunnySlippers
CSHttp
Calculon
CheeseBot
ChinaClaw
Cloud mapping
Cocolyzebot
Collector
Copier
Copyscape
Craftbot
Crawling at Home Project
CrazyWebCrawler
Custo
DBLBot
DSearch
DTS Agent
DataCha0s
Deusu
Dirbuster
Disco
Dispatch
DittoSpyder
DnyzBot
DomainCrawler
DomainSigmaCrawler
Dotbot
Download Wonder
Drip
ECCP/1.0
EMail Wolf
Ebingbong
EroCrawler
Express WebPictures
ExtractorPro
Ezooms
FHscan
FemtosearchBot
FlashGet
Flunky
Freeuploader
FrontPage
Fuzz
Fyrebot
G-i-g-a-b-o-t
GalaxyBot
Genieo
GermCrawler
Getintent
GoZilla
GrabNet
Grafula
HTMLparser
HTTP::Lite
HaosouSpider
Harvest
Havij
Heritrix
Hloader
Humanlinks
IDBTE4M
IDBot
Iblog
Id-search
IlseBot
Image Sucker
IndeedBot
InterGET
Iria
Iskanie
JennyBot
JetCar
Jetty
JikeSpider
Kinza
LWP::Simple
Lanshanbot
LeechGet
Lftp
Libwhisker
LieBaoFast
Linkdexbot
LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Spider
MFC_Tear_Sample
MIIxpc
MJ12bot
MQQBrowser
MSFrontPage
Mail.RU_Bot
Majestic12
MarkMonitor
MarkWatch
Mata Hari
MegaIndex.ru
MicroMessenger
Microsoft Data Access
Moblie Safari
Msrabot
NICErsPRO
Name Intelligence
Nameprotect
NetZIP
Netcraft
Netvibes
NextGenSearchBot
Nibbler
Niki-bot
Ninja
Octopus
Offline Explorer
Offline Navigator
OpenLinkProfiler
Openfind
OrangeBot
PECL::HTTP
POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer
PageGrabber
PageThing.com
Papa Foto
Pi-Monster
Picscout
PictureFinder
Piepmatz
Pimonster
Psbot
Pump
Quick-Crawler
RankActive
Ripper
Rogerbot
SEOkicks-Robot
SEOprofiler
SMTBot
Scrapy
Screaming
Searchestate
SentiBot
Site Sucker
SiteExplorer
SiteLockSpider
SiteSucker
Sitevigil
Snoopy
Sociscraper
Sottopop
Spanner
Spbot
Sqworm
Sucuri
Suzuran
T0PHackTeam
Teleport
TeleportPro
Telesoft
Telesphoreo
TheNomad
Thumbor
Titan
Toata
Tracemyfile
True_Robot
Turingos
Turnitin
URLy.Warning
UnisterBot
Upflow
VB Project
VCI
VelenPublicWebCrawler
VidibleScraper
VoidEYE
WASALive-Bot
WWW::Mechanize
Wallpapers
WallpapersHD
WeSEE
Web Sucker
WebCopier
WebFetch
WebImageCollector
WebSucker
WebWhacker
WebZIP
WebmasterWorldForumBot
WebsiteQuester
Webster
Widow
WinHTTrack
WiseGuys Robot
Wonderbot
Woobot
Xaldon WebSpider
Zade
Zauba
Zeus
ZoomBot
arquivo.pt
backlink-check
coccocbot-web
crawl.sogou.com
crawler4j
demandbase-bot
facebookscraper
isitwp.com
meanpathbot
mediawords
muhstik-scan
page scorer
plumanalytics
s1z.ru
satoristudio.net
scalaj-http
seobility
sexsearcher
sitechecker.pro
sogouspider
webprosbot
x09Mozilla
x22Mozilla
xpymep1.exe