nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-09-15 10:09:01 +00:00

250 lines
2.6 KiB
Text

01h4x.com
360Spider
Aboundexbot
Acunetix
AfD-Verbotsverfahren
AhrefsBot
Aipbot
Alexibot
Anarchie
Apexoo
Asterias
BackStreet
BacklinkCrawler
Badass
Bigfoot
Black Hole
BlackWidow
BotALot
Brandprotect
Brandwatch
BunnySlippers
Calculon
CazoodleBot
CheeseBot
CherryPicker
ChinaClaw
Cocolyzebot
Copier
CopyRightCheck
Cosmos
Craftbot
Crawling at Home Project
Curious
Custo
CyotekWebCopy
DIIbot
DSearch
DataCha0s
Devil
Digincore
DigitalPebble
Dirbuster
DittoSpyder
DomCopBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
EasyDL
Ecxi
EroCrawler
Express WebPictures
ExtractorPro
EyeNetIE
Firefox/7.0
FlashGet
Foobot
Freeuploader
FyberSpider
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
Genieo
GetRight
Getintent
Go-Ahead-Got-It
Gotit
HEADMasterSEO
HTMLparser
HTTP::Lite
HTTrack
Harvest
Hloader
IDBTE4M
IDBot
Iblog
Image Fetch
Image Sucker
IndeedBot
Indy Library
InfoTekies
Intelliseek
InterGET
InternetSeer
IstellaBot
Jbrofuzz
JennyBot
JetCar
Jetty
Kenjin Spider
Keyword Density
LNSpiderguy
Leap
Lftp
LibWeb
Libwhisker
LieBaoFast
LinkScan
LinkextractorPro
LinksManager
Lipperhey Spider
Ltx71
MFC_Tear_Sample
MIDown tool
MQQBrowser
Mag-Net
Magnet
Mail.RU_Bot
Majestic-SEO
MarkMonitor
MauiBot
MeanPath Bot
MicroMessenger
Mister PiX
Moblie Safari
Name Intelligence
Nameprotect
Navroad
NearSite
Nessus
NetMechanic
NetSpider
Netvibes
Nuclei
Octopus
Offline Explorer
Offline Navigator
Openfind
OutclicksBot
PHPCrawl
POE-Component-Client-HTTP
PageThing.com
Panscient
PeoplePal
Picsearch
PictureFinder
Pimonster
PleaseCrawl
ProPowerBot
ProWebWalker
Pu_iN
PxBroker
Quick-Crawler
RankFlex
RankingBot
Re-re
RealDownload
Recorder
RedesScrapy
RocketCrawler
SEOkicks-Robot
SEOprofiler
Scanbot
Scrapy
Semrush
SemrushBot
Shodan
Site Sucker
SiteCheckerBotCrawler
SiteSnagger
SiteSucker
Sitebeam
Siteimprove
Snake
Snapbot
Snoopy
Sottopop
Spammen
Spbot
Sqlworm
Steeler
Stripper
SuperHTTP
Surfbot
Szukacz
TeleportPro
Telesoft
TheNomad
TightTwatBot
Toata
Toweyabot
Trendiction
True_Robot
Turnitin
TwengaBot
Twice
URLy.Warning
Upflow
VCI
Vacuum
VeriCiteCrawler
VidibleScraper
WEBDAV
WISENutbot
WWW-Collector-E
WWWOFFLE
WallpapersHD
WeSEE
Web Fuck
Web Sucker
WebAuto
WebEnhancer
WebFuck
WebGo IS
WebImageCollector
WebPix
WebReaper
WebZIP
Webalta
WebmasterWorldForumBot
Webshag
WebsiteQuester
Whack
Whatweb
Widow
WinHTTrack
Wonderbot
Wotbox
Xaldon_WebSpider
YoudaoBot
Zauba
Zermelo
Zeus
Zitebot
ZoominfoBot
ZyBorg
archive.org_bot
arquivo.pt
cah.io.community
clark-crawler
cognitiveseo
demandbase-bot
evc-batch
facebookscraper
instabid
lwp-trivial
magpie-crawler
mediawords
muhstik-scan
netEstate NE Crawler
ripz
s1z.ru
satoristudio.net
seoscanners
sexsearcher
sitechecker.pro
siteripz
spyfu
ubermetrics-technologies.com
webprosbot
x22Mozilla