nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-11-18 22:07:11 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404checker
ADmantX
AIBOT
ASPSeek
Abonti
AfD-Verbotsverfahren
AiHitBot
AlphaBot
Anarchie
Anarchy99
Ankit
Anthill
Aspiegel
Atomseobot
Attach
BDCbot
BackStreet
BacklinkCrawler
Bandit
BetaBot
Black Hole
BlackWidow
Boardreader
BuiltWith
Cegbfeieh
CheeseBot
CherryPicker
Citoid
Claritybot
Cogentbot
Copier
CopyRightCheck
Cosmos
Crawling at Home Project
CyotekWebCopy
DSearch
DatabaseDriverMysqli
Deusu
Devil
Digincore
DigitalPebble
Disco
DittoSpyder
DnBCrawler-Analytics
DnyzBot
DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Domains Project
Dotbot
Download Wonder
Drip
EasyDL
Ecxi
Evil
Exabot
Extractor
Extreme Picture Finder
EyeNetIE
FDM
GT::WWW
GalaxyBot
GermCrawler
Go!Zilla
Go-Ahead-Got-It
GoZilla
Gotit
GrabNet
GrapeshotCrawler
HEADMasterSEO
HTTP::Lite
HaosouSpider
Harvest
Hloader
IDBTE4M
Iblog
Image Fetch
IndeedBot
Indy Library
Intelliseek
InterGET
Internet Ninja
InternetSeer
JamesBOT
JennyBot
Jorgee
JustView
Jyxobot
Keyword Density
LNSpiderguy
Lanshanbot
Lftp
LibWeb
Lightspeedsystems
LinkextractorPro
LinkpadBot
LinqiaMetadataDownloaderBot
Lipperhey
Lmspider
MSFrontPage
MTRobot
Mag-Net
Magnet
Mediatoolkitbot
Microsoft URL Control
Mister PiX
Mojeek
MolokaiBot
Morfeus Fucking Scanner
Mozlila
Musobot
Nameprotect
Navroad
NearSite
Nessus
NetMechanic
Netcraft
NextGenSearchBot
NimbleCrawler
Ninja
Not
Nutch
Octopus
Offline Navigator
OnCrawl
Openfind
OrangeSpider
OutfoxBot
PHPCrawl
PageScorer
Pandalytics
Panscient
Papa Foto
Pavuk
Piepmatz
Pixray
PleaseCrawl
Pockey
ProPowerBot
ProWebWalker
Pu_iN
Pump
Quick-Crawler
RSSingBot
RankFlex
RankingBot2
Rankivabot
ReGet
Reaper
Recorder
SEOkicks-Robot
ScanAlert
Scanbot
Screaming
Searchestate
SearchmetricsBot
Seekport
Semrush
SemrushBot
SeoSiteCheckup
Seomoz
Siphon
Site Sucker
SiteCheckerBotCrawler
Sitebeam
SmartDownload
Snoopy
Sosospider
Sqlworm
Sucker
SuperBot
Swiftbot
T0PHackTeam
T8Abot
TheNomad
Thumbor
Trendictionbot
True_Robot
Turnitin
TurnitinBot
TwengaBot
URLy.Warning
Upflow
VCI
Vagabondo
VeriCiteCrawler
WISENutbot
WWW::Mechanize
WallpapersHD
WeSEE
Web Auto
Web Collage
Web Enhancer
Web Sucker
WebAuto
WebBandit
WebCollage
WebCopier
WebEnhancer
WebFetch
WebFuck
WebImageCollector
WebLeacher
WebReaper
WebStripper
WebWhacker
WebsiteQuester
Webster
Whatweb
Who.is Bot
Wonderbot
Woobot
Wotbox
Xaldon WebSpider
Zauba
Zitebot
ZoomBot
ZyBorg
arquivo-web-crawler
backlink-check
coccocbot
crawl.sogou.com
crawler4j
domainsproject.org
eCatch
gopher
internetVista monitor
iubenda-radar
lwp-request
meanpathbot
muhstik-scan
oBot
page scorer
pcBrowser
ripz
s1z.ru
scan.lol
seobility
seoscanners
serpstatbot
sexsearcher
sogouspider
spyfu
sysscan
tAkeOut
trendiction.com
trendiction.de
voyagerx.com
zauba.io