nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-12-25 22:07:13 +00:00

250 lines
2.4 KiB
Text

ALittle Client
Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
AiHitBot
Aipbot
AllSubmitter
AlphaBot
Anarchie
Ankit
Apexoo
Aspiegel
Attach
AwarioRssBot
BDFetch
BackWeb
Badass
BatchFTP
Bigfoot
Bitacle
Blow
Bolt
BotALot
Brandwatch
Buck
Buddy
Bullseye
BuzzSumo
CCBot
CSHttp
Calculon
CazoodleBot
CensysInspect
CheeseBot
CherryPicker
Chlooe
Cocolyzebot
Craftbot
Crawling at Home Project
CrazyWebCrawler
CrunchBot
Custo
CyotekWebCopy
DIIbot
Digincore
DigitalPebble
DittoSpyder
DomainStatsBot
Dragonfly
EMail Siphon
EMail Wolf
Ebingbong
Evil
Exabot
Extractor
EyeNetIE
FDM
Foobot
Fuzz
G-i-g-a-b-o-t
GalaxyBot
Genieo
Go!Zilla
Gotit
Grafula
GrapeFX
HMView
HTMLparser
HTTrack
Havij
Humanlinks
Iblog
Id-search
IndeedBot
Iskanie
IstellaBot
JOC Web Spider
Jbrofuzz
JikeSpider
Joomla
Kenjin Spider
Keyword Density
LWP::Simple
Larbin
Leap
LeechGet
Lftp
LibWeb
LieBaoFast
Lightspeedsystems
LinkpadBot
LinqiaScrapeBot
Lipperhey
Lipperhey Spider
Litemage_walker
Ltx71
MIIxpc
MSFrontPage
MSIECrawler
Mag-Net
Mail.RU_Bot
Majestic12
Mass Downloader
Masscan
Mata Hari
Mb2345Browser
MegaIndex.ru
Mojeek
Mr.4x3
Musobot
Nameprotect
Navroad
Nessus
NetLyzer
NetSpider
NetZIP
Netcraft
NextGenSearchBot
Nibbler
Niki-bot
Ninja
Nmap
Not
Nuclei
Nutch
OnCrawl
OpenVAS
Openvas
OrangeSpider
OutfoxBot
PHPCrawl
PageScorer
Pavuk
PeoplePal
Petalbot
Pi-Monster
Picscout
Pixray
Pockey
ProWebWalker
Psbot
Pump
PyCurl
QueryN Metasearch
RSSingBot
RankActiveLinkBot
RankFlex
Rankivabot
Re-re
Recorder
RepoMonkey
RocketCrawler
SEOprofiler
SMTBot
Semrush
SeoSiteCheckup
SiteExplorer
SiteSucker
Sitebeam
Siteimprove
SlySearch
Snapbot
Snoopy
Sociscraper
Sosospider
Sottopop
Spammen
SputnikBot
Steeler
Stripper
SuperBot
SuperHTTP
Swiftbot
Telesphorep
The Intraformant
Thumbor
Titan
Toata
Tracemyfile
Trendiction
Turnitin
TurnitinBot
TwengaBot
Twice
URLy Warning
URLy.Warning
UnisterBot
VB Project
VeriCiteCrawler
Virusdie
Voil
Voltron
WASALive-Bot
WEBDAV
WPScan
WWWOFFLE
Wallpapers
WallpapersHD
Web Fetch
Web Fuck
Web Pix
Web Sauger
Web Sucker
WebAuto
WebCopier
WebFetch
WebLeacher
WebSauger
WebStripper
WebWhacker
WebZIP
Webalta
WebsiteQuester
Whatweb
Who.is Bot
Woobot
Wotbox
Xaldon_WebSpider
Zade
Zermelo
Zeus
Zitebot
ZoomBot
ZumBot
ZyBorg
arquivo-web-crawler
arquivo.pt
backlink-check
clark-crawler
eCatch
gopher
heritrix
isitwp.com
meanpathbot
mediawords
page scorer
pcBrowser
ripz
s1z.ru
satoristudio.net
scalaj-http
scan.lol
seoscanners
seostar
sexsearcher
siteripz
spyfu
trendiction.de
webmeup-crawler
zauba.io