nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-08-15 22:06:58 +00:00

250 lines
2.6 KiB
Text

01h4x.com
360Spider
404checker
ASPSeek
Abonti
Aboundex
Acunetix
AiHitBot
Alexibot
Alligator
Anarchie
Anarchy
Ankit
Anthill
Aspiegel
Asterias
Attach
AwarioRssBot
BDFetch
BackWeb
Barkrowler
BetaBot
Black Hole
BlowFish
BotALot
Brandprotect
Bullseye
Calculon
Chlooe
Cliqzbot
Cloud mapping
Cogentbot
Collector
Copier
Copyscape
Craftbot
Crescent
CyotekWebCopy
DIIbot
DatabaseDriverMysqli
Deusu
Dirbuster
Disco
Dispatch
DnyzBot
DomCopBot
DomainAppender
DomainStatsBot
Dotbot
Drip
EMail Siphon
EasyDL
EirGrabber
EroCrawler
Evil
Extractor
EyeNetIE
FDM
Flunky
Fuzz
FyberSpider
GT::WWW
GalaxyBot
GetRight
Getintent
Gigabot
Go!Zilla
Go-Ahead-Got-It
Gotit
GrabNet
GrapeFX
GridBot
HTMLparser
Haansoft
Harvest
Havij
Humanlinks
IRLbot
IndeedBot
Indy Library
InfoNaviRobot
Internet Ninja
InternetSeer
Iria
Jbrofuzz
JetCar
Jyxobot
LNSpiderguy
LWP::Simple
Leap
LeechFTP
LexiBot
Lftp
LibWeb
Lightspeedsystems
LinkWalker
Linkbot
LinkextractorPro
LinkpadBot
LinksManager
LinqiaScrapeBot
MIDown tool
MIIxpc
MJ12bot
MQQBrowser
MSIECrawler
Magnet
Majestic SEO
Majestic-SEO
Masscan
Mata Hari
Mediatoolkitbot
Microsoft URL Control
Moblie Safari
Mozlila
Musobot
Name Intelligence
NetSpider
NetZIP
NextGenSearchBot
Nikto
NimbleCrawler
Nimbostratus
Ninja
Nmap
Nuclei
Offline Explorer
Offline Navigator
Openfind
OrangeBot
OutclicksBot
OutfoxBot
POE-Component-Client-HTTP
PageAnalyzer
PageGrabber
PageScorer
Pandalytics
Panscient
Pavuk
Petalbot
Pimonster
Pixray
Pockey
Psbot
PyCurl
RSSingBot
RankActiveLinkBot
RedesScrapy
Ripper
Rogerbot
SISTRIX
ScanAlert
Scanbot
ScoutJet
Scrapy
SearchmetricsBot
SeoSiteCheckup
SeobilityBot
Siphon
SiteCheckerBotCrawler
SiteSucker
Sitebeam
Siteimprove
Sitevigil
SmartDownload
Snapbot
SocialRankIOBot
Sociscraper
Sogou web spider
Sottopop
SpankBot
Spbot
SputnikBot
Sqlworm
Steeler
Sucuri
SuperHTTP
SurveyBot
Swiftbot
Szukacz
T0PHackTeam
TeleportPro
Telesoft
Telesphorep
The Intraformant
TheNomad
Thumbor
Toweyabot
Trendiction
True_Robot
Turingos
TwengaBot
URLy Warning
UnisterBot
Upflow
V-BOT
VB Project
Vacuum
VelenPublicWebCrawler
Voltron
WBSearchBot
WEBDAV
WISENutbot
WPScan
WWWOFFLE
Wallpapers/3.0
WeSEE
Web Auto
Web Enhancer
Web Pix
Web Sucker
WebAuto
WebEnhancer
WebImageCollector
WebLeacher
WebmasterWorldForumBot
WebsiteQuester
Webster
Who.is Bot
WinHTTrack
Wonderbot
Xaldon_WebSpider
Xenu
YoudaoBot
Zeus
ZumBot
archive.org_bot
clark-crawler
com.plumanalytics
crawler4j
eCatch
facebookscraper
gopher
heritrix
instabid
internetVista monitor
isitwp.com
lwp-trivial
meanpathbot
mediawords
netEstate NE Crawler
plumanalytics
polaris version
scan.lol
sexsearcher
siteripz
webpros.com
x22Mozilla
zauba.io
zgrab