nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2022-03-30 10:19:44 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404enemy
ADmantX
AIBOT
Aboundex
Aboundexbot
AiHitBot
Aipbot
Alexibot
AllSubmitter
Anarchy
Ankit
Aspiegel
Asterias
BDCbot
BackStreet
BacklinkCrawler
Badass
BatchFTP
Bigfoot
Bitacle
Black Hole
Blackboard
BlowFish
BuiltBotTough
Bullseye
BunnySlippers
BuzzSumo
CCBot
CazoodleBot
Cegbfeieh
CheTeam
CheeseBot
Chlooe
Cloud mapping
Cocolyzebot
Collector
Copier
CopyRightCheck
Crawling at Home Project
Crescent
DataCha0s
Discobot
DnyzBot
DomCopBot
DomainCrawler
Dotbot
Download Wonder
Drip
EMail Siphon
EasyDL
Ebingbong
EirGrabber
EroCrawler
Exabot
ExtLinksBot
Extractor
ExtractorPro
EyeNetIE
FDM
Flunky
Foobot
FrontPage
FyberSpider
GT::WWW
Getintent
Gigabot
Go!Zilla
GoZilla
GrabNet
Grabber
GrapeFX
GrapeshotCrawler
HMView
Haansoft
HaosouSpider
Harvest
Heritrix
Hloader
HonoluluBot
Humanlinks
IDBTE4M
IRLbot
Iblog
IlseBot
Image Fetch
Image Sucker
InternetSeer
Iskanie
Jetty
JustView
Jyxobot
Keyword Density
LNSpiderguy
LWP::Simple
Lanshanbot
LeechFTP
LibWeb
Libwhisker
LieBaoFast
Lightspeedsystems
Linkbot
Linkdexbot
LinkextractorPro
LinqiaRSSBot
Litemage_walker
MIDown tool
MTRobot
Magnet
Majestic-SEO
Majestic12
MarkMonitor
Mass Downloader
Masscan
Mata Hari
Meanpathbot
MegaIndex.ru
Mister PiX
Mojolicious
Msrabot
NICErsPRO
NPbot
Navroad
NearSite
Net Vampire
NetAnts
NetLyzer
NetMechanic
NetZIP
Netcraft
Nuclei
Offline Navigator
OnCrawl
OrangeBot
OrangeSpider
PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageThing.com
Pandalytics
Papa Foto
Pavuk
Petalbot
Pixray
PxBroker
QueryN Metasearch
RankActive
RankFlex
ReGet
Reaper
RebelMouse
RedesScrapy
Ripper
SBIder
SEOkicks
SEOkicks-Robot
Scanbot
ScoutJet
Scrapy
Searchestate
Seekport
SentiBot
Seomoz
Siteimprove
SlySearch
Sociscraper
Sogou web spider
Spbot
Sqworm
Sucker
Sucuri
SuperBot
SuperHTTP
Surfbot
SurveyBot
Suzuran
Swiftbot
Szukacz
Telesphorep
TightTwatBot
Toata
Toweyabot
Tracemyfile
Trendictionbot
Turingos
TurnitinBot
TwengaBot
VCI
Vacuum
VelenPublicWebCrawler
Voltron
WPScan
WWW-Collector-E
WWW::Mechanize
Wallpapers/3.0
WeSEE
Web Auto
Web Collage
Web Enhancer
Web Sauger
WebAuto
WebCollage
WebFetch
WebFuck
WebGo IS
WebImageCollector
WebLeacher
WebSauger
WebStripper
WebSucker
WebmasterWorldForumBot
Website Quester
Whack
Whacker
Whatweb
WiseGuys Robot
Xaldon WebSpider
Xenu
YoudaoBot
Zade
Zitebot
ZoomBot
adscanner
archive.org_bot
autoemailspider
coccocbot
cognitiveseo
com.plumanalytics
domainsproject.org
isitwp.com
iubenda-radar
lwp-request
magpie-crawler
netEstate NE Crawler
oBot
page scorer
plumanalytics
polaris version
satoristudio.net
scan.lol
seostar
serpstatbot
sitechecker.pro
trendiction.com
trendiction.de
webgains-bot
x09Mozilla
zauba.io
zgrab