nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2023-10-06 22:04:56 +00:00

100 lines
1.1 KiB
Text

404checker
Alexibot
Anarchie
Anthill
Atomseobot
BBBike
Backlink-Ceck
Barkrowler
Bitacle
Black Hole
Blow
BuiltWith
BunnySlippers
CCBot
CopyRightCheck
Crawling at Home Project
Crescent
Custo
DittoSpyder
DomCopBot
DomainCrawler
Extractor
Flunky
Foobot
GT::WWW
Go!Zilla
Gotit
GrapeFX
GrapeshotCrawler
HTMLparser
HaosouSpider
Internet Ninja
JOC Web Spider
JennyBot
Keyword Density
LWP::Simple
LexiBot
Lftp
Lightspeedsystems
LinkWalker
LinqiaScrapeBot
Lipperhey Spider
Ltx71
MQQBrowser
Magnet
Majestic12
MarkMonitor
Mass Downloader
Mata Hari
Minefield
Morfeus Fucking Scanner
Msrabot
NPbot
NearSite
NetMechanic
Offline Explorer
OpenLinkProfiler
PHPCrawl
PageThing.com
Papa Foto
Pump
RankurBot
SEOkicks-Robot
ScanAlert
Scrapy
SeekportBot
Site Sucker
SiteSnagger
Sitebeam
Sitevigil
SocialRankIOBot
Sosospider
Sucker
Sucuri
T8Abot
The Intraformant
Turingos
TwengaBot
WWW::Mechanize
Web Pix
Web Sucker
WebFetch
WebImageCollector
WebPix
WebZIP
WebsiteExtractor
WiseGuys Robot
Wprecon
ZyBorg
archive.org_bot
check1.exe
crawler.feedback
facebookscraper
magpie-crawler
oBot
plumanalytics
scalaj-http
sitechecker.pro
x22Mozilla
xpymep1.exe