nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-11-15 10:07:44 +00:00

100 lines
1.1 KiB
Text

404enemy
80legs
Alligator
Ankit
Aspiegel
Battleztar Bazinga
Bigfoot
Black Hole
BlackWidow
Bolt
BotALot
BunnySlippers
CATExplorador
CopyRightCheck
Craftbot
Crawling at Home Project
CrazyWebCrawler
Crescent
Drip
EroCrawler
FDM
FHscan
Fimap
Freeuploader
GalaxyBot
Go-Ahead-Got-It
HEADMasterSEO
HTMLparser
Iblog
IlseBot
Internet Ninja
Iskanie
JennyBot
Jetty
Leap
Lftp
Likse
LinksManager
MIDown tool
MeanPath Bot
Minefield
Mr.4x3
Name Intelligence
NetSpider
NextGenSearchBot
Nutch
Offline Explorer
OpenLinkProfiler
OutfoxBot
PxBroker
Quick-Crawler
RSSingBot
Recorder
RedesScrapy
SEOkicks
SEOlyt
SEOstats
SISTRIX
ScanAlert
Scanbot
Screaming
ScrepyBot
SemanticJuice
SemrushBot
Site Sucker
SiteCheckerBotCrawler
Snapbot
Sosospider
Sottopop
Spbot
Sqworm
Suzuran
Tracemyfile
Turnitin
Twice
URLy.Warning
UnisterBot
VeriCiteCrawler
VidibleScraper
WASALive-Bot
WWW-Collector-E
WebEnhancer
WebImageCollector
WebZIP
Website Quester
archive.org_bot
cah.io.community
clark-crawler
cohere-ai
demandbase-bot
instabid
iubenda-radar
linkdexbot
lwp-trivial
muhstik-scan
netEstate NE Crawler
openai
plumanalytics
seocompany.store
sexsearcher