nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-10-23 10:07:37 +00:00

100 lines
1.1 KiB
Text

Abonti
AiHitBot
Backlink-Ceck
Bandit
BatchFTP
Boardreader
Buddy
Bullseye
BunnySlippers
CheTeam
Cliqzbot
Cloud mapping
Cocolyzebot
Copyscape
CrunchBot
Custo
DigitalPebble
Dirbuster
DnBCrawler-Analytics
DomainSigmaCrawler
Dragonfly
EyeNetIE
Ezooms
FacebookBot
Genieo
Gigabot
HTTrack
Haansoft
Harvest
Image Fetch
IndeedBot
InternetSeer
Jyxobot
LWP::Simple
LinqiaScrapeBot
Litemage_walker
Lmspider
MSIECrawler
MTRobot
Majestic12
Mass Downloader
Masscan
MauiBot
Mediatoolkitbot
Musobot
Navroad
Nmap
Nuclei
OpenLinkProfiler
PageThing.com
Picscout
QueryN Metasearch
Re-re
RocketCrawler
Rogerbot
SEOlyticsCrawler
SalesIntelligent
ScoutJet
Semrush
SemrushBot
SiteLockSpider
SiteSucker
SmartDownload
Spammen
Spider_Bot/3.0
Telesphorep
Titan
TwengaBot
VelenPublicWebCrawler
WWWOFFLE
WallpapersHD
Web Collage
WebBandit
WebEnhancer
WebGo IS
WebImageCollector
WebSauger
Webshag
Xaldon_WebSpider
YaK
Zade
ZmEu
archive.org_bot
arquivo-web-crawler
arquivo.pt
awario.com
demandbase-bot
imagesift.com
internetVista monitor
lwp-trivial
mediawords
page scorer
probe-image-size
s1z.ru
siteripz
spyfu
webgains-bot
webpros.com
x22Mozilla
zgrab