nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-07-20 22:04:47 +00:00

100 lines
1,020 B
Text

360Spider
AdsTxtCrawlerTP
AfD-Verbotsverfahren
AhrefsBot
Apexoo
Attach
Backlink-Ceck
Bandit
Calculon
CheTeam
CheeseBot
Citoid
Claritybot
Copyscape
CrunchBot
DTS Agent
Dispatch
DnyzBot
EasyDL
EirGrabber
Exabot
FacebookBot
FlashGet
Fyrebot
GPTBot
Google-Extended
Grafula
GrapeFX
Haansoft
Havij
Heritrix
IDBTE4M
InternetSeer
IstellaBot
JOC Web Spider
JamesBOT
JikeSpider
Keybot Translation-Search-Machine
Kinza
LexiBot
Likse
LinkWalker
Linkbot
LinqiaRSSBot
MJ12bot
MSIECrawler
MTRobot
Mass Downloader
Mb2345Browser
Mojeek
Msrabot
Musobot
NextGenSearchBot
Nikto
Nmap
OnCrawl
PageScorer
Pavuk
Picscout
ProPowerBot
Pu_iN
PyCurl
Reaper
Recorder
RedesScrapy
SBIder
SiteSnagger
SiteSucker
Sosospider
Sqworm
Teleport
Toata
TwengaBot
URLy.Warning
Voltron
Web Fetch
Web Fuck
WebFetch
WebsiteExtractor
Whatweb
Xaldon WebSpider
YaK
Zermelo
Zitebot
cah.io.community
check1.exe
coccocbot
cognitiveseo
dataforseobot
heritrix
instabid
ips-agent
iubenda-radar
linkdexbot
linkfluence
magpie-crawler
scan.lol
sogouspider
webprosbot
zauba.io