nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2023-12-25 22:04:51 +00:00

100 lines
1,017 B
Text

ASPSeek
AfD-Verbotsverfahren
Alligator
Anarchie
BackWeb
Black Hole
BlackWidow
BlowFish
Brandprotect
BuzzSumo
CheeseBot
Chlooe
Cogentbot
Collector
Copyscape
Cosmos
CrunchBot
Custo
DSearch
Digincore
Disco
Domains Project
Dotbot
Evil
Exabot
Extractor
Fuzz
Gotit
Grafula
HTMLparser
HaosouSpider
HonoluluBot
Humanlinks
IDBTE4M
Image Fetch
JOC Web Spider
Jbrofuzz
LNSpiderguy
Libwhisker
LinkWalker
Linkbot
Lipperhey Spider
MarkMonitor
Mediatoolkitbot
Metauri
Mister PiX
Msrabot
Navroad
NearSite
Needle
NetAnts
NetSpider
NetZIP
Netcraft
Nutch
Pimonster
PleaseCrawl
Pockey
ProWebWalker
Proximic
Psbot
RankFlex
ScoutJet
Searchestate
Semrush
SemrushBot
SiteExplorer
SiteSucker
Siteimprove
Sitevigil
Sosospider
Spammen
Spanner
Sqlmap
Swiftbot
Telesphoreo
Telesphorep
URLy.Warning
VeriCiteCrawler
WISENutbot
WWWOFFLE
Web Collage
WebReaper
Webster
Wonderbot
Xaldon_WebSpider
Xenu
YoudaoBot
Zauba
Zitebot
arquivo-web-crawler
check1.exe
instabid
magpie-crawler
meanpathbot
page scorer
seobility
voyagerx.com
webgains-bot
x09Mozilla