nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2023-07-15 10:07:28 +00:00

100 lines
1.1 KiB
Text

360Spider
404enemy
AdsTxtCrawlerTP
AllSubmitter
Ankit
Attach
BDFetch
BLEXBot
Bigfoot
Bitacle
CazoodleBot
Chlooe
Cliqzbot
Cloud mapping
Cocolyzebot
Copier
Cosmos
CrunchBot
DTS Agent
Disco
Dragonfly
Drip
EirGrabber
ExtLinksBot
FlashGet
FyberSpider
G-i-g-a-b-o-t
GalaxyBot
GrabNet
Grabber
HTTrack
Harvest
Heritrix
IlseBot
Image Sucker
InterGET
JamesBOT
Jetty
Jorgee
Kenjin Spider
Leap
Lightspeedsystems
LinkScan
Linkbot
MQQBrowser
MSIECrawler
Mb2345Browser
Msrabot
Nameprotect
Netcraft
Netvibes
Nutch
OnCrawl
OpenVAS
Panscient
PeoplePal
PxBroker
Quick-Crawler
RSSingBot
RankActive
RankFlex
Recorder
RedesScrapy
Rogerbot
SEOkicks-Robot
SEOlyticsCrawler
Scrapy
ScreenerBot
Searchestate
SeekportBot
SiteCheckerBotCrawler
SiteExplorer
SiteSnagger
Sottopop
TeleportPro
Telesphoreo
Trendiction
Turingos
Turnitin
VB Project
VelenPublicWebCrawler
WeSEE
WebImageCollector
WebReaper
WebSucker
WebmasterWorldForumBot
WebsiteExtractor
Whatweb
Woobot
arquivo.pt
autoemailspider
cah.io.community
com.plumanalytics
evc-batch
facebookscraper
instabid
ips-agent
iubenda-radar
meanpathbot
x22Mozilla