nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2024-01-13 10:07:12 +00:00

100 lines
1.1 KiB
Text

Aipbot
Alligator
Atomseobot
BackWeb
Blackboard
BlowFish
BotALot
Bullseye
CCBot
Citoid
Cloud mapping
Collector
DSearch
DatabaseDriverMysqli
Dispatch
DittoSpyder
Ebingbong
Extractor
Fimap
Fyrebot
Genieo
GetRight
GetWeb
Getintent
Gigabot
Go!Zilla
GridBot
HTTrack
Haansoft
InterGET
JamesBOT
Keyword Density
LWP::Simple
Larbin
LeechGet
LinkScan
LinqiaScrapeBot
MFC_Tear_Sample
Magnet
Mass Downloader
Microsoft Data Access
Mojeek
MolokaiBot
NetSpider
Nutch
Octopus
OpenVAS
OrangeBot
PHPCrawl
PageScorer
Rankivabot
Re-re
SEOkicks
ScreenerBot
Searchestate
Site Sucker
SiteSnagger
SocialRankIOBot
Sociscraper
Sogou web spider
SpaceBison
Spbot
Spinn3r
Stripper
Sucuri
SurveyBot
Suzuran
Telesphorep
Thumbor
TightTwatBot
Titan
Tracemyfile
Trendictionbot
TurnitinBot
Typhoeus
VCI
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
WWW::Mechanize
WallpapersHD
Web Auto
Web Fetch
Web Pix
WebEnhancer
WebImageCollector
WebLeacher
WebSauger
Website Quester
Xenu
crawl.sogou.com
evc-batch
heritrix
internetVista monitor
linkdexbot
mediawords
netEstate NE Crawler
scalaj-http
trendiction.com
x09Mozilla