nginx-ultimate-bad-bot-blocker/dev-tools/test_units/random-bots-for-whitelist-test.list
2021-08-30 22:07:13 +00:00

250 lines
2.6 KiB
Text

01h4x.com
404checker
404enemy
80legs
AIBOT
Abonti
AhrefsBot
AiHitBot
Aipbot
Alexibot
AllSubmitter
AlphaBot
Anarchy
Apexoo
AwarioRssBot
BLEXBot
BackStreet
Backlink-Ceck
Badass
Barkrowler
BatchFTP
Bigfoot
Blackboard
BlowFish
Brandprotect
Bullseye
BunnySlippers
CODE87
CSHttp
Calculon
Cegbfeieh
CensysInspect
CherryPicker
Chlooe
Cloud mapping
Cogentbot
Copyscape
Craftbot
Crescent
CrunchBot
DTS Agent
DataCha0s
Demon
Deusu
Dirbuster
Discobot
Dispatch
DittoSpyder
DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Drip
ECCP/1.0
EMail Siphon
EMail Wolf
Ebingbong
Ecxi
EroCrawler
ExtractorPro
EyeNetIE
Foobot
GT::WWW
GetRight
GetWeb
Gigabot
Go-Ahead-Got-It
GoZilla
Gotit
HEADMasterSEO
HTMLparser
HTTP::Lite
HTTrack
HaosouSpider
Havij
Hloader
IDBTE4M
Id-search
Image Fetch
Indy Library
InfoNaviRobot
Internet Ninja
Iria
Iskanie
IstellaBot
JOC Web Spider
JamesBOT
JennyBot
Jetty
Kenjin Spider
Kinza
LNSpiderguy
LexiBot
LieBaoFast
Lightspeedsystems
Linkdexbot
LinkextractorPro
LinqiaScrapeBot
Lipperhey
Litemage_walker
Ltx71
MFC_Tear_Sample
MJ12bot
MQQBrowser
MSFrontPage
MSIECrawler
MTRobot
Majestic-SEO
MarkMonitor
Mass Downloader
Mata Hari
MauiBot
Metauri
MicroMessenger
Mister PiX
Moblie Safari
Mozlila
Mr.4x3
Msrabot
NICErsPRO
Name Intelligence
Needle
NetSpider
NetZIP
Netcraft
Netvibes
Nibbler
Nmap
Nutch
Offline Explorer
OnCrawl
OpenLinkProfiler
OpenVAS
Openfind
OrangeSpider
OutclicksBot
PageThing.com
Panscient
Papa Foto
Petalbot
Picscout
PictureFinder
Pixray
PleaseCrawl
Pu_iN
PxBroker
PyCurl
QueryN Metasearch
RSSingBot
RankActiveLinkBot
RankingBot
RankurBot
Re-re
RealDownload
Reaper
Ripper
SEOkicks
SEOkicks-Robot
SEOprofiler
Screaming
Searchestate
SemrushBot
Shodan
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
SmartDownload
Snapbot
Sogou web spider
Sosospider
Sottopop
Spanner
Spbot
Spinn3r
Sqlmap
Sqlworm
T0PHackTeam
Teleport
TeleportPro
Telesphoreo
Telesphorep
The Intraformant
TightTwatBot
Toweyabot
Tracemyfile
Trendictionbot
Turingos
Turnitin
Twice
URLy Warning
Upflow
V-BOT
VidibleScraper
Voil
WASALive-Bot
Wallpapers
Wallpapers/3.0
WallpapersHD
Web Auto
Web Enhancer
Web Fuck
Web Pix
Web Sauger
Web Sucker
WebBandit
WebFuck
WebImageCollector
WebSucker
WebZIP
WebsiteQuester
Webster
Whatweb
Who.is Bot
Wonderbot
Wotbox
Wprecon
Xaldon_WebSpider
Zade
Zermelo
Zeus
archive.org_bot
arquivo.pt
clark-crawler
cognitiveseo
crawl.sogou.com
crawler.feedback
dataforseo.com
demandbase-bot
domainsproject.org
heritrix
internetVista monitor
isitwp.com
lwp-trivial
magpie-crawler
mediawords
page scorer
pcBrowser
ripz
satoristudio.net
scalaj-http
seobility
serpstatbot
sexsearcher
sysscan
tAkeOut
trendiction.com
webpros.com
webprosbot
x09Mozilla
x22Mozilla
zauba.io