vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2912 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-11 22:07:03 +00:00
parent 7e094dccdd
commit 910c0669d5
47 changed files with 82569 additions and 82579 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2911 #### Version: V4.2022.02.2912
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2911 #### Version: V4.2022.02.2912
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2911 #### Version: V4.2022.02.2912
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10186
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2911 #### Version: V4.2022.02.2912
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10190
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10190
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10190
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10190
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 1; 10.1.1.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 0; 10.1.1.87 0;
127.0.0.1 0; 127.0.0.1 0;

10190
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 1; 10.1.1.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 0; 10.1.1.87 0;
127.0.0.1 0; 127.0.0.1 0;

10190
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 1; 10.1.1.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 0; 10.1.1.87 0;
127.0.0.1 0; 127.0.0.1 0;

10190
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2911 2912

10190
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10190
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.121.18.76 0; 52.226.113.98 0;

10190
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 1; 10.1.1.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 0; 10.1.1.87 0;
127.0.0.1 0; 127.0.0.1 0;

10190
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 1; 10.1.1.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 0; 10.1.1.87 0;
127.0.0.1 0; 127.0.0.1 0;

10190
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 1; 10.1.1.87 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 0; 10.1.1.87 0;
127.0.0.1 0; 127.0.0.1 0;

10190
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.121.18.76 0; 52.226.113.98 0;

10190
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.121.18.76 0; 52.226.113.98 0;

10190
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 1; 10.1.1.87 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BotALot CCBot
CazoodleBot ExtLinksBot
ECCP/1.0 FemtosearchBot
GetRight Jbrofuzz
LeechFTP Nmap
Not Rogerbot
RankFlex Shodan
Searchestate TightTwatBot
Voltron WWW-Collector-E
dataforseo.com WebImageCollector

302
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 80legs
404enemy
ALittle Client ALittle Client
Aboundex Abonti
AhrefsBot
AiHitBot AiHitBot
Alexibot
AllSubmitter AllSubmitter
Ankit Alligator
Anthill AlphaBot
Apexoo Anarchy99
Atomseobot AwarioRssBot
BLEXBot BBBike
BackWeb BackWeb
Badass Backlink-Ceck
Bandit BacklinkCrawler
BetaBot BetaBot
Bitacle Bigfoot
BlackWidow Blackboard
Blow BlowFish
BotALot BotALot
Brandprotect BunnySlippers
Brandwatch BuzzSumo
BuiltBotTough
BuiltWith
CATExplorador CATExplorador
CODE87 CODE87
CazoodleBot CSHttp
Cegbfeieh CensysInspect
CheTeam
CheeseBot CheeseBot
Chlooe Citoid
Claritybot
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
CopyRightCheck Cocolyzebot
Cosmos Cogentbot
Craftbot Collector
Crescent Copier
CyotekWebCopy Copyscape
DBLBot Crawling at Home Project
DSearch DataCha0s
Demon Deusu
Digincore Devil
Disco DigitalPebble
Discoverybot Dirbuster
DittoSpyder DittoSpyder
DnBCrawler-Analytics
DomainAppender DomainAppender
DomainCrawler DomainCrawler
Dotbot DomainStatsBot
Download Wonder
Dragonfly Dragonfly
EMail Siphon ECCP/1.0
EMail Wolf
EasyDL EasyDL
Ecxi Ebingbong
Evil Evil
Exabot Exabot
ExtLinksBot ExtLinksBot
Extractor
Extreme Picture Finder Extreme Picture Finder
EyeNetIE FDM
Foobot FemtosearchBot
GT::WWW Fimap
GermCrawler Freeuploader
FyberSpider
GalaxyBot
GetRight GetRight
Getintent
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet GrabNet
HEADMasterSEO GridBot
HTTP::Lite HTTP::Lite
HTTrack
Haansoft
Harvest Harvest
Havij Havij
HybridBot HonoluluBot
IDBot
IRLbot IRLbot
Iblog Iblog
Id-search Id-search
IlseBot IlseBot
Image Fetch
Image Sucker Image Sucker
Indy Library IndeedBot
InfoTekies InterGET
InternetSeer Iria
JamesBOT JOC Web Spider
Jbrofuzz
JennyBot
JetCar
Jetty Jetty
JikeSpider JikeSpider
JustView Joomla
Lanshanbot Keyword Density
LWP::Simple
Larbin
Leap
LeechFTP
LeechGet
LibWeb LibWeb
Libwhisker LieBaoFast
LinkScan Lightspeedsystems
LinkextractorPro Likse
LinkpadBot LinkWalker
Linkbot
Linkdexbot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker MFC_Tear_Sample
MIIxpc MIIxpc
MQQBrowser MSFrontPage
MSIECrawler Magnet
MTRobot Mass Downloader
Majestic-SEO
Majestic12
MarkMonitor
MarkWatch
Masscan Masscan
MauiBot Mata Hari
Meanpathbot MeanPath Bot
MegaIndex.ru Mediatoolkitbot
Metauri
Microsoft Data Access
Microsoft URL Control
Mister PiX Mister PiX
Moblie Safari Moblie Safari
MolokaiBot Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3 Mr.4x3
Msrabot Msrabot
NPbot Nameprotect
Name Intelligence
Needle Needle
Nessus Nessus
NetAnts Net Vampire
NetSpider NetMechanic
NetZIP
Netcraft Netcraft
Nettrack Nettrack
Netvibes
Nibbler
Niki-bot Niki-bot
NimbleCrawler Offline Navigator
Ninja
Nmap
Nutch
Offline Explorer
OnCrawl OnCrawl
Openfind
Openvas Openvas
OrangeBot
OutfoxBot
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageAnalyzer PageAnalyzer
PageThing.com PageThing.com
Pandalytics
Papa Foto Papa Foto
ProPowerBot Pavuk
PeoplePal
Petalbot
Pimonster
ProWebWalker
Probethenet
Psbot Psbot
PxBroker
PyCurl
RankActive
RankFlex RankFlex
Rankivabot
RankurBot
Re-re Re-re
Reaper RedesScrapy
RebelMouse Ripper
RepoMonkey
Rogerbot Rogerbot
ScanAlert SEOkicks-Robot
ScoutJet SEOlyticsCrawler
Searchestate SEOprofiler
SEOstats
SalesIntelligent
SemanticJuice SemanticJuice
Semrush
SentiBot
SeobilityBot SeobilityBot
Site Sucker Shodan
SiteExplorer
SiteLockSpider
SiteSucker SiteSucker
Sitebeam SlySearch
Siteimprove
Sitevigil
Snake Snake
Snapbot Sociscraper
Snoopy Sottopop
SocialRankIOBot
SpaceBison
Spbot
Spinn3r
SputnikBot
Sqlmap Sqlmap
Surfbot Sqlworm
Sucuri
SuperBot
SuperHTTP
SurveyBot
T0PHackTeam T0PHackTeam
T8Abot
Teleport Teleport
Telesoft
Telesphoreo Telesphoreo
TheNomad
Thumbor Thumbor
TightTwatBot
Titan
Toweyabot Toweyabot
Trendictionbot
Turingos Turingos
Turnitin Turnitin
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Twice
Typhoeus Typhoeus
URLy Warning
URLy.Warning URLy.Warning
V-BOT UnisterBot
Vacuum Upflow
VB Project
VCI
Vagabondo Vagabondo
VidibleScraper
Virusdie
VoidEYE VoidEYE
WEBDAV Voil
WWW-Collector-E Voltron
WISENutbot
WPScan
WWWOFFLE WWWOFFLE
WallpapersHD Wallpapers/3.0
Web Auto
Web Fetch Web Fetch
Web Sauger Web Sauger
WebAuto WebBandit
WebCollage WebCopier
WebEnhancer WebFetch
WebFuck WebGo IS
WebImageCollector WebLeacher
WebPix WebPix
WebStripper WebSauger
WebWhacker WebSucker
Webalta WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Website Quester
WebsiteExtractor
Webster
Whacker Whacker
Whatweb Whatweb
WiseGuys Robot
Woobot
Xaldon WebSpider Xaldon WebSpider
Xenu YoudaoBot
Zauba Zermelo
Zeus
ZoominfoBot ZoominfoBot
cah.io.community archive.org_bot
autoemailspider
backlink-check
clark-crawler
coccocbot-web coccocbot-web
cognitiveseo cognitiveseo
crawler.feedback crawler4j
dataforseo.com dataforseo.com
demandbase-bot heritrix
domainsproject.org instabid
evc-batch
internetVista monitor
ips-agent ips-agent
lwp-trivial isitwp.com
oBot lwp-request
netEstate NE Crawler
pcBrowser
probe-image-size probe-image-size
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scalaj-http scan.lol
seobility seostar
serpstatbot serpstatbot
sogouspider sitechecker.pro
siteripz
sp_auditbot
sysscan
tAkeOut tAkeOut
ubermetrics-technologies.com
webgains-bot webgains-bot
x09Mozilla webmeup-crawler
zauba.io
zgrab zgrab

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404checker
404enemy 404enemy
ALittle Client ADmantX
AIBOT
ASPSeek ASPSeek
Abonti
Aboundex
Acunetix Acunetix
AhrefsBot AhrefsBot
Alexibot Aipbot
AlphaBot AllSubmitter
Anarchy
Anarchy99
Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BackDoorBot BBBike
BackStreet Bandit
BackWeb
BacklinkCrawler
BetaBot BetaBot
Bigfoot
Black Hole Black Hole
Blackboard BlackWidow
Blow Bolt
BotALot BotALot
Brandprotect Brandprotect
CODE87 BuzzSumo
CheeseBot
Chlooe Chlooe
Citoid Cocolyzebot
Cliqzbot Copier
Cloud mapping
Cogentbot
Copyscape Copyscape
Cosmos
Craftbot Craftbot
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
CrunchBot Curious
DIIbot Custo
CyotekWebCopy
DBLBot
DSearch DSearch
DatabaseDriverMysqli DataCha0s
Deusu Digincore
Devil
DigitalPebble DigitalPebble
Disco
Dispatch
DittoSpyder
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Download Wonder DomainStatsBot
Dotbot
ECCP/1.0 ECCP/1.0
EasyDL
Ebingbong
Ecxi
Evil Evil
ExtLinksBot
ExtractorPro ExtractorPro
Extreme Picture Finder EyeNetIE
Ezooms Foobot
FemtosearchBot
Firefox/7.0
FlashGet
Flunky
Freeuploader
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t GalaxyBot
GT::WWW GetWeb
Gigabot
Go!Zilla
GoZilla
Gotit Gotit
GrabNet Grabber
GrapeFX GrapeFX
Haansoft GrapeshotCrawler
HaosouSpider GridBot
Harvest HTMLparser
IDBTE4M HonoluluBot
IDBot
IRLbot IRLbot
Iblog Iblog
Id-search
IlseBot IlseBot
IndeedBot Image Fetch
InfoNaviRobot
Intelliseek Intelliseek
InterGET Internet Ninja
InternetSeer InternetSeer
Iria Iskanie
IstellaBot IstellaBot
JOC Web Spider Jbrofuzz
JamesBOT Jetty
JikeSpider JikeSpider
Jyxobot JustView
Keyword Density Keyword Density
Kinza Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Larbin Larbin
Leap Lightspeedsystems
LeechGet Likse
LibWeb
Libwhisker
LieBaoFast
LinkScan
Linkdexbot
LinkextractorPro LinkextractorPro
LinkpadBot LinksManager
LinqiaMetadataDownloaderBot Ltx71
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Spider
Lmspider
MFC_Tear_Sample
MIIxpc
MJ12bot MJ12bot
MQQBrowser MSIECrawler
MSFrontPage MTRobot
Majestic-SEO Mag-Net
Magnet
Mail.RU_Bot
Majestic SEO
MarkMonitor MarkMonitor
MarkWatch MarkWatch
Masscan
Mata Hari
MauiBot
MeanPath Bot MeanPath Bot
Meanpathbot Mediatoolkitbot
Mister PiX Metauri
MicroMessenger
Microsoft URL Control
Mojeek
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3 Mr.4x3
Msrabot Msrabot
NPbot NICErsPRO
Nameprotect Net Vampire
Nessus NetLyzer
NextGenSearchBot NetMechanic
NetZIP
Netcraft
Netvibes
Nibbler
NimbleCrawler NimbleCrawler
Nimbostratus
Not
Nuclei
Octopus Octopus
OnCrawl Offline Explorer
OpenLinkProfiler Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider
PECL::HTTP PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP Page Analyzer
PageScorer Papa Foto
Pavuk Picscout
Petalbot PictureFinder
Picsearch
Piepmatz
Pimonster
Pixray Pixray
ProPowerBot ProPowerBot
Probethenet ProWebWalker
Psbot Psbot
PxBroker PxBroker
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RSSingBot
RankFlex RankActiveLinkBot
RankingBot Rankivabot
RankingBot2
RankurBot RankurBot
Reaper Re-re
Recorder
RedesScrapy RedesScrapy
RepoMonkey SEOlyticsCrawler
RocketCrawler
SEOstats
SISTRIX SISTRIX
SMTBot SMTBot
Scanbot
ScoutJet ScoutJet
ScreenerBot ScreenerBot
SearchmetricsBot Searchestate
Seekport SemanticJuice
Semrush SemrushBot
SentiBot
SeobilityBot SeobilityBot
Shodan Seomoz
Siphon
Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteSnagger SiteSnagger
Siteimprove SiteSucker
SlySearch Sitebeam
Snake Snake
Sociscraper Snapbot
Snoopy
SocialRankIOBot
Sogou web spider Sogou web spider
Sottopop
Spammen
SpankBot SpankBot
Steeler Spanner
Spbot
Spinn3r
SputnikBot
Sqlmap
Stripper
Sucker Sucker
SuperHTTP Sucuri
Szukacz Szukacz
T8Abot Teleport
TeleportPro
Telesphoreo Telesphoreo
Telesphorep Toweyabot
Toata
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot
Turingos
Turnitin
TwengaBot
Twice
URLy Warning
UnisterBot UnisterBot
Upflow VCI
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler
VoidEYE VoidEYE
Voil WEBDAV
WASALive-Bot
WBSearchBot
WPScan
WWW-Mechanize
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
WeSEE WeSEE
Web Auto Web Auto
Web Collage Web Fuck
Web Sucker Web Sucker
WebBandit
WebEnhancer
WebFetch WebFetch
WebGo IS WebGo IS
WebImageCollector
WebLeacher
WebPix WebPix
WebStripper
WebSucker WebSucker
WebWhacker
WebZIP
WebmasterWorldForumBot
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster
Whacker Whacker
WiseGuys Robot Who.is Bot
Wonderbot Wonderbot
Woobot Woobot
Wprecon Xaldon WebSpider
YoudaoBot YoudaoBot
Zade adscanner
Zermelo arquivo.pt
Zeus
archive.org_bot
backlink-check backlink-check
cah.io.community
check1.exe
cognitiveseo cognitiveseo
com.plumanalytics crawler4j
crawl.sogou.com domainsproject.org
eCatch eCatch
instabid facebookscraper
heritrix
internetVista monitor internetVista monitor
ips-agent isitwp.com
lwp-request
lwp-trivial lwp-trivial
meanpathbot
mediawords mediawords
oBot
page scorer
plumanalytics plumanalytics
ripz probe-image-size
seocompany.store scalaj-http
seoscanners scan.lol
seostar sexsearcher
sitechecker.pro sitechecker.pro
sogouspider trendiction.de
sp_auditbot
tAkeOut
trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot webgains-bot
webmeup-crawler
webprosbot webprosbot
x09Mozilla
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
best-games.today bookmaker-bet.com
businescoop.men conciergegroup.org
cacheimages.com ddpills.com
chatroulette.video eu-cookie-law.info
distver.ru gorabagrata.ru
gdcentre.ru porno-home365.com
gsbs.com.ua taraz.xkaz.org
keywordblocks.com textads.men
raavidesigns.com tizanidineinfo.blogspot.com
swinginwithme.ru zigarettenonlinekaufen.tumblr.com

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 1-free-share-buttons.com
03p.info 10bet.com
0n-line.tv 1flag.co.za
108shot.com 1qingdao.com
12u.info 20pascals.nl
13tabs.com 21h2o.com
1adult.com 2itech.ru
23kw.ru 4kpics.rocks
57883.net 7xc4n.com
7search.com 8kisses.eu
a14download.com abcguru.xyz
aa08daf7e13b6345e09e92f771507fa5f4.com actionnooz.com
aaa38852e886ac4af1a3cff9b47cab6272.com actulite.com
aac52006ec82a24e08b665f4db2b5013f7.com admatic.com.tr
acunetix-referrer.com ads-cool.pro
adexprts.com advocatemsk.ru
ads.gold afora.ru
ads.tremorhub.com ahrefs.com
adservme.com
adult-shop.com.ua
affiliate-fr.com
agardomains.com
airlandsea.info
akama.com
akvamaster.dp.ua
alborzan.com alborzan.com
albumsuper.info alessandraleone.com
alert-hgd.xyz alvtank.se
alf-img.com amateurgalls.com
alienwheel.es amazon-seo-service.com
alive-ua.com amyfoxfitness.com
allornamenti.com analytics-ads.xyz
anabolics.shop android-systems.ru
angel1777.kz angigreene.com
angry-fermi-7633.arukascloud.io animalia-life.club
animal-drawings.com anniemation.com
apartamentwroclaw.eu antons-transporte.de
apiitsthirteende-a.akamaihd.net apimountainbikei-a.akamaihd.net
apilinkswiftco-a.akamaihd.net apple.com-cleaner.systems
apiportalmorecom-a.akamaihd.net arabgirls.adultgalls.com
appfastplay.com arquivo.pt
aramaicmedia.org artparquet.ru
arclk.net asseenontv.ru
arvut.org astrochicks.com
astana.xxxkaz.org atout-energie-69.com
atmovs.com australia-opening-times.com
aversis.net auto-moto-elektronika.cz
avtolombard-krasnodar.com autodo.info
avtolombard-krasnodar.ru autoloans.com
azte.ch autoplate.info
baltgem.com aviav.org
banan.tv aviav.ru.com
bard-real.com.ua avitocars.ru
bedandbreakfast.com avramstroy.ru
belsetirehafi.tk avtochehli.by
bestsexyblog.com ayakino.net
bg6s0.com azadnegar.com
biglistofwebsites.com bala.getenjoyment.net
bio-optomarket.ru banksrf.ru
biteg.xyz bannertesting.com
blog.koreadaily.com beachfix.co
bloggerads.net best-seo-solution.com
blogporn.in bestofpicture.com
blogqpot.com bestplacetobuyeriacta.jw.lt
bloke.com bestserials.com
boleznikogi.com beta.hotkeys.com
bombla.org betterhdporn.com
bongiornos.info bimmerpost.com
boostmyppc.com bird1.ru
bjgugu.net.ua
bkgr.se
blogrankers.com
bobba.dzaba.com
bobinoz.com
bonzbuddy.com
bosman.pluto.ro
braip.com.br
buron.pw
buttons-for-website.com
buyclomidonlaine.com buyclomidonlaine.com
bzero1jewelry.net buzzurl.jp
calcularpagerank.com bvlgariwallet.xyz
capture-room.com calc-for-credit.ru
carabela.com.do callawaygolfstore.online
cardsharp1.ru camdolls.com
carrauterie.be capacitacionyaprendizaje.com
cartierlovebraceletreplica.xyz cartierlovestore.com
cartierreplica.top cartierreplica.xyz
casablancamanor.co.za
casinopinup-wins.com
cb.iphantom.com cb.iphantom.com
cbb1smartlist12.click cbcseward.com
centraletermice.us cdnanalytics.xyz
cherrypointplace.ca cellfun.mobi
christianlouboutinreplica.pw chatroulette.world
ci.ua cheap-trusted-backlinks.com
classicakuhni.ru cheapestjerseys-wholesale.com
cleanersoft.com cheapjerseysfootballshop.com
cheapmoncler.pw
cheapwebsitehoster.com
check-this-out-now.online
cl.s7.exct.net
clickbanksites.info
clickintext.net
clipartnew.com clipartnew.com
cognitiveseo.com coindirect.io
compliance-elena.xyz compliance-don.top
compliance-ivan.xyz connectingsingles.com
compliance-jess.top constantaservice.net
compliance-nelson.top contextualyield.com
computernetworksonline.com coolbar.pro
conboy.us counterbot.com
connectionstrenth.com crazyboost.pro
contentsexpress.com cubook.supernew.org
cookie-law-enforcement-aa.xyz customcatchcan.com
copypaste.traderzplanet.in
corta.co
coslab.club
cpabegins.ru
cranly.net
culturevie.info
customsua.com.ua
cwetochki.ru
dafatiri.com
dairyindia.in
daneshetabiat.com
daretodonate.co daretodonate.co
data-mining.tk darodar.com
dating-time-now.com davidovic.info
daydream-studio.ru deanmoore.ie
defenderxtactical.com dear-diary.ru
deniven.1bb.ru
derevesendeco.com derevesendeco.com
digitalfaq.com detskie-zabavi.ru
distver.ru detsky-nabytek.info
doctissimo.fr dfwu1013.info
domainanalyzing.xyz discovertreasurenow.com
domination.ml domashneeruporno.com
donvito.unas.cz downloads-whatsapp.com
dorratex.tn dpihatinh.gov.vn
dptaughtme.com
drhomes.biz drhomes.biz
drpornogratisx.xxx drunkenstepfather.com
drugstoreforyou.com e2click.com
dvd-famille.com
e-poker-2005.com
easyukraine.com easyukraine.com
ecommerce-seo.com editmedios.com
ekobata.ru ekspertmed.com
electrik-avenue.com elektronischezi.livejournal.com
elektrozigarette1.dreamwidth.org elitedollars.com
elite-sex-finders.com erotik-kostenlos.net
eloconcream.blogspot.com esfchat.tk
emailaccountlogin.co estibot.com
energy-ua.com everypony.ru
engines-usa.com ewebarticle.info
eralph.tk extads.net
essaypro.com ezigarettekaufen2.blox.pl
etenininrade.ga faracontrol.ir
euroskat.ru fast-wordpress-start.com
exhibitionplus.eu fastfixing.tech
exlarseva.webblog.es fickluder69.com
expdom.com fidelityfunding.com
ezofest.sk find1friend.com
fapgon.com findanysex.com
farm26.ru fireads.men
farmprofi.net firstdrugmall.ru
fastcrawl.com flirt4free.com
feel-planet.com forensicpsychiatry.ru
ferieboligkbh.dk fortwosmartcar.pw
fingerprintjs.com fototravel.eu
firstsiteguide.com fr-bearings.ru
flashbannernow.com
food.dtu.dk
footbalive.org
forex-indextop20.ru
forexunion.net
forum69.info
forzeronly.com
foto-basa.com
franklinfire.co franklinfire.co
free-fbook-traffic.com fullgirl.ru
free-traffic.xyz fun-mobi.pl
freshmac.space funponsel.com
ftns.ru funtoonez.com
fugarif.ga galeon.com
fyl.com.ru game300.ru
gamblingnerd.com gay.adultgalls.com
gamebackyard.com gdebestkupit.ru
gateway.zscalertwo.net generousdeal-a.akamaihd.net
ge0ip.net gentamicineyedrops.blogspot.com
genericlowlatencyasiodriverhq.aircus.com gepezz.info
get-your-social-buttons.info get-free-social-traffic.com
gettpromos.com gg.zzyjxs.com
ggiaro.com girls-ufa.ru
given2.com gloverid.site
gk-atlant.info go2mike.ru
glavtral.ru gobongo.info
global-ics.co.za goldadpremium.com
golaya.pw goldpanningtools.com
goodwinmetals.co gomusix.com
gosreg.amchs.ru googlsucks.com
gsasearchengineranker.pw gpirate.com
gsasearchengineranker.top graphid.com
gsasearchenginerankerdiscount.com growmyfunds.ca
gta-club.ru gsasearchengineranker.site
gwagka.com guardlink.org
gwhwpxbw.bloger.index.hr habermetre.com
hdimagelib.com hauleddes.com
hikesearch.net hdapp1008-a.akamaihd.net
himgaws.pw helpmymacfaster.trade
hitmuzik.ru heroz.fr
hledejvshopech.cz hidefiles.org
homelygarden.com histats.com
huhn.altervista.org histock.info
ideashome.id hkladys.com
ilovevitaly.ru hop.clickbank.net
imadedinner.net hotel-mkad.ru
import-sales.com hotenergy.ru
inbabes.sexushost.com housekuba.org
incolors.club houseofrose.com
infobabki.ru houston-vikings.com
houtings.xyz
howmuchdoestizanidinecost.blogspot.com
hulfingtonpost.com
hully.altervista.org
husky-shop.cz
i-service.kz
iamsport.org
idc.com.ua
ie.57883.net
ilbe.club
imgata.com
incep.imagine-ex.co
indianmedicaltourismshop.com
infobanks.ru infobanks.ru
innodgfdriverhm.aircus.com
innoslicon.com innoslicon.com
inspiring-desperate.tk int.search.tb.ask.com
instabid.tech intellekt21.ru
int.search.mywebsearch.com investyb.com
ispac.org inzn.ru
istanbulit.com ipchicken.com
istizanidineanopiate.blogspot.com isitwp.com
izismile.com iusstf.org
j33x.com janettabridal.com
japan-bearings.ru jav-fetish.com
javatex.co.id javitas.info
javxxx18.com
jerseywholesalechinabiz.com
jobmarket.com.ua jobmarket.com.ua
journalhome.com joessmogtestonly.com
jus0wil.pp.ua kaidalibor.de
kakadu-interior.com.ua kasino-money.pw
kaz.kz katjimej.blog.fc2.com
kerei.ru kazinogames.lv
keywordblocks.com kemerovo.zrus.org
keywordteam.net kenaba.su
kidskunst.info kino-filmi.com
kinky-fetishes.com kinoplen.ru
kinobaks.com kol-energo.ru
kladrus.ru kosynka-games.ru
koleso24.com.ua labelwater.se
kometars.xyz lacapilla.info
komputers-best.ru lanasshop.ru
konica.kz letolove.ru
konturkrasoty.ru letsrepair.in
kountrylife.com lexaprogeneric.link
kredit-blog.ru lidoradio.com
kriokomora.info limads.men
kskjco.club linkredirect.biz
lankarns.com littlecity.ch
lapitec.eu liver-chintai.org
larutti.ru locustdesign.co
lexiangwan.com lol-smurfs.com
lietaer.com
lincolntheatre.com
lion.misba.us
lisque.batcave.net
lollypopgaming.com lollypopgaming.com
losangeles-ads.com lombia.com
lostfilm-online.ru louboutinreplica.xyz
lrac.info
ltvperf.com ltvperf.com
luckyshop.net.ua luckyshop.net.ua
lyrics.home-task.com luxup.ru
m.b00kmarks.com mabdoola.blogspot.com
magicplayer-s.acestream.net magento-crew.net
magnetic-bracelets.ru magicdiet.gq
manimpotence.com magnetpress.sk
manuscript.su mail.allnews24.in
marcoislandvacations.net make-money-online.com
martlinker.com makedo.ru
masthopehomes.com mamasuper.prom.ua
maximpartnerspr.com maridan.com.ua
medicine-4u.org marmotstore.online
megaindex.ru medizinreisen.de
memberty.com meduza-consult.ru
menhealed.net megaapteka.ru
meteocast.net mex-annushka.ru
mhtr.be micasainvest.com
microsearch.ru microsoftportal.net
mirobuvi.com.ua mindbox.co.za
mirtorrent.net miniads.ca
mixtapetorrent.com misandesign.se
mobicover.com.ua mixed-wrestling.ru
modabutik.ru mmofreegames.online
moivestiy.biz mnimmigrantrights.net
monetizationking.net mockupui.com
moneymaster.ru moje-recenze.cz
moneyviking-a.akamaihd.net monarhs.info
morefastermac.trade moncleroutletonline.win
morlat.altervista.org monsterdivx.com
mosdverka.ru muizre.ru
mp3downloadhq.com mundoaberrante.com
mrvideospornogratis.xxx mydirtyhobby.com
mtmtv.info myindospace.com
mxgetcode.com naperehresti.info
mydoctorok.ru narkologiya-peterburg.ru
mydownlodablefiles.com narutonaruto.ru
mylesosibirsk.ru nbsproject.ru
myliveblog.ru negociosdasha.com
myonlinepayday.co neks.info
myperiod.club new7ob.com
mywallpaper.top news-speaker.com
myxdate.info nextbackgroundcheck.gq
nakrutka.cc ningessaybe.me
nardulan.com nl.netlog.com
narosty.com noclegonline.info
nastroyke.net nofreezingmac.click
naturalshair.site nonews.co
naughtyconnect.com normalegal.ru
nelc.edu.eg norththeface.store
next-dentists.tk
nisuturnetdgu.tk
nlfjjunb5.ru
nomuos.it
nordstar.pro
noumeda.com noumeda.com
o-o-11-o-o.com nrjmobile.fr
oda.as nuit-artisanale.com
odesproperty.com oecnhs.info
offside2.5v.pl offside2.5v.pl
offtopic.biz ok-ua.info
ohmyrings.com okmedia.sk
old-rock.com oksrv.com
olgacvetmet.com oneclickfiles.com
online-x.ru online-hd.pl
onlyforemont.ru onlineporno.site
onload.pw
onlyporno.ru onlyporno.ru
onlythegames.com openfrost.com
orhonit.com osb.se11.ru
ourtherapy.ru ozas.net
outpersonals.com pacificair.com
ovirus.ru panamaforbeginners.com
paparazzistudios.com.au paphoselectricianandplumber.com
pcads.ru partner-host.men
pearlisland.ru partner-pop.men
pestomou.info partner-print.men
phormchina.com pateaswing.com
pixelrz.com pc4download.co
plastjulgranar.se pechikamini.ru
pochtovyi-index.ru perederni.net
podshipniki-ntn.ru perfectplanned.com
pokerniydom.ru picmoonco.pw
polcin.de picscout.com
poptool.net picture-group.com
popup-fdm.xyz pinkduck.ga
popup-jdh.xyz pinstake.com
porn555.com pintattoos.com
pornogratisdiario.com pizza-tycoon.com
pornosexrolik.com pl.id-forex.com
porodasobak.net platezhka.net
privatefx.all4invest.info play-mp3.com
prizefestival.mobi plugingeorgia.com
prizesbook.online popads.net
pro-tec.kz poquoson.org
porno-video-chati.ru
porno.simple-image.com.ua
pornoplen.com
pornosee.info
potoideas.us
profolan.pl profolan.pl
prom23.ru
promgirldresses.xyz promgirldresses.xyz
promoheads.com promoforum.ru
pron.pro proposal-engine.com
prosperent.com propranolol40mg.blogspot.com
proprostatit.com
prostitutki-belgoroda.org
prostitutki-rostova.org prostitutki-rostova.org
pulse33.ru prostitutki-yaroslavlya.org
purchasepillsnorx.com
qcstrtvt.bloger.index.hr qcstrtvt.bloger.index.hr
qexyfu.bugs3.com qpypcx.com
quick-seeker.com
quickbuck.com quickbuck.com
quickchange.cc rapidgator-porn.ga
rada.ru rating-bestcasino.com
raschtextil.com.ua realitykings.com
rating-casino2021.ru realizmobi.com
registry-clean-up.net realting-moscow.ru
reklama-i-rabota.ru rebelmouse.com
retreatia.com recipedays.com
rewardpoll.com recipedays.ru
rixpix.ru relayblog.com
rome2rio.com remont-ustanovka-tehniki.ru
royalads.net renecaovillasale.online
ruclicks.com rentehno.ru
ruf777.com rusenvironmental.net
running-line.ru s-forum.biz
rusoft-zone.ru sa-live.com
ruspdd.com sa-rewards.co.za
rustic-quiver.win sady-urala.ru
sakhboard.ru sakhboard.ru
sanatorrii.ru saveriopiazza.it
sanjosestartups.com sbdl.no
saratov.xrus.org sbprabooks.com
sarm.tk scanner-alex.top
scanner-fred.top scanner-donald.top
scanner-olga.top schalke04fc.info
scanner-walter.top schlampen-treffen.com
scat.porn searchinquire.com
sdelai-prosto.ru segol.tv
sdelatmebel.ru
sdi-pme.com
seadragonherbery.com
search.xtconnect.com
seasaltwithfood.com
secret.xn--oogle-wmc.com
security60-e.com
semaltmedia.com semaltmedia.com
seminarykansas904.ml sex-tracker.com
seoanalyses.com sex.hotblog.top
seoboxes.com sexiporno.net
seocheckupx.com sexkontakteao.info
seogadget.ru sexstream.pl
seorankinglinks.us sextracker.be
serdcenebolit.com sexyebonyteen.com
sethrollins.net sfd-chess.ru
sextracker.com shikiso.info
sextracker.de sladkoevideo.com
sharebutton.to slkrm.ru
sigmund-freud.co.uk smartshoppymac.com
site.ru sockshare.net
sitebeam.net softxaker.ru
sitiz.club
sivs.ru
skutecznetabletkinaporostwlosow.pl
sloopyjoes.com
smart-scripts.com
soblaznu.net
social-buttons.xyz
soft-program.com
sonata-arctica.wz.cz sonata-arctica.wz.cz
sr-rekneskap.no sovetogorod.ru
speedup-my.site
spidtest.space
sport-video-obzor.ru
sprttrack.com
staff.prairiesouth.ca
stard.shop stard.shop
stop-zavisimost.com starpages.net
streetfooduncovered.com steame.ru
strfls.com sugvant.ru
strigkaomsk.ru sukarame.net
studiokamyk.com.pl summerlinhomes411.info
suggest-keywords.com svarkagid.com
superkanpo.com
suture.co
swinger-mobil.net swinger-mobil.net
symphonyintegratedhealthcare.com szucs.ru
taaaak.com t-bygg.com
tabletkinaodchudzanie.com.pl takethatad.com
tacbibirfa.tk teguh.info
tampabaywatch.org telefonsexsofort.tk
telemetryverification.net telegramdownload10.com
terrafootwear.us template-kid.com
teslathemes.com teslathemes.com
tfxiq.com testingads.pro
tizanidine4mgtablets.blogspot.com thedownloadfreeonlinegames.blogspot.com
tizanidine6mg.blogspot.com thegalerie.eu
tizanidinehcl4mgtabinfo.blogspot.com thejournal.ru
tksn.ru timdreby.com
tizanidineinteractions.blogspot.com
tizanidinepills.blogspot.com
tizanidinevscyclobenzaprine.blogspot.com
tmm-kurs.ru
toon-families.com
top-study.work top-study.work
topappspro.com topcar-krasnodar.ru
topsy.com torrent-to-magnet.com
torrent-newgames.com trafficgenius.xyz
tozup.com truemfilelj.gq
tracfone.com tsstcorpcddvdwshbbdriverfb.aircus.com
tracklead.net tubo360.com
trudogolik.net twinderbella.com
tubeline.biz u-cheats.ru
tzritel.tk uggsale.online
ukr-lit.com unimodemhalfduplefw.pen.io
unitygame3d.com unitexindia.com
universals.com.ua univerfiles.com
uprour.com validccseller.com
usacasino.com vduplo.ru
utiblog.fr veopornogratis.xxx
velpanex.ru verymes.xyz
vesnatehno.com viberdownload10.com
video--production.com villacoloniale.com
video-woman.com villakohlanta.nu
videojam.tv vkak.ru
vikistars.com vladimir.zrus.org
vkmusics.ru voluumtracker1.com
vtcdns.com vozbujdenie.com
vvon.co.uk vsdshnik.com
vut.com.ru
vysigy.su
vzglyadriv.kg
wahicbefa31.soup.io
wallpapersdesk.info wallpapersdesk.info
wapsite.me wallpaperstock.net
wareseeker.com warningwar.ru
wcb.su waterefficiency.co
webextract.profound.net waterpurifier.club
webpromotion.ae watsonrealtycorp.com
webshoppermac.com webalan.ru
wellcome2slovenia.ru webkeyit.com
wesharepics.site webscouter.net
wholesalejerseysgaa.com website-datenbank.de
windowssearch-exp.com website-stealer.nufaq.com
winx-play.ru webtherapy.ru
wosik-dach.service-for-web.de whosonmyserver.com
wpthemedetector.co.uk win-spy.com
wtsindia.in wineitudes.wordpress.com
www.pinnacle-bets.com wordpress-crew.net
www.souvenirua.com worldis.me
wowas31.ucoz.ru
ws.ampower.me
x-diesel.biz x-diesel.biz
xfire.com x69ty.ru
xkaz.org xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xmladserver.com xn----9sbubg3ambdfl1j.xn--p1ai
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn----btbdvdh4aafrfciljm6k.xn--p1ai xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn----dtbndd4ae7eub.top xn--80aafb2a.xn--p1ai
xn----itbeirbjbi7bc6bh2d.xn--p1ai xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xsfetish.org xn--oogle-wmc.com
y8games-free.com xxxnatelefon.ru
youtubologia.it yadro.ru
zacreditom.ru yeartwit.com
zajm-pod-zalog-nedvizhimosti.ru youandcredit.ru
zajm-zalog-krasnodar.ru yourothersite.com
zakazfutbolki.com zaidia.xhost.ro
zarepta.com zeg-distribution.com
zeroredirect10.com zigarettenonl.canalblog.com
zigarettenonlinekaufen1.bloog.pl zigarettenonlinekaufen2.drupalgardens.com
zrelaya.pw zirondelli.it
znakomstva-moskva77.ru
zona-aqua.ru
zoogdiesney.com
zrus.org
zygophyceous.womanstars.site

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1000n1.ru 0c47f8422d3f.com
100searchengines.com
123movies.love 123movies.love
12u.info 1millionusd.xyz
aa625d84f1587749c1ab011d6f269f7d64.com 273-fz.ru
4senses.co
5ws.dating-app.ru
7wind.ru
8si.ru
9icmzvn6.website
aa8b68101d388c446389283820863176e7.com
aac52006ec82a24e08b665f4db2b5013f7.com aac52006ec82a24e08b665f4db2b5013f7.com
aarbur.com activecampaign.dreamhosters.com
abcguru.xyz activepr.ru
accessoires-mode.in acunetix-referrer.com
add-add.men adhome.biz
adinterax.com adloads.com
admitad.com adultfullhd.com
advokateg.xyz
agecheckadult.com agecheckadult.com
akuhni.by all-number.com
allblogroll.com allknow.info
allcredits.su apilinkswiftco-a.akamaihd.net
alveris.ru apparel-offer.com
aosexkontakte.net arquapetrarca.info
apartmentbay.ru asrv-a.akamaihd.net
apibrowsesmartne-a.akamaihd.net avirasecureserver.com
apptester.tk azadnegar.com
australia-opening-times.com baditri.com
avon-novosib.ru balla-boo.se
avto-oligarh.ru bannertesting.com
avtoarenda.by bard-real.com.ua
backlinkwatch.com bashtime.ru
balook.com beauty-bracelet.com
baoxaydung.com.vn belstaffstore.online
bazaronline24.ru berlininsl.com
bet365.com bestofferswalkmydogouteveryday.gq
bestpornuha.com
betterhdporn.com betterhdporn.com
blackle.com bio.trade-jp.net
blog-hits.com blogqpot.com
bloglag.com blogrankers.com
bosman.pluto.ro bukleteg.xyz
brakehawk.com buy-forum.ru
buyantiviralwp.com buyessayonline19.blogspot.ru
buyk.host.sk buyk.host.sk
buypanicdisorderpill.com buyparajumpers.online
cardsdumps.com bytimedance.ru
cartierlove2u.com cashkitten-a.akamaihd.net
cartierloveringreplica.xyz cezartabac.ro
casinos4dummies.co.uk changde.58.com
ceotrk.com checkhit.com
cheapmarmot.online ci.ua
chuckguilford.com
cialisovercounteratwalmartusa.com cialisovercounteratwalmartusa.com
cienum.fr civilwartheater.com
compliance-jane.top cleanpcnow.com
continent-e.tv cleansearch.net
cookie-law-enforcement-hh.xyz coderstate.com
coolwallpapers4k.info com-secure.download
dalavia.ru coolgramgoods.com
dating-app.ru covadhosting.biz
deanmoore.ie d0t.ru
delfin-aqua.com.ua delitime.info
demenageur.com
digest-project.ru
digitalfaq.com
dipstar.org
discovertreasure-a.akamaihd.net discovertreasure-a.akamaihd.net
discovertreasurenow.com dnsrsearch.com
djihispano.com domaincrawler.com
domcran.net
domini.cat
dorratex.tn
downloadeer.net downloadeer.net
earl-brown.info ecommerce-seo.com
easyshoppermac.com electricwheelchairsarea.com
ecommerce-seo.org elektrozigarettekaufen1.devhub.com
elektrozigarette2.webs.com englate.com
energy-ua.com eonpal.com
energydiet24.ru eugenevaultstorage.com
englishgamer.com exlarseva.webblog.es
exhibitionplus.eu feargames.ru
extremepornos.net femmesdenudees.com
farmingworm.com filkhbr.com
favornews.com finteks.ru
fealq.com fitnesspiks.com
fickblock18.com focalink.com
flashahead.info freeuploader.com
forexgb.ru gamedayassist.com
free-gluten.ru gay-file.com
freemags.cc gaytube.com
freshmac.space gkvector.ru
froggytube.com glass-msk.ru
frvo.alptandem.ru gne8.com
gazporno.com gogalleryawesome.com
gcup.ru golmau.host.sk
get-free-traffic-now.com gorodservis.ru
greendream.com.ua guigyverpo.cf
guildebzh.info handicapvansarea.com
hd720kino.ru hatedriveapart.com
hdimagelib.com hdpixion.com
healbio.ru healmytrauma.info
hollywoodactress.info helvetia.com.ua
hosting-tracker.com hermesreplica.win
hotelcrocenzi.sm hulfingtonpost.com
howtoclean.club i-midias.net.br
hspline.com idegenvezeto.eu
hubbble.com ie.57883.net
informatiecentro.be ihtec2019.org
infoupdate.org imabase.com
interior-stickers.ru imadedinner.net
investmac.com imgata.com
iqoption.pro imitex-plus.ru
isitwp.com iqupdatetmz.win
ixora.pro itrevolution.cf
izhevsk.zrus.org iweblist.info
izoll.ru j-times.ru
j33x.com javitas.info
jongose.ninja jaxcube.info
kino-filmi.com karadene.com
kinocccp.net keywordsdoctor.com
kinofak.net klejonka.info
koptims.tiu.ru kollekcioner.ru
krassh.ru krasivye-devushki.net
ldrtrack.com lechenie-gemorroya.com
leftofcentrist.com livetsomudvekslingstudent.bloggersdelight.dk
lexaprogeneric.link localflirtbuddies.com
libertybilisim.com lost-alpha.ru
low-format.ru lovi-moment.com.ua
luxup.ru luxup.ru
mac-shield.com m0r0zk0-krava.ru
magazintiande.ru macresource.co.uk
makenahartlin.com matpre.top
martlinker.com metaxalonevstizanidine.blogspot.com
mcadamssupplyco.com miniature.io
medkletki.ru mp3films.ru
member-quiz.com musicas.baixar-musicas-gratis.com
metabar.ru musicpro.monster
methodsmarketing.com my-aladin.com
mex-annushka.ru myfreetutorials.com
mhtr.be myhydros.org
minegam.com na-telefon.biz
miniads.ca naphukete.ru
mixed-wrestling.ru naturalpharm.com.ua
mmofreegames.online net-radar.com
mmoguider.ru nfljerseyscheapbiz.us
mobstarr.com nicefloor.co.uk
monsterdivx.com onefilms.net
mural.co oneshotdate.com
musezone.su onstrapon.purplesphere.in
musflashtv.com open-odyssey.org
mydirtyhobby.com ordernorxx.com
mylesosibirsk.ru ownshop.cf
mymercy.info park.above.com
nac-bearings.ru pastaleads.com
nalogovyykodeks.ru pdn-4.com
nastydollars.com pinkduck.ga
net-profits.xyz plaff-go.ru
nisuturnetdgu.tk play.leadzupc.com
nonews.co ploenjitmedia.azurewebsites.net
nordvpn.com porn-w.org
notaria-desalas.com pornhub-forum.ga
notebook-pro.ru pornodojd.ru
novosibirsk.xrus.org pornoklad.net
o00.in pp-budpostach.com.ua
obiavo.kz preconnubial.usuby.site
oecnhs.info printingpeach.com
online-hit.info professionalwritingservices15.blogspot.ru
online247.ml prom23.ru
orel-reshka.net radiodigital.co
ownshop.win rangjued.com
partner-cdn.men reelheroes.net
pathwhelp.org repeatlogo.co.uk
pdamods.ru royalcar-ufa.ru
plastweb.ru rusoft-zone.ru
playfortuna-play.ru sabaapress.com
polytopesexempt.com sady-urala.ru
popunder.ru
pornohd1080.online
pornoxxx.com.mx
pricheski-video.com
psvita.ru
rabot.host.sk
randki-sex.com
rankscanner.com
ratemodels.net
redirect.trafficreceiver.club
replica-watch.ru
rutor.vip
ryetaw.com
sabizonline.com
saletool.ru saletool.ru
samoiedo.it sampleletters.net
savetubevideo.com saneyes.com
sbprabooks.com sankt-peterburg.nodup.ru
sbtdesign.co.uk sbwealthsolutions.ca
scanner-margo.top scanner-andrew.top
scanner-walt.top seeingmeerkat.com
searchaddis.com seksvideoonlain.com
seksotur.ru semrush.com
seo-traffic-ranking.info
seoanalyses.com seoanalyses.com
serdcenebolit.com seonetwizard.com
shiksabd.com sevendays.com.ua
sims-sims.ru sex-foto.pw
sinestesia.host.sk sex-tracker.de
sitiz.club sexsaoy.com
small-games.biz shitting.pro
soft1.ru sibtest.ru
softomix.net sirpornogratis.xxx
sohoindia.net sixcooler.de
sukarame.net skatestick.bid
supermodni.com.ua skutecznetabletkinaporostwlosow.pl
svarkagid.com sky-mine.ru
swagbucks.com skytraf.xyz
symbaloo.com slkrm.ru
smstraf.ru
spb.afora.ru
spinazdrav.ru
spl63.fr
strag-invest.ru
stretchmate.net
studiofmp.com
studiokamyk.com.pl
suhanpacktech.com
summerlinhomes411.info
superkanpo.com
tabakur77.com
takethatad.com
tedy.su
teenporn18.net
tehngr.ru tehngr.ru
thaismartloan.com theallgirlarcade.com
tiens2010.ru thefotosgratis.eu
tizanidine4mgstreetprice.blogspot.com themeforest.net
tizanidinepain.blogspot.com tizanidinehcl4mgtablet.blogspot.com
tnaionline.org top10-online-games.com
toon-families.com top10registrycleaners.com
topkarkas.com tourcroatia.co.uk
tracklead.net toyota.7zap.com
traffixer.com tvteleport.ru
traffmonster.info u17795.netangels.ru
tri-slona.org ugguk.online
truck-land.life ugogo.info
udayavani.com urdoot.win
ultramart.biz ustion.ru
umnovocaminho.com vapeface.club
unitexindia.com ventelnos.com
valmetrundan.se videochat.bz
velen.io videositename.com
venerologiya.com voloslove.ru
vids18.site vvpg.ru
vk-mus.ru vykup-avto-krasnodar.ru
vsesubwaysurfers.com vzubah.com
vzube.com w7s.ru
webkeyit.com webmasterhome.cn
website-speed-up.top witclub.info
websites-reviews.com womensterritory.ru
wellcome2slovenia.ru x-diesel.biz
westermarkanjou.se xn----ctbbcjd3dbsehgi.xn--p1ai
wormix-cheats.ru xn--1-8sbcpb0bdm8k6a.xn--p1ai
xaylapdiendanang.com xtraffic.plus
xclicks.net xxxtubesafari.com
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai you-shall-not-pass.is74.ru
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai youdao.com
xnxxandxvideos.com yun56.co
xxxguitars.com z-master.ru
yellowstonesafaritours.com za-music.mymobiplanet.com
yourporn.com zastenchivosti.net
ypmuseum.ru zigzog.ru
zakonobosago.ru
zapatosenventa.info
zatjmuzu.info
zed21.net
zmoda.hostreo.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.68 0; 10.1.1.87 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2911 ### Version: V4.2022.02.2912
### Updated: Fri Feb 11 10:03:37 UTC 2022 ### Updated: Fri Feb 11 22:01:12 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2911 ### Version: V4.2022.02.2912
### Updated: Fri Feb 11 10:03:37 UTC 2022 ### Updated: Fri Feb 11 22:01:12 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##