vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.01.2871 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-01-22 22:07:04 +00:00
parent 5308c63cc0
commit 787f8a95ab
47 changed files with 98164 additions and 98154 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2870 #### Version: V4.2022.01.2871
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2870 #### Version: V4.2022.01.2871
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.01.2870 #### Version: V4.2022.01.2871
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12134
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.01.2870 #### Version: V4.2022.01.2871
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12138
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12138
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12138
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12138
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 1; 10.1.1.62 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 0; 10.1.1.62 0;
127.0.0.1 0; 127.0.0.1 0;

12138
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 1; 10.1.1.62 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 0; 10.1.1.62 0;
127.0.0.1 0; 127.0.0.1 0;

12138
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 1; 10.1.1.62 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 0; 10.1.1.62 0;
127.0.0.1 0; 127.0.0.1 0;

12138
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2870 2871

12138
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12138
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.196.122 0; 52.250.64.78 0;

12138
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 1; 10.1.1.62 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 0; 10.1.1.62 0;
127.0.0.1 0; 127.0.0.1 0;

12138
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 1; 10.1.1.62 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 0; 10.1.1.62 0;
127.0.0.1 0; 127.0.0.1 0;

12138
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 1; 10.1.1.62 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 0; 10.1.1.62 0;
127.0.0.1 0; 127.0.0.1 0;

12138
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.196.122 0; 52.250.64.78 0;

12138
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.121.196.122 0; 52.250.64.78 0;

12138
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 1; 10.1.1.62 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Apexoo Aboundex
BackStreet Chlooe
CCBot Gigabot
Mister PiX Nutch
Msrabot SpaceBison
NetZIP URLy.Warning
OutclicksBot VB Project
Pi-Monster Vacuum
WebPix WebCopier
ZoomBot sp_auditbot

298
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker
404enemy 404enemy
80legs
AIBOT AIBOT
ALittle Client ASPSeek
AhrefsBot
AiHitBot AiHitBot
Alexibot Anarchy
Alligator Ankit
Anarchy99 Aspiegel
Apexoo Asterias
BBBike Attach
AwarioSmartBot
BDCbot
BLEXBot
BackDoorBot
BackStreet BackStreet
BacklinkCrawler Backlink-Ceck
Bandit
Barkrowler
Battleztar Bazinga
BetaBot BetaBot
Bigfoot Black Hole
Bitacle
Blackboard
BlowFish BlowFish
Boardreader
Bolt
Brandprotect Brandprotect
Buddy
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
Bullseye
BunnySlippers BunnySlippers
BuzzSumo
CCBot CCBot
Cegbfeieh Calculon
Cocolyzebot ChinaClaw
Copier Chlooe
Citoid
Collector
CopyRightCheck
Copyscape Copyscape
Cosmos
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler
CrunchBot
Curious Curious
CyotekWebCopy
DIIbot DIIbot
DSearch
DataCha0s DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Devil Digincore
DigitalPebble Disco
Dirbuster
Discobot
Discoverybot
Dispatch Dispatch
DnyzBot
DomCopBot
DomainAppender DomainAppender
DomainCrawler Dotbot
DomainSigmaCrawler ECCP/1.0
Domains Project EirGrabber
ExtLinksBot ExtLinksBot
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms Foobot
FHscan FrontPage
FemtosearchBot
Flunky
Fuzz
FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo GalaxyBot
GermCrawler GermCrawler
GetRight GetRight
Gigabot GetWeb
Getintent
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
Gotit Gotit
GrabNet GrabNet
GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HMView HMView
HTMLparser HTTP::Lite
HTTrack
HaosouSpider HaosouSpider
Harvest HonoluluBot
Havij
Heritrix
Humanlinks
HybridBot HybridBot
IlseBot IDBTE4M
IndeedBot Image Fetch
InfoTekies InfoNaviRobot
Intelliseek Intelliseek
InterGET
Iskanie Iskanie
IstellaBot JOC Web Spider
Jbrofuzz Jbrofuzz
JetCar JetCar
JikeSpider JikeSpider
Joomla
Jorgee
JustView JustView
Jyxobot
Kenjin Spider Kenjin Spider
Kinza LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot
Leap Leap
LieBaoFast Linkbot
Lightspeedsystems
Likse
Linkdexbot
LinkextractorPro LinkextractorPro
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider Litemage_walker
MIDown tool MIDown tool
MIIxpc MIIxpc
MJ12bot
MQQBrowser
MSIECrawler MSIECrawler
Magnet
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
MarkMonitor Majestic-SEO
Mass Downloader
Masscan
Mata Hari Mata Hari
Mb2345Browser Mb2345Browser
Mediatoolkitbot MegaIndex.ru
Microsoft URL Control Metauri
Moblie Safari MicroMessenger
Mojolicious Musobot
Morfeus Fucking Scanner
Mozlila
NICErsPRO
NPbot
Name Intelligence
Navroad Navroad
Needle Needle
Net Vampire Nessus
NetMechanic NetMechanic
Niki-bot NetZIP
Ninja Netcraft
Nmap Nikto
Offline Navigator Nimbostratus
Not
Octopus
Offline Explorer
OnCrawl OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openvas OpenVAS
OutclicksBot OrangeSpider
OutfoxBot OutfoxBot
PHPCrawl PHPCrawl
Page Analyzer Page Analyzer
Panscient PageAnalyzer
Pi-Monster PageGrabber
Picscout Pavuk
Pimonster Pimonster
PleaseCrawl PleaseCrawl
Pockey Pu_iN
ProWebWalker PyCurl
Probethenet QueryN Metasearch
RSSingBot Rankivabot
RankActive RankurBot
RankFlex
Re-re
RealDownload RealDownload
RebelMouse
Recorder Recorder
RepoMonkey RocketCrawler
Ripper Rogerbot
SBIder
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
ScanAlert SEOprofiler
SEOstats
Scanbot Scanbot
Scrapy Scrapy
ScreenerBot
Seekport
SemanticJuice SemanticJuice
SentiBot Semrush
SeobilityBot Shodan
Site Sucker SiteExplorer
SiteCheckerBotCrawler SiteLockSpider
Sitevigil Sitevigil
SlySearch SlySearch
SmartDownload SmartDownload
Snapbot
Snoopy
Sociscraper
Sogou web spider
Sosospider
Sottopop
SpaceBison SpaceBison
Spammen SpankBot
Spanner
Spinn3r
Sqworm Sqworm
Steeler Steeler
Sucker Stripper
Sucuri SurveyBot
SuperHTTP Suzuran
Surfbot Swiftbot
Szukacz
T8Abot T8Abot
TeleportPro Telesoft
Telesphorep The Intraformant
Toweyabot TheNomad
Tracemyfile TightTwatBot
Typhoeus Toata
Turnitin
Twice
UnisterBot UnisterBot
Upflow
Vacuum Vacuum
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler Voil
VidibleScraper Voltron
WASALive-Bot WBSearchBot
WEBDAV WEBDAV
WPScan WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
Web Auto Wallpapers/3.0
Web Enhancer WeSEE
Web Fetch
Web Pix Web Pix
Web Sauger Web Sucker
WebAuto WebAuto
WebCopier WebCollage
WebEnhancer
WebFetch WebFetch
WebGo IS WebLeacher
WebPix WebPix
WebWhacker WebStripper
WebZIP
WebmasterWorldForumBot
Webshag Webshag
Website Quester
Webster Webster
Whack
Who.is Bot Who.is Bot
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wprecon Woobot
Xaldon WebSpider Xaldon WebSpider
Xenu Xaldon_WebSpider
YoudaoBot
Zade
Zauba
Zeus Zeus
Zitebot
ZmEu
ZoomBot ZoomBot
ZumBot
adscanner adscanner
arquivo.pt
autoemailspider
check1.exe check1.exe
clark-crawler clark-crawler
com.plumanalytics demandbase-bot
crawl.sogou.com domainsproject.org
crawler4j
dataforseo.com
evc-batch evc-batch
gopher gopher
instabid heritrix
internetVista monitor ips-agent
lwp-request meanpathbot
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
page scorer pcBrowser
s1z.ru plumanalytics
seoscanners probe-image-size
ripz
satoristudio.net
scalaj-http
seobility
seocompany.store
serpstatbot
sitechecker.pro sitechecker.pro
siteripz siteripz
sogouspider
sp_auditbot sp_auditbot
tAkeOut
trendiction.com trendiction.com
trendiction.de ubermetrics-technologies.com
webpros.com webgains-bot
webprosbot webmeup-crawler
xpymep1.exe x09Mozilla
zauba.io zauba.io
zgrab

308
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 360Spider
404enemy 404enemy
AIBOT ADmantX
AfD-Verbotsverfahren Aboundex
Acunetix
AhrefsBot AhrefsBot
AllSubmitter AiHitBot
Alligator
AlphaBot AlphaBot
Anarchie Anarchy
Ankit Asterias
Anthill
Apexoo
Atomseobot Atomseobot
Attach Attach
AwarioSmartBot AwarioRssBot
BBBike BDFetch
BackWeb BackDoorBot
BacklinkCrawler BacklinkCrawler
Barkrowler Bandit
BatchFTP BatchFTP
BetaBot Bigfoot
Blow Bitacle
Bolt Black Hole
BotALot Boardreader
Brandprotect
Buck Buck
BuiltBotTough Buddy
BuiltWith
Bullseye Bullseye
BunnySlippers BunnySlippers
CCBot Cegbfeieh
Calculon CensysInspect
CheTeam
CheeseBot CheeseBot
ChinaClaw Citoid
Cogentbot Claritybot
Copyscape CopyRightCheck
Craftbot
Crawling at Home Project Crawling at Home Project
Curious
Custo Custo
DatabaseDriverMysqli CyotekWebCopy
DIIbot
DSearch
Demon Demon
Deusu Digincore
Devil DigitalPebble
Dirbuster Dirbuster
Disco
Discoverybot Discoverybot
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot DomCopBot
DomainAppender DomainAppender
DomainCrawler DomainCrawler
DomainStatsBot
Dotbot Dotbot
EMail Wolf ECCP/1.0
Ebingbong EasyDL
Ecxi
EirGrabber EirGrabber
EroCrawler
Evil
ExtLinksBot
ExtractorPro
Extreme Picture Finder
EyeNetIE
FDM FDM
Fimap
Firefox/7.0 Firefox/7.0
FlashGet FrontPage
Foobot
Freeuploader
Fuzz Fuzz
Fyrebot Fyrebot
GetRight GT::WWW
GalaxyBot
Genieo
GetWeb GetWeb
Getintent Getintent
Gigabot Gigabot
Go-Ahead-Got-It GoZilla
Gotit
Grafula Grafula
GrapeFX GrapeFX
GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HMView HMView
HTTP::Lite HTMLparser
Humanlinks Haansoft
Havij
Hloader
IDBTE4M
Iblog Iblog
Id-search IlseBot
Image Fetch
Image Sucker Image Sucker
IndeedBot IndeedBot
InfoNaviRobot InfoTekies
Intelliseek Intelliseek
Internet Ninja Internet Ninja
Iria
JamesBOT JamesBOT
JetCar Jbrofuzz
Jetty Jetty
Jorgee JikeSpider
Kenjin Spider Joomla
Keyword Density Keyword Density
Larbin Kinza
LexiBot Leap
LieBaoFast Lightspeedsystems
Likse Likse
LinkWalker Lipperhey
Linkdexbot
LinkextractorPro
Lmspider Lmspider
MIIxpc Ltx71
MIDown tool
MJ12bot MJ12bot
MQQBrowser
Mag-Net Mag-Net
Majestic12 Magnet
Majestic SEO
MarkMonitor
MarkWatch
Mass Downloader
Masscan Masscan
Mata Hari
MauiBot
Mb2345Browser Mb2345Browser
Mediatoolkitbot
Metauri
Microsoft Data Access Microsoft Data Access
Moblie Safari Mojeek
Morfeus Fucking Scanner
Msrabot Msrabot
Musobot
NPbot NPbot
Navroad Nameprotect
Net Vampire NearSite
NetMechanic NetSpider
NetZIP Netcraft
Nibbler Nikto
Ninja NimbleCrawler
Nuclei Not
Nutch Nutch
OpenLinkProfiler OpenLinkProfiler
OpenVAS Openfind
Openvas
OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageGrabber PageThing.com
Pandalytics PeoplePal
Pi-Monster Petalbot
Picscout Picsearch
PictureFinder
Piepmatz Piepmatz
Pimonster PxBroker
Pixray
Pockey
Pu_iN
Pump
PyCurl PyCurl
RankActive Quick-Crawler
RankActiveLinkBot RSSingBot
RankFlex
RankingBot RankingBot
RankingBot2 Re-re
Rankivabot ReGet
Reaper Ripper
Recorder RocketCrawler
Rogerbot
SBIder SBIder
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler ScanAlert
SEOprofiler
SalesIntelligent
Scanbot Scanbot
ScoutJet Searchestate
Scrapy SemanticJuice
Screaming
ScreenerBot
SemrushBot
SentiBot SentiBot
SeoSiteCheckup SeoSiteCheckup
Site Sucker SeobilityBot
SiteCheckerBotCrawler Shodan
SiteExplorer Siteimprove
SiteSnagger Snake
SiteSucker Snapbot
Snoopy Snoopy
Sogou web spider Sottopop
SpaceBison
Spammen Spammen
SpankBot
Spbot Spbot
Sqlworm Spinn3r
SputnikBot
Sqlmap
Sqworm
Stripper
Sucker
Sucuri
SuperBot SuperBot
SuperHTTP SurveyBot
T8Abot Szukacz
TeleportPro T0PHackTeam
Teleport
Telesphoreo Telesphoreo
TightTwatBot The Intraformant
Titan Titan
Toata Toweyabot
Tracemyfile TwengaBot
Trendiction
True_Robot
Turingos
TurnitinBot
Twice
Upflow
V-BOT V-BOT
Vacuum VB Project
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VidibleScraper
Voil Voil
Voltron
WEBDAV WEBDAV
WISENutbot
WPScan WPScan
WWW-Collector-E WWW-Collector-E
WallpapersHD WWW::Mechanize
Web Fuck WWWOFFLE
Web Pix Web Pix
Web Sucker Web Sauger
WebBandit WebAuto
WebCollage
WebEnhancer
WebFetch WebFetch
WebGo IS WebFuck
WebLeacher
WebReaper WebReaper
WebSauger
WebStripper WebStripper
WebWhacker WebWhacker
WebZIP
WebsiteExtractor
WebsiteQuester
Webster Webster
Whack Whack
Whacker
Whatweb
Who.is Bot Who.is Bot
Widow
WinHTTrack
Wonderbot Wonderbot
Woobot Woobot
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider YoudaoBot
Zauba Zade
adscanner Zermelo
arquivo-web-crawler Zeus
arquivo.pt Zitebot
autoemailspider ZoomBot
cah.io.community ZumBot
check1.exe
clark-crawler clark-crawler
eCatch coccocbot-web
internetVista monitor crawl.sogou.com
crawler.feedback
domainsproject.org
heritrix
instabid
ips-agent
isitwp.com
iubenda-radar
lwp-trivial
meanpathbot meanpathbot
netEstate NE Crawler page scorer
pcBrowser satoristudio.net
plumanalytics scalaj-http
s1z.ru seobility
scan.lol
seocompany.store
serpstatbot
sitechecker.pro sitechecker.pro
trendiction.de siteripz
ubermetrics-technologies.com sysscan
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
webprosbot webpros.com
x09Mozilla
zauba.io zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
apccargo.com bluesalt.co
apiwoodensealcom-a.akamaihd.net buyhoverboard.com
cementaresearch.se elektronischezigarettekaufen1.myblog.de
compliance-don.top felizporno.com
hybrid.ru hahashka.ru
marwer.info jwss.cc
onemactrckr.com linkrr.com
swinon.site picurams.pw
trustaffs.com shinikiev.com.ua
tyumen.xrus.org soheavyblog.com

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0c47f8422d3f.com
101billion.com 101billion.com
123movies.love 13tabs.com
12u.info 1j7740kd.website
1bet.com 256bit.by
1millionusd.xyz 2ads.co.uk
300richmond.co.nz 3rbseyes.com
5kstore.com 5i2.net
8lufu.com 7minuteworkout.com
aa81bf391151884adfa3dd677e41f94be1.com 83net.jp
abiente.ru 8xv8.com
abwa.tk 91abcw.com
adktrailmap.com 991mostfm.co.id
admanaerofoil.com 999webdesign.com
adsland.men a1security.com.ua
aerogo.com aac52006ec82a24e08b665f4db2b5013f7.com
agahinameh.com acc.eu.org
adf.ly
adidas.frwebs.fr
adman.se
adultfullhd.com
adventureparkcostarica.com
afslankpillen2017nl.eu
ahrntal.verymes.xyz
aihelen.net aihelen.net
alice.it albuteroli.com
alienwheel.es alekseevec.ru
allknow.info alert-hgd.xyz
allornamenti.com alfa9.com
allwomen.info all4invest.info
amanda-porn.ga alpharma.net
amyfoxfitness.com amateurgalls.com
angigreene.com amateurlivechat.org
animebox.com.ua amazingninja.com
anmysite.com ampower.me
anticrawler.org angkortours.vn
apccargo.com anlimebel.kiev.ua
apibrowsesmartne-a.akamaihd.net api.stathat.com
arkartex.ru apishops.ru
atmovs.com apparel-offer.com
appfixing.space
approved.su
arenda-avtoprokat-krasnodar.ru
arendatora.ru
arius.tech
arkkivoltti.net
articlesdirectoryme.info
atleticpharm.org
australia-opening-times.com
autoua.top
aversis.net
aviav.ru.com
avtoarenda.by
azartclub.org azartclub.org
azinoofficial777.ru
backgroundpictures.net
baersaratov.ru baersaratov.ru
bagcionderlab.com bagcionderlab.com
bagsonsale.online baksman.com
bang-hotties.com balook.com
basedecor.ru bangla-chat-uk-paralud.ga
battleforupdating.site bankmib.ru
bazaronline24.ru bankofthewext.com
begalka.xyz bdsmgalls.net
bestdooz.com beget.tech
bestpriceninja.com bestcurs.org
bestsexyblog.com bestplacetobuyeriacta.jw.lt
bet365.com betune.onlinewebshop.net
bezsporno.ru bif-ru.info
bigpenisguide.com bin-brokers.com
billiard-classic.com.ua bingo8888.com
bjetjt.com
bkgr.se
blackbabesporn.com
blackcurranthumidifiers.site blackcurranthumidifiers.site
blackplanet.com blobar.org
blavia.00author.com
bloggedporn.com
blogporn.in
bluerobot.info bluerobot.info
boazpower.com boleznikogi.com
bocoarchives.org
bookmaker-bet.com
brainboostingsupplements.org brainboostingsupplements.org
braintobe.win brewdom.ru
brainzpod.win bristolhostel.com
brendbutik.ru bumskontakte.org
brimstonehillfortress.org buyhoverboard.com
burkesales.com cakesplus.com.au
buynorxx.com calcularpagerank.com
buypanicdisorderpill.com canyougethighofftizanidine.blogspot.com
buzzonclick.com carina-sy.de
cacheimages.com cezartabac.ro
calvet.altervista.org cfacarrosserie74.com
cartierbracelet.xyz checktext.org
cartujano-pre.de chiblackhawks-jerseys.com
casinopinup-wins.com chocolateslim-en-espana.com
cbb1smartlist12.click chocolateslim-original.com
ccbill.com chocoslim.pro
cercacamion.it ciekawinki.pl
check-this-out-now.online clarithromycin500mg.com
checkhit.com clickbank.net
christianlouboutinreplica.pw club-samodelkin.ru
cleanersoft.com coaching-netz.info
clevernt.com colehaanoutlet.store
clickintext.net compliance-andrew.xyz
clodo.ru compliance-elena.top
co.lumb.co compliance-olga.top
cocolyze.com coolwallpapers-hd.com
coderstate.com coop-gamers.ru
codysbbq.com cosmeticswomens-womensports.rhcloud.com
collegerentals.com countbertwistdisp26.soup.io
com-supportcenter.website cowblog.fr
comeondog.info cpabegins.ru
compliance-george.xyz credit-respect.ru
compliance-ivan.top cv.wallhade.co
compliance-mark.top cwetochki.ru
compliance-viktor.top d0t.ru
cookie-law-enforcement-ee.xyz darkbooks.org
copenergo.ru davebestdeals.com
copyrightclaims.org delitime.info
creatives.adbetclickin.pink descargar-musica-gratis.net
crest-poloski.ru designdevise.com
crystalslot.com dev.citetick.com
culmimg.pw dfwu1019.info
cypernhuset.se dgfitness.co
cyse.tk
dame-ns.kz
daptravel.com
dealighted.com
decoratinghomes.ga
decorazilla.com
dekoration.us
dekorkeramik.ru
deluxedumps.com
demenageur.com
demian.kz
detoxmed24.ru
digital-sale.su
dirtpics.pw dirtpics.pw
discovertreasurenow.com djonwatch.ru
divatraffic.com dokfilms.net
diy-handmade-ideas.com domain.webkeyit.com
djekxa.ru domznaniy.ru
djihispano.com download-walpaperhd.blogspot.com
dlya-android.org downloads-whatsapp.com
dogoimage.com drhomes.biz
dojki365.online drunkenstepfather.com
domain-tracker.com dtnlyss.com
doublepimp.com
drillsaw.ru
drugstoreforyou.com
drupa.com
dustyorate.com
dvd-famille.com dvd-famille.com
dyshagi.ru dvr.biz.ua
e2click.com e-poker-2005.com
efnor-ac.com eachdayisagift.review
electro-prom.com easync.io
elektrischezigarette1.blog.pl ecookna.com.ua
elektronischezigarette2.mex.tl ed-shop01.ru
elvenar.com editmedios.com
englishgamer.com ekspertmed.com
enpolis.ru elektronischezigarettekaufen2.tumblr.com
enternet.ee elektrozigaretten1.tumblr.com
eredijovon.com elitepcgames.com
escortslet.net elvenmachine.com
espaceinventoristes.com eonpal.com
etur.ru eropornosex.ru
evehemming.blogspot.com.au errorfixing.space
everflora.ru escortplius.com
exchangeit.gq estibot.com
extractorandburner.com european-torches.ru
ezaz.info evening-dating-club.info
ezigarettekaufen1.hpage.com expresstoplivo.ru
f012.de extrabot.com
fable.in.ua extstat.com
feeriaclub.ru eyelike.com.ua
fickkontaktehobbyhuren.com f0815.de
filefilter.weebly.com facebook-mobile.xyz
filmci.pro facecup.top
fanoboi.com
fanpagerobot.com
fapgon.com
fastcrawl.com
favornews.com
faz99.com
fbdownloader.com
femmesdenudees.com
fermersovet.ru
filmania-x.ru
fishingwholesale.us fishingwholesale.us
flowersbazar.com fitness-video.net
fotopop.club floridamhca.org
fotoxxxru.com forex-indextop20.ru
franecki.net forex21.ru
frankofficial.ru forexgb.ru
free-share-buttons.blogspot.com forpostlock.ru
free-share-buttons.top fortunejack.com
freemags.cc forumprofi.de
freesitetest.com foxjuegos.com
freeuploader.com
freshwallpapers.info
friendflnder.com friendflnder.com
frighteningremain.cf fuckingawesome.com
fyl.com.ru fuel-gas.com
gallerily.com fungamelands.com
gambarxkata.co gabeshop.ru
gameprimary.com galleryawesome.com
gamevalue7.weebly.com gameplexcity.com
gate5.co.za gamesprite.me
gamewrath.com
gaygalls.net
gdebestkupit.ru gdebestkupit.ru
ghazel.ru geilehausfrauen.net
gemara.com
get-your-social-buttons.info
getmiro.com
gigapeta.com gigapeta.com
girlsatgames.ru gilsonchiro.xyz
gk-atlant.info
go2jump.org go2jump.org
gobongo.info goldandcard.ru
gogalleryawesome.com googlsucks.com
gonextmedia.com gotorussia.com
goo.ne.jp greenidesign.co
googlepositions.com gribokstop.com
gorabagrata.ru growmyfunds.ca
gourcy.altervista.org gsasearchengineranker.xyz
graphicwe.org gsasearchenginerankerhelp.com
growshop.es gymi.name
gsmtlf.ru hasfun.com
hanink.biz.ly hayate.biz
hatdc.org hdhc.ru
headpress.ru hdpixion.com
healthcarestore.info
heartofpayne.xyz heartofpayne.xyz
hesteel.pl hermesbirkinhandbagoutlets.com
hitsbox.info hobbyhuren-datenbank.com
hiwibyh.bugs3.com home.myplaycity.com
home-task.com horoshieokna.com
homedecorpicture.us host-protection.com
homedo.fabpage.com hotel-mkad.ru
hotdl.in houtings.xyz
hoztorg-opt.ru howmuchdoestizanidinecost.blogspot.com
hreade.com hyhj.info
huhn.altervista.org
hustoon.over-blog.com
ibmdatamanagement.co
iflycapetown.co.za
ilovevitaly.xyz
images-free.net images-free.net
imediadesk.com
iminent.com iminent.com
innoslicon.com imperia31.ru
int.search.mywebsearch.com incomekey.net
iusstf.org infazavr.ru
iweblist.info infektsii.com
j-times.ru insider.pro
jav-way.site ip-guide.com
javcoast.com ipornox.xxx
javrip.net iyasimasennka.com
jbl-charge.info jav-fetish.com
jerseyschinabizwholesale.us jerseywholesalebizchina.com
joyceblog.top jjbabskoe.ru
kakablog.net junketjuice.blogspot.com
kfon.eu jyvopys.com
khadastoafarde.tk kanimage.com
kinohall.ru karpun-iris.ru
kazan.zrus.org
kenaba.su
keywordhut.com
kino-key.info
klejonka.info
klikbonus.com
klitimg.pw
kollekcioner.ru
kostenlos-sexvideos.com
kotaku.com
kountrylife.com
kovrenok.ru
krasula.pp.ua krasula.pp.ua
kvartir-remont.biz kredit-blog.ru
laapp.com laexotic.com
labelwater.se lalalove.ru
landmania.ru leadn.pl
le-clos-des-alouettes.com lerporn.info
legalrc.biz linkbolic.com
lernur.net
letmacwork.world
letslowbefast.site
lincolntheatre.com
linkbux.ru linkbux.ru
linksharingt.com
lion.bolegapakistan.com
listiseltemournlan.gq listiseltemournlan.gq
livetsomudvekslingstudent.bloggersdelight.dk lizace.com
loginduepunti.it ljusihus.se
lombia.co llastbuy.ru
lrac.info locustdesign.co
ltvperf.com loftdigital.eu
lubetube.com longgreen.info
lost-alpha.ru
luchshie-topcasino.ru
luxup.ru
lyrics.home-task.com lyrics.home-task.com
madot.onlinewebshop.net maglid.ru
manve.info mailemedicinals.com
manyresultshub-a.akamaihd.net makedo.ru
masserect.com mararoom.ru
master-muznachas.ru marwer.info
matb3aa.com matrixalchemy.com
max-p.men maximpartnerspr.com
maxxtor.eu meble-bogart.info
mebel-ekb.com medi-fitt.hu
med-zdorovie.com.ua megagrabber.ru
mediafresh.online megahdporno.net
meetingrainstorm.bid metalonly.info
meltwater.com mezaruk.info
member-quiz.com micasainvest.com
mere.host.sk microsoftportal.net
mielec.pl midst.eu
mindeyegames.com minegam.com
mir-business-24.ru mixed-wrestling.ru
misswell.net mlf.hordo.win
monetizer.com-01.site mobplayer.net
moneyteam24.com mojowhois.com
mosrif.ru monclercheap.xyz
most-kerch.org moncleroutletonline.pw
mrinsidesales.com moonci.ru
mrvideospornogratis.xxx moscow.online-podarki.com
mug-na-chas-moscow.ru msfsaar.de
muschisexbilder.com multgo.ru
musicspire.online musicas.baixar-musicas-gratis.com
mvpicton.co.uk musicktab.com
myftpupload.com my-aladin.com
myindospace.com mylesosibirsk.ru
myperiod.club myliveblog.ru
myprintscreen.com mylovelibrabry.com
nagdak.ru nakozhe.com
namecrumilchlet.tk networkcheck.xyz
nardulan.com nfljerseyscheapbiz.us
naughtyconnect.com niki-mlt.ru
navalwiki.info novosti-hi-tech.ru
negociosdasha.com nw-servis.ru
negral.pluto.ro o-dachnik.ru
net-profits.xyz obesidadealgarve.com
newstudio.tv obiavo.uz
nfljerseysforsalewholesaler.com oivcvx.website
nikkiewart.ru olympescort.com
ningessaybe.me onko-24.com
nisuturnetdgu.tk onlinadverts.com
noscrapleftbehind.co
november-lax.com
nrv.co.za
oktube.ru
okuos.com
online-hd.pl
onlineslotmaschine.com onlineslotmaschine.com
onlinewritingjobs17.blogspot.ru onlinetvseries.me
onlythegames.com openfrost.com
oranga.host.sk optitrade24.com
orenburg-gsm.ru paramountmarble.co.uk
orgasmus-virtual.com partnerworkroom.men
orion-code-access.net
osb.se11.ru
ossmalta.com
outpersonals.com
owohho.com
oynat.info
ozas.net
papasdelivery.ru
par-fallen.ga
parlament.biz
partner-high.men
partner-trustworthy.men
partnerline.men
partners-ship.pro
parvezmia.xyz
pathwhelp.org pathwhelp.org
pay2me.pl perso.wanadoo.es
paydayloanslocal.com pharmacyincity.com
pcimforum.com phuketscreen.com
pearlisland.ru picphotos.net
perfectplanned.com picturesify.com
pestomou.info pinapchik.com
photo-clip.ru pk-services.ru
pic-re.blogspot.com pliks.pl
picsforkeywordsuggestion.com podshipniki-ntn.ru
picurams.pw pokemongooo.ml
pinup-casino1.ru polska-poezja.com
pizda.lol porndroids.com
pizza-imperia.com pornhub-ru.com
plaff-go.ru pornoelita.info
poffet.net pornoforadult.com
pokerniydom.ru pornzone.tv
popcash.net postclass.com
popmarker.com pretty-mart.com
pornhubforum.tk printie.com
porno-transsexuals.ru privatefx.all4invest.info
potolokelekor.ru prizesbook.online
predominant-invent.tk pro-tec.kz
pricheski-video.com prod2016.com
progonrumarket.ru profitsport.club
propranolol40mg.blogspot.com prom23.ru
psoriasis-file.trade prostitutki-belgoroda.org
queerspace.com psvita.ru
quelle.ru q-moto.ru
quickchange.cc qcstrtvt.bloger.index.hr
rainbowice.ru quickcashlimited.com
rank-checker.online quit-smoking.ga
rankexperience.com r-e-f-e-r-e-r.com
ranksonic.info rapidokbrain.com
rapidgator-porn.ga raschtextil.com.ua
razvratnoe.org refads.pro
realizmobi.com reklamuss.ru
realmonte.net remote-dba.de
registry-clean-up.net renhacklids.tk
registrydomainservices.com responsinator.com
reklama-i-rabota.ru ricorsogiustizia.org
remont-komputerov-notebook.ru robotixix.com
rentehno.ru rocis.site
repeatlogo.co.uk roznica.com.ua
retreatia.com rusenvironmental.net
rewardit.com sad-torg.com.ua
roleforum.ru savingsslider-a.akamaihd.net
rvtv.ru scanhub.ru
s-forum.biz scanner-alexa.top
salutmontreal.com scanner-josh.top
sanidumps.com scanner-mark.top
sashagreyblog.ga schoolfiles.net
sbprabooks.com scmor.ilxc.cc
scanner-alex.top scrapy.org
scansafe.net seasaltwithfood.com
scrnet.biz.ua security60-e.com
semprofile.com seemoreresultshu-a.akamaihd.net
seo-traffic-ranking.info sel-hoz.com
seocheki.net semalt.com
seolab.top serw.clicksor.com
seonetwizard.com sexiporno.net
seorankinglinks.us sharebutton.org
seozoom.it sibvitr.ru
serpstat.com sisiynas.ru
serviporno.com skatestick.bid
sexphoto.site slkrm.ru
sextracker.de sms2x2.ru
ship-marvel.co.ua socialseet.ru
shoppingmiracles.co.uk sockshare.net
sildenafilcitratemed.com spabali.org
silvermature.net startwp.org
sitebeam.net stathat.com
sitevaluation.org stefanbakosab.se
sitevalued.com summerlinhomes411.info
skachat-besplatno-obrazcy.ru supermama.top
slujbauborki.ru supermodni.com.ua
smartadserver.com supervesti.ru
snsdeainavi.info sureone.pro
social-search.me swagbucks.com
sodexo.com swinon.site
sotkal.lark.ru sygraem.com
souvenirua.com syvertsen-da.no
sovetogorod.ru taaaak.com
spaceshipad.com tagil.zrus.org
spasswelt.net techart24.com
spinazdrav.ru tecnoteakviareggio.it
start.myplaycity.com tecspb.ru
starwars.wikia.com teencastingporn.com
strana-solnca.ru telefonsexsofort.tk
streamin.to tfxiq.com
students-cheapskate.ml the-trader.net
surcentro.com thegolfclub.info
surgut.zrus.org thenetinfo.com
tacbibirfa.tk thexart.club
takeprofitsystem.com ticketsys.inetwd.com
tedxrj.com tizanidinehcl2mgsideeffects.blogspot.com
teesdaleflyballclub.co.uk tizanidinehydrochloride4mgstreetvalue.blogspot.com
thaisamkok.com tizanidineoral.blogspot.com
thedownloadfreeonlinegames.blogspot.com tizanidinerecreationaluse.blogspot.com
theheroes.ru tkanorganizma.ru
thepornsex.org tmm-kurs.ru
thetattoohut.com toon-families.com
tizanidineandhydrocodone.blogspot.com tootoo.to
tizanidinepills.blogspot.com touchmods.fr
top1-seo-service.com toursmaps.com
topshef.ru toy-shop.top
torrentgamer.net toyota.7zap.com
track112.site tradedeals.biz
tracxn.com traffic-club.info
trafficjunky.com treasuretrack-a.akamaihd.net
trafficreceiver.club try-rx.com
trustedmaccleaner.com
tsstcorpcddvdwshbbdriverfb.aircus.com
tuckpointingmasonrysystems.com
tupper-posuda.ru
tupper-shop.ru tupper-shop.ru
twu.com.ua twinderbella.com
typimga.pw uchebavchehii.ru
u-cheats.ru udsgame.online
uhodzalijami.ru uhodzalijami.ru
ukrtvir.com.ua ukrtextbook.com
umekana.ru ukrup.com
umg-stroy.ru
unacittaconte.org unacittaconte.org
univerfiles.com underthesite.com
uptimebot.net universals.com.ua
urlcut.ru upstore.me
urzedowski.eu urll.eu
uzporno.mobi usadacha.net
v-doc.co userequip.com
valmetrundan.se uvozdeckych.info
vchulkah.net vbikse.com
vesnatehno.ru video-chat.in
via-energy-cumpara.com videochat.bz
viagraneggrx.com
viagroid.ru
video-chat.cn
videochat.mx
videos-for-your-business.com
videospornogratisx.net
vids18.site vids18.site
vitanail.ru vilingstore.net
vitoriacabos.com vinsit.ru
viven.host.sk viteonlusarezzo.it
vrnelectro.ru voloslove.ru
vsdshnik.com vostoktrade.info
vsexkontakte.net wanker.us
vysigy.su waycash.net
vzlomfb.com wdrake.com
w3data.co webads.co.nz
w3javascript.com webenlace.com.ar
webcamdevochka.com webextract.profound.net
webcamtalk.net webkeyit.com
wholesalejerseys-cheapest.com webmasterhome.cn
withstandingheartwarming.com webnode.me
wma-x.com webshoppermac.com
writersgroup580.web.fc2.com website-analyzer.info
while.cheapwebsitehoster.com
wnoz.de
womens-journal.net
wowcasinoonline.ooo
writingservices17.blogspot.ru writingservices17.blogspot.ru
wttavern.com xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn----itbeirbjbi7bc6bh2d.xn--p1ai xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn--c1acygb.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xwatt.ru xn--80aikhbrhr.net
youandcredit.ru xn--e1afanlbnfckd7c3d.xn--p1ai
yougetsignal.com xn--h1ahbi.com.ua
youghbould.wordpress.com xnxx-n.com
youraticles.pl xnxxandxvideos.com
yourporn.com xpresscare.ru
youthreaders.com xtube.com
youtuhe.com xxxnatelefon.ru
yaoguangdj.com
yellowstonesafaritours.com
yoshkarola.zrus.org
youbrainboost.asia
yourads.website
youtubologia.it
yuarra.pluto.ro yuarra.pluto.ro
zaimite.ru yurgorod.ru
zajm-pod-zalog-nedvizhimosti.ru zakon-ob-obrazovanii.ru
zaloro.com zastroyka.org
zarabotok--doma.ru zeg-distribution.com
zero1.it zeroredirect6.com
zeroredirect.com zionstar.net
zigarettenonlinekaufen2.drupalgardens.com zoogdinsney.com
zoogdiesney.com
zvuker.net

470
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
108shot.com 12u.info
20pascals.nl 1adult.com
2girls1cup-free.com 5kstore.com
404.mn aaa38852e886ac4af1a3cff9b47cab6272.com
40cg.com ananumous.ru
55wmz.ru android4fun.org
activecampaign.dreamhosters.com animal-drawings.com
actulite.com animebox.com.ua
acunetix-referrer.com apibetweenlinesn-a.akamaihd.net
afmuseum.com
alert.scansafe.net
alibestsale.com
all4invest.info
allproblog.com
amoi.tn
apiitsthirteende-a.akamaihd.net apiitsthirteende-a.akamaihd.net
aquarium-pro.ru asiengirls.net
arate.ru avitocars.ru
autoloans.com aviva-limoux.com
autolombard-krasnodar.ru bad-stars.net
badavit.com.ua bausparen.bz.it
bagcionderlab.com bensbargains.net
balois.worldbreak.com
barbourjackets.online
bard-real.com.ua
bestcurs.org
betshuckclean.com betshuckclean.com
bget.ru billiard-classic.com.ua
bimatoprost-careprost.com bimatoprost-careprost.com
bingoporno.com bitcoin-ua.top
bio-optomarket.ru bjgugu.net.ua
bitcoins-live.ru blackwitchcraft.ru
bjetjt.com bluesman.nu
blacktwhite.com bobba.dzaba.com
blogos.kz boostmyppc.com
blogqpot.com brainboostingsupplements.org
bluesalt.co brandedleadgeneration.com
budilneg.xyz break-the-chains.com
buntube.net brk-rti.ru
burger-imperia.com brothers-smaller.ru
businesxxl.com buckspar.gq
buyfriend.ru buzzsumo.com
cejewelry.xyz cactussoft.biz
charmstroy.info cannibalcheerleader.com
check-this-out-now.online carmuffler.net
chlooe.com cdncash.org
classiquebijoux.ru cezartabac.ro
chatroulette.video
cheapbelstaff.online
chelnytruck.ru
ciekawinki.pl
citizenclsdriveri7.pen.io
clayvasedesigns.tk clayvasedesigns.tk
clickpapa.com
compliance-don.top
compliance-fred.xyz compliance-fred.xyz
compliance-jane.top compliance-john.top
credit-respect.ru conciergegroup.org
customerpromos-a.akamaihd.net consorzioilmosaico.org
dbmkfhqk.bloger.index.hr cool-wedding.net
digitalfaq.com counterbot.com
discountbarbour.online crazyprotocol.com
domaincrawler.com dame-ns.kz
domainroam.win dangphoto.trade
doska-vsem.ru discountliv.com
drivotracker.com diy-handmade-ideas.com
dustyorate.com dodlive.mil
dvr.biz.ua domcran.net
editmedios.com dominateforex.ml
ednorxmed.com domznaniy.ru
elektrozigarettekaufen2.blogse.nl doreenblog.online
eloxal.ru dstroy.su
escortplius.com efkt.jp
facecup.top electronicadirect.com
faithe.top elektrischezigarettekaufen2.cowblog.fr
fakehandbags.xyz elektrozigaretten2.yn.lt
fandlr.com elstal.com.pl
emailaccountlogin.co
eredijovon.com
european-torches.ru
extractorandburner.com
ezigarettekaufen1.hpage.com
ezigaretteshop2.mywapblog.com
fast-torrent.ru
fba-mexico.com fba-mexico.com
ff30236ddef1465f88547e760973d70a.com ferretsoft.com
flagcounter.me filesmonster.porn
forensicpsychiatry.ru film-one.ru
foto-weinberger.at filmfanatic.com
freewhatsappload.com financepoints.eu
funnel.co.za flavors.me
funnymama.com flex4launch.ru
fxund.us fordsonmajbor.cf
game-top.su free-share-buttons.top
gamingspark.com freewlan.info
gemara.com fuck-paid-share-buttons.xyz
glall.ru fungirlsgames.net
golaya.pw gamesprite.me
goodhumor24.com gay-site.store
greenshop.su globetrotting-culture.ru
grupografico-pilar.com.ar gotcher.us
guardlink.org greenidesign.co
guitar-master.org gsasearchengineranker.site
gz2.bbsoldes.fr hamacapty.com
hasshe.com hawaiielectriclight.com
hayate.biz hayate.biz
headpress.ru hdpixion.com
hotcore.info healbio.ru
ibb.com.ua health-medical-portal.info
import-sales.com hesteel.pl
infowarcraft.ru hobbyhuren-datenbank.com
internetproviderstucson.com homeart.space
homelygarden.com
hoztorg-opt.ru
hyhj.info
iframe-toloka.com
inclk.com
infodocsportal.com
intervsem.ru intervsem.ru
ip-guide.com intimshop-fantasy.ru
iphantom.com iplogger.org
iqs.biz.ua ipornox.xxx
iqupdatetmz.win iqoption.com
istripper.com je7.us
itrevolution.cf jewelryandfiligree.com
itservicesthatworkforyou.com jimmychoosale.online
izoll.ru job.icivil.ir
jolic2.com kerwinandcariza.com
keenoutlet.online keywordspay.com
keki.info kinozapas.com
kiskinhouse.com konteiner24.com
koronirealestate.gr koronirealestate.gr
kupiproday.com.ua kosmetyki.tm.pl
law-check-eight.xyz kostenlos-sexvideos.com
leboard.ru labelwater.se
lashstudia.ru
lignofix.ua
live-sexcam.tk
livefixer.com
lonfon.xyz
louboutinreplica.pw
lovasszovetseg.hu
lovelycraftyhome.com
lovi-moment.com.ua
lsp-awak-perikanan.com
lyngdalhudterapi.no lyngdalhudterapi.no
m1media.net m3gadownload.pl
makis.nu magazintiande.ru
malwareremovalcenter.com maxxtor.eu
manipulyator-peterburg.ru meds-online24.com
marcowebonyodziezowe.pl microsoftportal.net
maridan.com.ua mojpreskumanie.com
me-ke.com monclerjacketsoutlet.pw
megahdporno.net moscow.xrus.org
mil-stak.com moskva.nodup.ru
mirzonru.net mowser.com
mitrasound.ru mriyadh.com
mncrftpcs.com mtmtv.info
moneymaster.ru muznachas-service.ru
moneytop.ru mxgetcode.com
montazhnic.ru nailsimg.com
motherboard.vice.com net-profits.xyz
mrbitsandbytes.com nmrk.ru
mrcsa.com.au nucia.biz.ly
mrwhite.biz nvformula.ru
mturkcontent.com oakleyglassesonline.us
myra.top obnal.org
nastydollars.com offergroup.info
netpics.org offside2.5v.pl
notasprensa.info okuos.com
novosti-hi-tech.ru onblastblog.online
nuit-artisanale.com paceform.com
oil-td.ru parajumpersjakkesalgnorge.info
onescreen.cc parvezmia.xyz
openmultipleurl.com patol01.pw
optibuymac.com pensplan.com
paighambot.com pivka.xyz
pandroid.co pixgood.com
perso.wanadoo.es polimga.pw
pertlocogasilk.tk popads.net
photostudiolightings.com predominant-invent.tk
pistonclasico.com privatefx.all4invest.info
podshipniki-ntn.ru
pornmania.pl
porno-home365.com
pornoxxx.com.mx
printdirectforless.com
pro-tec.kz pro-tec.kz
producm.ru producm.ru
prostitutki-astana.org proprostatit.com
qpypcx.com puzo2arbuza.ru
qwarckoine.com r-control.ru
ranking2017.ga radiogambling.com
rankingchart.de rankexperience.com
ranksays.com ratemodels.net
regionshop.biz realnye-otzyvy.info
remont-mobile-phones.ru reimageplus.com
renhacklids.tk revistaindustria.com
replicalouboutin.xyz rewardpoll.com
retailwith.com rollercoin.com
rockprogblog.com rossmark.ru
roma-kukareku.livejournal.com sad-torg.com.ua
salmonfishingsacramentoriver.com saugatuck.com
samara.rosfirm.ru savefrom.com
sdelatmebel.ru sevendays.com.ua
searchimage.co sexystrippe.info
seeresultshub-a.akamaihd.net shemalegalls.blogporn.in
sei80.com shopvilleroyboch.com.ua
seo-smm.kz sibecoprom.ru
sex-watch.com sisiynas.ru
shop-electron.ru site-speed-checker.site
shopwme.ru sledstvie-veli.net
skylta.com small-games.biz
sladkoevideo.com snts.shell-pmr.ru
smailik.org soc-proof.su
snow.nvr163.com stanthonyscatholicchurch.org
social-buttons.com stefanbakosab.se
social-s-ggg.xyz stroimajor.ru
spidtest.org stroiminsk.com
spy-app.info suture.co
spymac.net
spywareit.com
star61.de
strag-invest.ru
streetfooduncovered.com
stroyhelp-dv.ru
students-cheapskate.ml
suralink.com
svbur.ru svbur.ru
symbaloo.com tackletarts.co
teenporn18.net taqywu51.soup.io
terrafootwear.us technopellet.gr
tizanidine4mgstreetprice.blogspot.com tedy.su
tizanidinehcl2mgtablet.blogspot.com telesvoboda.ru
tizanidinehydrochloride2mg.blogspot.com tizanidineandhydrocodone.blogspot.com
tizanidineingredients.blogspot.com
todohr.com todohr.com
topcar-krasnodar.ru top250movies.ru
tracfone.com totu.info
track112.site trafficjunky.com
traffic2money.com trafficstars.com
trafficnetzwerk.de traffpartners.com
truemfilelj.gq tripper.de
tsstcorpcddvdwshbbdriverfb.aircus.com tuckpointingmasonrysystems.com
turizm.bz twelvevisionspartyofcolorado.com
twittruth.com tytoona.com
ublaze.ru uhod-za-sobakoj.ru
uggsale.online uptime-gamma.net
uk-zheu20.ru usacasino.com
ukrtvory.in.ua utiblog.fr
valsalud.com vacances-voyages.info
vegan-foods.us
vellings.info
villacoloniale.com villacoloniale.com
vip2ch.com voprosotvet24.ru
virus-respirators.com warezaccess.com
vkontarkte.com webads.co.nz
voloo.ru webcamtalk.net
wapsite.me weblo.com
web-analytics.date website-speed-up.top
weblibrary.win wemarketing.se
weekes.biz.tc worldinternetauthority.com
whatzmyip.net worldlovers.ru
wnoz.de wyniki-lista.pl
worldmusicfests.com xfluro.com
ww1943.ru xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
www.souvenirua.com xn--b1adccaf1bzj.xn--p1ai
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai y8games-free.com
xn--e1afanlbnfckd7c3d.xn--p1ai zarabiaj-dzis.pl
xn--h1aakne2ba.xn--p1ai zarajbuilders.com
xtubeporno.net zawyna.ua
xwatt.ru zebradudka.com
yatrk.xyz zeroredirect9.com
youresponsive.com zhorapankratov7.blogspot.com
yousense.info znaturaloriginal.com
yurgorod.ru zootoplist.com
zacreditom.ru
zaim-pod-zalog-krasnodar.ru
zigzog.ru
zootravel.com

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.9 0; 10.1.1.62 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2870 ### Version: V4.2022.01.2871
### Updated: Sat Jan 22 10:05:03 UTC 2022 ### Updated: Sat Jan 22 22:01:14 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2870 ### Version: V4.2022.01.2871
### Updated: Sat Jan 22 10:05:03 UTC 2022 ### Updated: Sat Jan 22 22:01:14 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##