vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.12.3518 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-12-15 10:10:05 +00:00
parent 3dbe83214b
commit 3a2eec4ff4
47 changed files with 101823 additions and 101833 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3517 #### Version: V4.2022.12.3518
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3517 #### Version: V4.2022.12.3518
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.12.3517 #### Version: V4.2022.12.3518
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12594
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.12.3517 #### Version: V4.2022.12.3518
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12598
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12598
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12598
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12598
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 1; 10.1.0.43 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 0; 10.1.0.43 0;
127.0.0.1 0; 127.0.0.1 0;

12598
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 1; 10.1.0.43 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 0; 10.1.0.43 0;
127.0.0.1 0; 127.0.0.1 0;

12598
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 1; 10.1.0.43 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 0; 10.1.0.43 0;
127.0.0.1 0; 127.0.0.1 0;

12598
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3517 3518

12598
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12598
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.248.43 0; 52.226.126.48 0;

12598
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 1; 10.1.0.43 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 0; 10.1.0.43 0;
127.0.0.1 0; 127.0.0.1 0;

12598
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 1; 10.1.0.43 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 0; 10.1.0.43 0;
127.0.0.1 0; 127.0.0.1 0;

12598
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 1; 10.1.0.43 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 0; 10.1.0.43 0;
127.0.0.1 0; 127.0.0.1 0;

12598
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.248.43 0; 52.226.126.48 0;

12598
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.248.43 0; 52.226.126.48 0;

12598
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 1; 10.1.0.43 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Cosmos DigitalPebble
GetWeb Getintent
Mojeek Leap
Searchestate Magnet
SemrushBot Nameprotect
Spbot SEOkicks-Robot
Titan SEOstats
WWW::Mechanize Seomoz
Whacker Siphon
crawler.feedback Sitevigil

294
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404checker 404checker
404enemy
80legs 80legs
ADmantX
ALittle Client ALittle Client
Acunetix ASPSeek
Aipbot Aboundexbot
AllSubmitter AfD-Verbotsverfahren
AhrefsBot
AiHitBot
Alexibot
Alligator Alligator
Anarchy AlphaBot
Ankit Anthill
Apexoo Apexoo
Aspiegel Asterias
Attach
AwarioSmartBot
BDCbot BDCbot
BDFetch Backlink-Ceck
BackDoorBot
BackStreet
BackWeb
BacklinkCrawler BacklinkCrawler
Badass Badass
Bandit Bandit
Barkrowler
BatchFTP BatchFTP
BetaBot
Bigfoot Bigfoot
BlackWidow Blackboard
Boardreader Blow
Brandprotect Bolt
Brandwatch
Buck Buck
Buddy BuiltBotTough
BuiltWith BunnySlippers
CCBot CATExplorador
CSHttp CODE87
CensysInspect CherryPicker
ChinaClaw Chlooe
Cliqzbot
Cocolyzebot Cocolyzebot
Cogentbot Cogentbot
Collector
CopyRightCheck CopyRightCheck
Cosmos
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
CrunchBot
Curious
Custo
CyotekWebCopy
DBLBot DBLBot
DSearch DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli Devil
Deusu Digincore
Discoverybot DigitalPebble
Disco
DnyzBot DnyzBot
DomainAppender
DomainCrawler DomainCrawler
DomainStatsBot Domains Project
Dotbot Drip
Download Wonder
EasyDL
Ebingbong Ebingbong
EirGrabber
EroCrawler EroCrawler
Evil
ExtLinksBot ExtLinksBot
Extractor ExtractorPro
Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms FHscan
FDM FemtosearchBot
Foobot Firefox/7.0
Freeuploader Freeuploader
Fuzz Fuzz
G-i-g-a-b-o-t FyberSpider
GT::WWW Genieo
GetRight
GetWeb GetWeb
Getintent Getintent
Go!Zilla Gigabot
GoZilla GoZilla
GrabNet Grabber
GrapeFX Grafula
HEADMasterSEO GridBot
HTMLparser HTTP::Lite
Haansoft
Harvest Harvest
Hloader Havij
Humanlinks HonoluluBot
HybridBot IRLbot
IndeedBot Iblog
IlseBot
Image Fetch
InfoNaviRobot
InfoTekies InfoTekies
InterGET Internet Ninja
Iria
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
Jetty JikeSpider
JustView Joomla
Kenjin Spider Kenjin Spider
Kozmosbot Keyword Density
LWP::Simple LNSpiderguy
Lanshanbot Lanshanbot
Larbin Larbin
Leap
LeechGet LeechGet
LieBaoFast LibWeb
LinkextractorPro Libwhisker
Lightspeedsystems
LinkpadBot LinkpadBot
LinksManager LinqiaScrapeBot
Lipperhey
Lmspider Lmspider
MFC_Tear_Sample MFC_Tear_Sample
MJ12bot MSFrontPage
MTRobot Mag-Net
Mail.RU_Bot
Majestic SEO Majestic SEO
MarkMonitor MarkMonitor
Mass Downloader Mass Downloader
Masscan Masscan
Mata Hari Mata Hari
Mediatoolkitbot MeanPath Bot
MicroMessenger MegaIndex.ru
Microsoft URL Control Microsoft Data Access
Mister PiX
Moblie Safari
Mojeek
Mojolicious Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Mr.4x3
Musobot Musobot
Nessus NPbot
Net Vampire Nameprotect
NetLyzer Navroad
NetMechanic NetMechanic
NetZIP NetZIP
Netcraft Netvibes
Nettrack Nibbler
Nimbostratus Niki-bot
Nikto
NimbleCrawler
Ninja Ninja
Not Octopus
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
Openvas Openvas
OrangeBot OrangeSpider
PageThing.com OutfoxBot
PHPCrawl
PageGrabber
Panscient Panscient
Papa Foto Papa Foto
Pavuk Pavuk
Petalbot PeoplePal
Picscout Pi-Monster
Picsearch Picsearch
PictureFinder PictureFinder
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey Pockey
Psbot ProPowerBot
ProWebWalker
Pu_iN
Pump Pump
PyCurl QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot
RankingBot2 RankingBot2
Rankivabot
ReGet
RealDownload
Reaper Reaper
RebelMouse
Recorder Recorder
Ripper
RocketCrawler
SEOkicks SEOkicks
SEOkicks-Robot SEOkicks-Robot
SMTBot SMTBot
ScanAlert Scanbot
Scrapy Seekport
ScreenerBot Semrush
ScrepyBot
SentiBot SentiBot
SiteSucker SeoSiteCheckup
Site Sucker
SiteLockSpider
Sitebeam Sitebeam
Siteimprove Siteimprove
Snake SmartDownload
Snapbot
SocialRankIOBot SocialRankIOBot
Sosospider Sosospider
Spammen Spammen
SpankBot
SputnikBot SputnikBot
Sqlmap
Sqlworm Sqlworm
Stripper Sqworm
Surfbot Steeler
SurveyBot Sucuri
Szukacz SuperBot
T8Abot SuperHTTP
Suzuran
T0PHackTeam
Telesoft Telesoft
Telesphoreo
Telesphorep Telesphorep
TheNomad
Titan Titan
Toata Toweyabot
Trendiction Trendiction
Turnitin Turingos
TurnitinBot TurnitinBot
Typhoeus
URLy.Warning URLy.Warning
VelenPublicWebCrawler VB Project
VidibleScraper Vacuum
WEBDAV Vagabondo
WWW-Mechanize Virusdie
Voltron
WPScan
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE Web Sauger
Web Collage
Web Fetch
Web Sucker
WebBandit WebBandit
WebCopier WebSucker
WebGo IS Webshag
WebLeacher
WebPix
WebSauger
WebZIP
Webalta
Website Quester
WebsiteQuester WebsiteQuester
Webster Whack
Who.is Bot Whacker
Wotbox Woobot
Wprecon Wprecon
Xaldon_WebSpider Xaldon WebSpider
YoudaoBot
Zade Zade
Zauba Zauba
Zermelo
Zitebot
ZmEu
ZoomBot
ZoominfoBot ZoominfoBot
adscanner archive.org_bot
cah.io.community backlink-check
cognitiveseo coccocbot
com.plumanalytics
dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org evc-batch
instabid facebookscraper
gopher
heritrix
internetVista monitor internetVista monitor
iubenda-radar
lwp-request
magpie-crawler
oBot oBot
page scorer scan.lol
pcBrowser
scalaj-http
seobility seobility
seocompany.store seoscanners
seostar
siteripz
sogouspider sogouspider
sp_auditbot sysscan
spyfu tAkeOut
trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com webpros.com
webprosbot
xpymep1.exe

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker 404enemy
80legs ADmantX
ALittle Client
ASPSeek ASPSeek
Aboundex Aboundex
Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Alligator AiHitBot
AlphaBot Alexibot
Anarchy99 Anarchie
Ankit Ankit
Aspiegel Aspiegel
Asterias
Atomseobot
BBBike
BDFetch
BLEXBot
BackDoorBot BackDoorBot
BackStreet
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Bandit Bandit
BatchFTP BetaBot
Battleztar Bazinga Bigfoot
Bitacle Black Hole
BlackWidow BlackWidow
Blackboard Blackboard
Blow Blow
Bolt Bolt
BotALot Buck
Brandwatch
Bullseye Bullseye
BunnySlippers BunnySlippers
Calculon CODE87
CazoodleBot
Cegbfeieh
CensysInspect
CherryPicker
ChinaClaw ChinaClaw
Citoid
Claritybot
Cliqzbot Cliqzbot
Cloud mapping
Cocolyzebot
Collector Collector
Copier
Copyscape Copyscape
Cosmos Cosmos
Crawling at Home Project
Crescent
CrunchBot
Curious Curious
Custo Custo
DBLBot DBLBot
DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Deusu Demon
DigitalPebble DigitalPebble
Disco Dirbuster
Dispatch DittoSpyder
DnyzBot DnBCrawler-Analytics
DomCopBot DomainStatsBot
Domains Project EirGrabber
Dotbot ExtLinksBot
Download Wonder Extractor
Drip EyeNetIE
ECCP/1.0
Ebingbong
Ecxi
Express WebPictures
Ezooms Ezooms
FemtosearchBot FHscan
Firefox/7.0 Firefox/7.0
FlashGet Fyrebot
GetRight GalaxyBot
Gotit Genieo
GetWeb
Go!Zilla
Grabber
GrapeFX GrapeFX
GrapeshotCrawler HEADMasterSEO
GridBot HTMLparser
HTTP::Lite Haansoft
HTTrack IRLbot
HaosouSpider IlseBot
IDBTE4M
IDBot
Image Fetch
Image Sucker
IndeedBot IndeedBot
InterGET InfoTekies
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz JetCar
JennyBot JikeSpider
Jetty Joomla
Jorgee
JustView JustView
Kenjin Spider
Kozmosbot Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple LWP::Simple
Lanshanbot Larbin
Leap LibWeb
LeechGet LieBaoFast
Lightspeedsystems
Likse Likse
LinkWalker LinkWalker
Linkbot LinkpadBot
LinksManager
LinqiaRSSBot
Lipperhey Spider Lipperhey Spider
Litemage_walker MFC_Tear_Sample
Lmspider MIDown tool
Ltx71 MIIxpc
MJ12bot MSFrontPage
MSIECrawler MSIECrawler
MTRobot
Magnet
Mail.RU_Bot
Majestic SEO Majestic SEO
MauiBot MarkWatch
MeanPath Bot Mb2345Browser
Mediatoolkitbot MegaIndex.ru
MicroMessenger Minefield
Microsoft Data Access
Microsoft URL Control
Mister PiX Mister PiX
Moblie Safari Mojeek
Mozlila Mojolicious
Morfeus Fucking Scanner
Msrabot
Musobot Musobot
NICErsPRO
NPbot NPbot
Navroad Navroad
Needle Net Vampire
NetZIP NetAnts
Nettrack NetMechanic
NextGenSearchBot NetSpider
NimbleCrawler Netcraft
Nimbostratus
Ninja Ninja
Nmap Nutch
Not Octopus
Offline Explorer Offline Explorer
Offline Navigator Offline Navigator
OutclicksBot OrangeBot
OutfoxBot OutfoxBot
PECL::HTTP
PHPCrawl PHPCrawl
Page Analyzer
PageThing.com
Pandalytics Pandalytics
Panscient Panscient
Papa Foto Papa Foto
Pavuk Pavuk
PeoplePal Pi-Monster
Picsearch Picsearch
PictureFinder PictureFinder
Piepmatz Pixray
Pu_iN Pockey
Probethenet
Pump Pump
PxBroker
PyCurl
Quick-Crawler Quick-Crawler
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot2 RankingBot
Rankivabot RealDownload
ReGet Reaper
RebelMouse
Recorder
RedesScrapy
RepoMonkey RepoMonkey
Ripper
Rogerbot Rogerbot
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SISTRIX SEOstats
SMTBot ScanAlert
Screaming Scanbot
ScreenerBot ScreenerBot
ScrepyBot ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
SeoSiteCheckup Seekport
SeobilityBot SentiBot
Seomoz Shodan
Siphon Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteSnagger SiteSnagger
Sitebeam SiteSucker
Sociscraper Siteimprove
SlySearch
Snake
Snapbot
Sogou web spider Sogou web spider
Sosospider SpaceBison
Sottopop SpankBot
Spanner
Spbot Spbot
Spinn3r
Sqlmap Sqlmap
Sucuri Sqworm
SuperHTTP SuperBot
Szukacz Swiftbot
Teleport Teleport
TeleportPro TeleportPro
Thumbor Telesoft
TightTwatBot TightTwatBot
Tracemyfile Tracemyfile
Trendictionbot True_Robot
Turnitin Turingos
Twice TurnitinBot
Typhoeus
URLy Warning
V-BOT V-BOT
VB Project VB Project
VidibleScraper VCI
VoidEYE Vacuum
Virusdie
Voil Voil
WBSearchBot Voltron
WEBDAV WISENutbot
WWW-Mechanize WWW-Mechanize
Wallpapers/3.0 WallpapersHD
Web Auto Web Auto
Web Collage Web Collage
Web Enhancer Web Fuck
Web Sucker Web Sucker
WebAuto
WebCopier WebCopier
WebFetch WebEnhancer
WebFuck WebFuck
WebGo IS
WebReaper WebReaper
WebStripper WebWhacker
Webalta Webshag
WebmasterWorldForumBot Website Quester
Whack Whack
Whacker
Whatweb Whatweb
Who.is Bot Who.is Bot
WinHTTrack
Wonderbot Wonderbot
Woobot
Wotbox Wotbox
Wprecon Wprecon
Xaldon_WebSpider
Xenu
YoudaoBot YoudaoBot
Zeus Zade
Zauba
Zermelo
Zitebot
ZmEu
ZoomBot ZoomBot
adscanner ZoominfoBot
archive.org_bot
autoemailspider
backlink-check backlink-check
cah.io.community cah.io.community
check1.exe
clark-crawler
crawler4j
demandbase-bot demandbase-bot
evc-batch domainsproject.org
gopher facebookscraper
iubenda-radar heritrix
lwp-trivial lwp-trivial
magpie-crawler magpie-crawler
mediawords oBot
page scorer pcBrowser
probe-image-size polaris version
ripz satoristudio.net
scalaj-http
scan.lol scan.lol
seoscanners seobility
sp_auditbot seocompany.store
spyfu sitechecker.pro
trendiction.com siteripz
sysscan
tAkeOut
trendiction.de trendiction.de
voyagerx.com voyagerx.com
webgains-bot webgains-bot
webpros.com webmeup-crawler
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bestprofits-there.com 1xbet4you.com
brandimensions.com balkanfarma.ru
candypeople.se deplim.com
designdevise.com heygidday.biz
immobiliaremassaro.com krasnodar.ru
jav-idol.com mastershef.club
naughtyconnect.com mountainstream.ms
novgorod.xrus.org nippon-bearings.ru
russia-tao.ru percin.biz.ly
wpthemedetector.co.uk takeflyte.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
256bit.by 03e.info
7a2rnuey1tw9ar.ru 11pikachu.ru
8jn.dating-app.ru 13tabs.com
91abcw.com 1qingdao.com
a14download.com 2020iscoming.info
aa625d84f1587749c1ab011d6f269f7d64.com 3-letter-domains.net
aac52006ec82a24e08b665f4db2b5013f7.com 4webmasters.org
abcdefh.xyz 66cpwgln.space
acmebtn.ml 7minuteworkout.com
addbin.men 991mostfm.co.id
adhome.biz aarbur.com
ads-cool.pro acunetix-referrer.com
ads.gold addray.pro
adssafeprotected.com adexprts.com
ads.tremorhub.com
adservme.com
adult3dgames.com adult3dgames.com
adultwebhosting.info adviceforum.info
advancedcleaner.com akvamaster.dp.ua
advokat-grodno.by alice.it
airlandsea.info allbizne.co.ua
alarmobninsk.ru alldownload.pw
alert-jdh.xyz amoi.tn
alf-img.com anaksma.info
alibestsale.com anniemation.com
all4invest.info apimossnetinfo-a.akamaihd.net
alyeskaresort.com apimountainbikei-a.akamaihd.net
amazon-seo-service.com aramaicmedia.org
amospalla.es arquivo.pt
anal-acrobats.hol.es artdeko.info
angkortours.vn auto-zapchasti.org
animenime.ru auto.rusvile.lt
aosexkontakte.net
apartment.ru
apibetweenlinesn-a.akamaihd.net
apibrowsesmartne-a.akamaihd.net
apilinkswiftco-a.akamaihd.net
apiqualitinknet-a.akamaihd.net
appfixing.space
architecturebest.com
astrochicks.com
atdedinotuho.tk
atleticpharm.org
atmagroup.ru
auto4style.ru
autobrennero.it autobrennero.it
autovideobroadcast.com autodo.info
aviav.ru.com avto-oligarh.ru
ayanaspa.com avtolombard-krasnodar.com
azarttoto.com axbocz.net
babyguns.ru azbukadiets.ru
baikaleminer.com azlex.uz
backlinkwatch.com
balans.shahterworld.org balans.shahterworld.org
balkanfarma.org bang-hotties.com
bambi.ck.ua bank.uz
barbourjackets.online best-seo-offer.com
barnfurnituremart.com bestbrainprod.win
belinvestdom.by bestcoin.cc
bestchoice.cf bestcurs.org
bezcmexa.ru bestdraws.com
bimatoprost-careprost.com.ua bidbuy.co.kr
bingo8888.com bigames.online
binomo.com bitcoinremote.com
biographiya.com bjetjt.com
bioscorp.ru blackplanet.com
bizzliving.com bloggers.nl
blogrankers.com bnt-team.com
blogs.rediff.com boanonihaca.tk
boltalko.xyz bolegapakistan.com
braindaily.xyz bolezniorganov.ru
breastaugmentation.co.za bolitgorlo.net
brg8.com bombla.org
brothers-smaller.ru bookmark4you.com.biz
buqayy0.livejournal.com brianhenry.co
buron.pw buybest1.biz
businesxxl.com buyviagraa.blogspot.com
buypuppies.ca callawaygolfoutlet.online
bvlgariring.xyz candycrushshop.com
cah.io.community candytiner.com
call-of-duty.info capsderfudd.tk
callstevens.com
cand.jp
candy-glam-hp.com
canyougethighofftizanidine.blogspot.com
cartierloveringreplica.xyz
cartierlovestore.xyz cartierlovestore.xyz
cartierwatch.xyz
casinopinup-wins.com
casite-513049.cloudaccess.net casite-513049.cloudaccess.net
celejihad.info chastnoeporno.com
chatroulette.world cheapbelstaff.online
cheapestjerseys-wholesale.com cherrypointplace.ca
cheapjerseysa.com choice-credit.ru
chelnytruck.ru chorus.terakeet.com
chinacheapelitejerseys.com civilwartheater.com
christianlouboutinreplica.pw cleanallspyware.com
cialischstgerts.com
cienum.fr
click2pawn.com
clickbank.net clickbank.net
clicksor.net clothesforcash.com
coaching-netz.info clubfashionista.com
codq.info cmd.kz
coindirect.io
colehaanoutlet.store colehaanoutlet.store
compliance-don.top compliance-alexa.top
compliance-jess.top compliance-andrew.top
compliance-olga.top consorzioilmosaico.org
contentlook.co contentsexpress.com
cookie-law-enforcement-hh.xyz
coop-gamers.ru
copyrightclaims.org
corta.co
cosmeticswomens-womensports.rhcloud.com cosmeticswomens-womensports.rhcloud.com
crest3d.ru costablue.xyz
cubs-jerseys.us cougfan.info
customchocolate.business-for-home.com counter.bloke.com
customsua.com.ua counterbot.com
cowlmash.com
crackzplanet.com
crynet.cc
cupday.com
cymbaltaandtizanidine.blogspot.com cymbaltaandtizanidine.blogspot.com
day-news.info dailyrank.net
dcj-nn.ru datract.com
ddsoldes.fr dayibiao.com
demon-tweeks.com dealwifi.com
design-ideas.info delayreferat.ru
diarioaconcagua.com demian.kz
digest-project.ru detskie-zabavi.ru
dimfour.com dichvuvesinhhanoi.com
dogoimage.com discountbarbour.online
domaincrawler.com dktr.ru
dolohen.com
domain-tracker.com
domainsatcost.com domainsatcost.com
drillsaw.ru domcran.net
dtnlyss.com donna7753191.ru
duplicashapp.com doreenblog.online
e-collantes.com download-of-the-warez.blogspot.com
eaplay.ru draniki.org
earthmagic.info dverimegapolis.ru
ecommerce-seo.com dvr.biz.ua
ee77ee.com dwomlink.info
ekaterinburg.xrus.org elektrischezigarette2.onsugar.com
ekto.ee
elektrischezigarette1.onsugar.com
elektrischezigaretten2.beeplog.com elektrischezigaretten2.beeplog.com
elektronischezigarettekaufen1.beeplog.com elektroniksigaraankara.info
elektrozigarettekaufen2.blogse.nl elektrozigaretten1.postbit.com
elmifarhangi.com elkacentr.ru
escortslet.net emediate.eu
essay-edu.biz erot.co
etur.ru event-tracking.com
eupornstar.info extstat.com
everytalk.tv f0815.de
ewebarticle.info facebook-mobile.xyz
exoclick.com famix.xyz
extractorandburner.com faptitans.com
ezaz.info fastfixing.tech
ezigarettenshop2.postbit.com favornews.com
faithe.top fba-mexico.com
fbdownloader.com feargames.ru
fenoyl.batcave.net fellowshipoftheminds.com
fetishinside.com fertilitetsradgivningen.se
fickkontakte.org
fitness-video.net
flashahead.info
floridamobilebillboards.com
flowwwers.com flowwwers.com
flytourisme.org footbalive.org
foodgid.net forex-indextop20.ru
formularz-konkurs.tk foto-weinberger.at
forpackningsutveckling.se fotosfotos.eu
forum.tvmir.org frameimage.org
forum69.info frankofficial.ru
forzeronly.com free-deals.faith
fototravel.eu free-traffic.xyz
foxydeal.com freegamesplay.online
freenom.link freelinkbuilding.website.tk
front.to froggytube.com
fuel-gas.com fxgallery.com
fun-mobi.pl galeria-zdjec.com
fusoradio.info gameonasia.com
fxtips.ru gameplexcity.com
g.starmoe.xyz gamingspark.com
galeon.com gateway.zscalertwo.net
gambarkatabaru.com gay-file.com
gamezblox.com gazobeton-p.com.ua
gay-site.store genericviagrasildenafiled.net
gaypornmovie.net get-free-social-traffic.com
gaytube.com
gearsadspromo.club
germetiki.com.ua
getaclueamerica.com getaclueamerica.com
getlamborghini.ga gilbertbanda.net
getmiro.com glasof.es
ghernnqr.skyrock.com goldadpremium.com
girlsatgames.ru golfresa.lucania.se
gktt.ru gomusix.com
glass-msk.ru gopro-online.info
google-liar.ru goroda-vsego-mira.ru
googlepositions.com gosarhivrt.ru
gq-catalog.gq gpirate.com
graphics8.info gsasearchengineranker.space
graphid.com gyffu.com
greatzip.com handicapvantoday.com
growboxbl.ru
gta-club.ru
guge.io
hackers-crackers.tk
handicapvansarea.com
hasfun.com
hasshe.com hasshe.com
hdseriale.pl havepussy.com
hdwalls.xyz highspeed5.net
heartofbeijing.blogspot.com hobbyhuren24.net
hesteel.pl hobild.net
hobbyhuren-datenbank.com holidaypics.org
hollywoodactress.info hollywoodactress.info
homedecoguide.info
hotblognetwork.com hotblognetwork.com
hotsocialz.com hoverboard360.de
hoverboard360.es husky-shop.cz
hoverboard360.se hystersister.com
howmuchdoestizanidinecost.blogspot.com i-hobot.ru
hunthillfarmtrust.org iideaidekonkatu.info
hybrid.ru ilovevitaly.com
hydropump.su imgata.com
hyhj.info immigrational.info
igithab.com imperiafilm.ru
ilmen.net impotentik.com
imperia31.ru in-tandem.co
interferencer.ru inboundlinks.win
internet-apteka.ru incanto.in.ua
invitefashion.com increasewwwtraffic.info
iron-age.info inmate-locator.us
ispaniya-costa-blanca.ru int.search.mywebsearch.com
istizanidinelikexanax.blogspot.com iomoio.net
istripper.com ispac.org
jav-idol.com itag.pw
je7.us iwanttodeliver.com
kakadu-interior.com.ua izhevsk.zrus.org
janavibekken.no
javspace.net
jerseywholesaleelitestore.com
jolic2.com
jose.mulinohouse.co
kaidalibor.de
kalandranis.gr
kamorel.com
kartiniresto.com kartiniresto.com
kashubadesign.ru ketrzyn.pl
kellyonline.xyz keywesthideaways.co
keyword-suggestions.com kimcurlrvsms.com
keywordhut.com kino-key.info
keywordsdoctor.com kollesa.ru
keywordspay.com komp-pomosch.ru
kinotorka.ru konturkrasoty.ru
kmgamex.cf koronirealestate.gr
konoplisemena.com krasnodar-avtolombards.ru
kosmetyki.tm.pl kreativperlen.ch
krasivoe-hd.com kustanay.kz
krynica.info larger.io
kutikomi.net layola.biz.tc
la-fa.ru lightinghomes.net
lacasamorett.com linerdrilling.com
lacave.ntic.fr littlesexdolls.com
languagecode.com liupis.com
laparfumotec.com live-cam6.info
lapolis.it locustdesign.co
laptoper.net lookingglassemb.com
law-check-seven.xyz
leftofcentrist.com
lesbian.xyz
lezbiyanki.net
life-instyle.com
likesdesign.com
linkbuddies.com
lion.misba.us
lion.snapmap.us
lirunet.ru
littleberry.ru
liver-chintai.org
livesex-amateure.info
lmrauction.com
lolnex.us
lonerangergames.com
luchshie-topcasino.ru
lunchrock.co lunchrock.co
lyrics.home-task.com m4ever.net
macnewtech.com mafa-free.com
madot.onlinewebshop.net magicdiet.gq
maladot.com magicplayer-s.acestream.net
manve.info magnetpress.sk
manyresultshub-a.akamaihd.net mainlinehobby.net
marketingtechniques.info manipulyator-peterburg.ru
mapquestz.us
masthopehomes.com
matchpal-a.akamaihd.net matchpal-a.akamaihd.net
mebel-alait.ru matrixalchemy.com
medicines-choice.com medkletki.ru
metalonly.info meget.co.za
mini-modus.ru menhealed.net
mistr-x.org mexicotravelnet.com
modnie-futbolki.net mhtr.be
moinozhki.com michaelkorsoutlet.store
moje-recenze.cz michellblog.online
monclercheap.xyz middlerush-a.akamaihd.net
monsterdivx.com milblueprint.com
moscow-region.ru mindeyegames.com
moxo.com minecraft-neo.ru
moyakuhnia.ru minecraft-rus.org
msk-diplomat.com miniads.ca
mtmtv.info mirzonru.net
musirc.com missclub.info
my-bc.ru mlf.hordo.win
myiptest.com mmgq.ru
mypornfree.ru mmofreegames.online
mysexpics.ru mockupui.com
mojpregled.com
mole.pluto.ro
mp3films.ru
muzaporn.com
mydownloadengine.com
mygameplus.ru
myhitmp3.club
myonlinepayday.co
mywallpaper.top
naj-filmy24.pl naj-filmy24.pl
narkologiya-peterburg.ru narco24.me
national-today-winning-winner.club naturalpharm.com.ua
net-radar.com naval.jislaaik.com
netcheckcdn.xyz neodownload.webcam
netvouz.com netallergy.ru
ngps1.ru nibbler.silktide.com
nicovideo.jp notasprensa.info
novodigs.com novodigs.com
o333o.com nzfilecloud.weebly.com
o-o-8-o-o.ru
oakleyglassesonline.us
ochistka-stokov.ru
olvanto.ru
omgtnc.com omgtnc.com
one.net.in onblastblog.online
onetravelguides.com online-sbank.ru
online7777.com
onlinemegax.com onlinemegax.com
ordernorxx.com onlinetvseries.me
ossmalta.com optom-deshevo.ru
ourtherapy.ru panchro.xyz
oynat.info paperwritingservice17.blogspot.ru
palocco.it
palvira.com.ua
panicatack.com
paparazzistudios.com.au
parajumpersoutlet.online parajumpersoutlet.online
partner-trustworthy.men parfusale.se
pawli.eu partner-high.men
pay2me.pl
pc-test.net
pechikamini.ru pechikamini.ru
pekori.to pelfind.me
pendelprognos.se pensplan4u.com
peugeot-club.org perfectpracticeweb.com
physfunc.ru perl.dp.ua
picquery.com pewit.pw
pinapchik.com pic2fly.com
playfortuna-play.ru picswe.com
polybuild.ru picturesfrom.com
popup.matchmaker.com pinsdaddy.com
pornmania.pl pinwallpaper.xyz
porno-home365.com piski.top
porno-raskazy.ru pl-vouchers.com
pornoklad.ru platesauto.com
pornowarp.info platezhka.net
portal-eu.ru ploenjitmedia.azurewebsites.net
pr0fit-b0x.com popup-jdh.xyz
pravoholding.ru poquoson.org
pornhive.org
pornogratisdiario.com
pornokorol.com
pornosok.ru
potoideas.us
predominant-invent.tk predominant-invent.tk
preparevideosafesystem4unow.site
pretty-mart.com
primedice.com primedice.com
proekt-gaz.ru printie.com
projectforte.ru privatamateure.com
promgirldresses.xyz prod2016.com
prospekt-st.ru profitfx.online
prosperent.com prom23.ru
prostitutki-rostova.org prostitutki-astana.org
psvita.ru pulse33.ru
ptr.ruvds.com qcstrtvt.bloger.index.hr
pulseonclick.com quick-offer.com
quebec-bin.com quickloanbank.com
queerspace.com rada.ru
quit-smoking.ga rangoman.date
ragecash.com rankingchart.de
rangjued.com rankscanner.com
ranksignals.com ranksonic.info
reklama1.ru refads.pro
reklamuss.ru rialp.getenjoyment.net
renecaovilla.online rniaeba.ga
renhacklids.tk
rentalcarnavi.info
repeatlogo.co.uk
replica-watch.ru
resant.ru
respublica-otel.ru
riders.ro
robertefuller.com robertefuller.com
rocketchange.ru rockprogblog.com
royal-investments.net
rp9.ru
runstocks.com
rus-teh.narod.ru rus-teh.narod.ru
rybalka-opt.ru sabizonline.com
sale-japan.com sc-specialhost.com
samo-soznanie.ru
sampleletters.net
sanyuprojects.com
sarahmilne.top
sardinie.us
scanner-julia.top scanner-julia.top
sdelai-prosto.ru sdi-pme.com
seadragonherbery.com sdsjweb.com
search.1and1.com search-goo.com
searchinquire.com search.xtconnect.com
seksvideoonlain.com searchinterneat-a.akamaihd.net
semprofile.com searchtooknow-a.akamaihd.net
seoboxes.com seasaltwithfood.com
seorank.info selectads.men
serptehnika.ru seofirmreviewsus.info
service-core.ru seogadget.ru
serviporno.com servisural.ru
sex-watch.com sevendays.com.ua
sexspornotub.com sexblog.pw
sexuria.net sexphoto.site
sexy-pings.com sexy.babes.frontend-stack.top
sexytrend.ru sexyteens.hol.es
sfd-chess.ru
shinikiev.com.ua
ship-marvel.co.ua
shopcheermakeup.info shopcheermakeup.info
shtaketniki.ru sibvitr.ru
si-unique.com sim-service.net
sierraapps.com siteexpress.co.il
siha.de sk.golden-praga.ru
sindragosa.comxa.com skanninge.se
site-speed-check.site skuteczna-dieta.co.pl
skytraf.xyz skylta.com
sledstvie-veli.net smartpet.ru
slutloadlive.com social-vestnik.ru
socialmediasuggest.com sogimlecal.tk
socialmonkee.com solitaire-game.ru
sockshares.tv soundfrost.org
softlinesolutions.me souvenir.cc
soietvousmaime.fr soviet-portal.do.am
sovetogorod.ru spanking.to
spasswelt.net spb.afora.ru
spywareit.com sprttrack.com
stairliftsarea.com spy-app.info
stairliftstrue.com squidoo.com
stal-rulon.ru sr-rekneskap.no
standardchartered-forex.com
steamoff.net
steebook.com
stefanbakosab.se stefanbakosab.se
streha-metalko.si stopnarco.ru
strongsignal-a.akamaihd.net strag-invest.ru
sukirgenk.dvrlists.com studworks.org
sunflowerdrawingpaintings.blogspot.com superinterstitial.com
swingerseiten.com surcentro.com
swinginwithme.ru svetlotorg.ru
szamponrevita.pl sweepstakes.rewardit.com
tacbelarus.ru swinger-mobil.net
takeprofitsystem.com swsociety.se
telsis.com sygraem.com
tengohydar.tk tastyfoodideas.com
theplacetoupdating.pw technika-remont.ru
thetardistimes.ovh the-torrent-tracker.blogspot.com
thedownloadfromwarez.blogspot.com
thegioixekhach.com
theheroes.ru
thetoiletpaper.com
thiegs.reco.ws
tiens2010.ru
tilido.com
tizanidine4mgtablets.blogspot.com
tizanidinemusclerelaxant.blogspot.com tizanidinemusclerelaxant.blogspot.com
tizanidinepain.blogspot.com tizanidinenarcotic.blogspot.com
tizanidineorflexeril.blogspot.com
tizanidinesleepaid.blogspot.com tizanidinesleepaid.blogspot.com
tnctrx.com tizanidinewithdrawal.blogspot.com
tobeyouday.win tkanorganizma.ru
top10registrycleaners.com tonivedu.it
topclickguru.com totu.info
topvidos.ru totu.us
torrentdownloadhub.com tpu.ru
trafficmp.com tracxn.com
traffixer.com trahvid.com
trailer.cinemaflix.website trainoffend.ml
trustedmaccleaner.com triplepanda.xyz
tubeline.biz
tuberkulezanet.ru tuberkulezanet.ru
tula.mdverey.ru turizm.bz
tvoystartup.ru ukrtvory.in.ua
ukrtvir.com.ua unrealcommander.org
unimodemhalfduplefw.pen.io urlcut.ru
valmetrundan.se urll.eu
vertaform.com urlopener.com
usacasino.com
v-doc.co
v24s.net
varikozdok.ru
video-production.com video-production.com
video-woman.com videochat.bz
videochat.life videospornogratisx.net
vietimgy.pw vilingstore.net
villacoloniale.com villakohlanta.nu
vozbujdenie.com vitalads.net
viveresaniesnelli.it
vkontaktemusic.ru
vladhistory.com
voronezh.xrus.org
vse-pesni.com vse-pesni.com
vzube.com warezaccess.com
w7s.ru webenlace.com.ar
wahicbefa31.soup.io webmasterhome.cn
wallpapersdesk.info webnode.me
warningzscaler.heraeus.com westen-z.life
watch-movies.ru while.cheapwebsitehoster.com
watchinf.com win-spy.com
wdfdocando.com wnoz.de
webinstantservice.com womens-journal.net
webjam.com wonderfulflowers.biz
weblo.com
webscouter.net
weddingdresses.xyz
westsextube.com
westvilletowingservices.co.za
whereverdesperate.gq
whosonmyserver.com
witherrom55.eklablog.fr
wma-x.com
woodyguthrie.se woodyguthrie.se
ww2awards.info wrona.it
xcc24.pl www.ehscloud.cn
xmronta.com www.kabbalah-red-bracelets.com
xn----7sbifcamovvfggw9d.xn--p1ai www.timer4web.com
x-stars.ru
x-true.info
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai xn----9sbubg3ambdfl1j.xn--p1ai
xn--lifehacer-1rb.com xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--80atua3d.xn--p1ai
xtube.com
xxxdatinglocal.us xxxdatinglocal.us
yaoguangdj.com xxxkaz.org
yes-com.com yaminecraft.ru
youghbould.wordpress.com yellowstonevisitortours.com
yourads.website yun56.co
youresponsive.com yunque.pluto.ro
yourmovies.pl zagadki.in.ua
yourtemplatefinder.com zaimite.ru
youtubedownload.org zapatosenventa.info
ytmnd.com zeroredirect6.com
zakazfutbolki.com zetgie.com.pl
zero1.it zetmaster.ru
zeroredirect10.com zigarettenonlinekaufen2.drupalgardens.com
zixizop.net.ru zionstar.net
zskdla.site zkjovpdgxivg.ga
zverokruh-shop.cz zolotoy-lis.ru
zvooq.eu zvooq.eu
zygophyceous.womanstars.site

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
000free.us 123cha.com
00it.com 3weekdiet.com
0912701309f8ce.com 83net.jp
1kinobig.ru aanapa.ru
2020iscoming.info abiente.ru
4kepics.com adpremium.org
4kplayer.pl ads.gold
51unlim.ru adtiger.tk
777-club.ru am15.net
7search.com amateurlivechat.org
aa625d84f1587749c1ab011d6f269f7d64.com anatomiy.com
aa9bd78f328a6a41279d0fad0a88df1901.com angel1777.kz
abctoppictures.net animalia-life.club
adimmix.com apple.com-webbrowsing-security.review
admitad.com atleticpharm.org
ald2014.org aviav.eu
alert-fdm.xyz avtolombard-krasnodar.ru
alert-hgd.xyz backiee.com
alibestsale.com balla-boo.se
altamayoztourism.com bangla-chat-uk-paralud.ga
animal36.com bankiem.pl
apilinkswiftco-a.akamaihd.net baoxaydung.com.vn
artdestshop.eu best-seo-software.xyz
aviav.co bharatdefencekavach.com
azbuka-mo.ru bidr.trellian.com
azlex.uz blackcurranthumidifiers.site
backlinkwatch.com blogsfunky672.weebly.com
bankmib.ru bluerobot.info
batanga.net bmw-ark.ru
best-businessman.ru bolezniorganov.ru
bet365.com boo-arts.com
bimatoprost-careprost.com boole.onlinewebshop.net
bio-optomarket.ru bronzeaid-a.akamaihd.net
biographiya.com brothers-smaller.ru
bjanshee.ru brus-vsem.ru
blpmovies.com buyessay3.blogspot.ru
bonanza-fortune.men buynorxx.com
bonzybuddy.com buzzonclick.com
brandehk.dk byme.se
briomotor.co cakemediahq.com.au
callmd5map.com callawaygolfoutlet.online
candelluxsklep.pl camdolls.com
canrioloadm.gq canyougethighofftizanidine.blogspot.com
cartierlove2u.com cascadelink.org
caveavins.fr casinopinup-wins.com
clayvasedesigns.tk celebsopics.com
clevernt.com centrdebut.ru
clickso.com cheapsergiorossi.online
cmd.kz chocolateslim-en-espana.com
compliance-barak.xyz clickzzs.nl
compliance-donald.xyz coinsspb.com
compliance-irvin.top com-onlinesupport.site
compliance-george.xyz
compliance-ivan.top
compliance-nelson.top
comsysnet.com
concordexoticrentals.com concordexoticrentals.com
crest-poloski.ru credit.co.ua
dafatiri.com dafatiri.com
delayreferat.ru daptravel.com
depositfiles-porn.ga daretodonate.co
detki-opt.ru dawlenie.com
discountbarbour.online day-news.info
djstools.com dcj-nn.ru
documentbase.net digitalassetmanagement.site
dolohen.com domaincdn.xyz
dosugrostov.site domashneeruporno.com
downloads-whatsapp.com dpihatinh.gov.vn
dustyorate.com e-biznes.info
dvr.biz.ua e90post.com
elmifarhangi.com ecookna.com.ua
englishdictionaryfree.com ekspertmed.com
erotikstories.ru elektroniksigaraankara.info
essay-edu.biz elektrozigaretten1.webs.com
evogarage.com expolicenciaslatam.co
family1st.ca ezaz.info
fanoboi.com ezigarettekaufen2.yolasite.com
femdom.twiclub.in ezigarettenkaufen1.deviantart.com
filmi-v.online fickenprivat.info
forums.d2jsp.org fighrofacciufreesig.ga
forums.toucharcade.com fil.ru
frbizlist.com filerockstar298.weebly.com
freejabs.com film-one.ru
freemaintenancesysforpcandmac.top filmgo.ru
fsalas.com fiverr.com
g33.org foxydeal.com
galaxyflowers.ru franecki.net
gloverid.site free-today.com
golden-praga.ru freelinkbuilding.website.tk
good-mummy.ru freewhatsappload.com
goodwriterssales.com ftns.ru
googglet.com globatur.ru
googlefeud.com gsasearchengineranker.site
gpirate.com gungamesz.com
greatgrace.ru hawaiielectriclight.com
gsasearchenginerankerhelp.com hdwalls.xyz
hard-porn.mobi healing-dysplasia.ru
hopeonthestreet.co.uk heliko.no
hotloans.ru hobbyhuren24.net
hoverboardforsaledirect.com hotgirlhdwallpaper.com
howtoclean.club houseofrose.com
https-legalrc.biz i-hobot.ru
humbmosquina.tk iblogpress.xyz
ibb.com.ua ideoworld.org
ikritikimou.gr igtools.club
imk.com.ua interior-stickers.ru
inboundlinks.win islandminingsupply.wordpress.com
infostatsvc.com ispaniya-costa-blanca.ru
intim-uslugi.info istanbulit.com
investsuccess.org jerseywholesaleelitestore.com
istizanidinelikexanax.blogspot.com johnrobertsoninc.com
iyasimasennka.com keyword-suggestions.com
javcoast.com keywordspay.com
joy-penguin.com kinofak.net
kazinogames.lv kndxbkdx.bloger.index.hr
kerwinandcariza.com kometars.xyz
kinohit1.ru konpax.com
kotaku.com kosynka-games.ru
kretpicf.pw landscapebackgrounds.blogspot.com
lechenie-gemorroya.com layola.biz.tc
lexaprogeneric.link lerporn.info
lookingglassemb.com lifeinsurancekb.com
lordzfilmz.me liupis.com
macdamaged.tech livefixer.com
mall.uk livingcanarias.com
marinetraffic.com locustdesign.co
marmitaco.cat longgreen.info
missvietnam.org louboutinreplica.xyz
morefastermac.trade m.b00kmarks.com
mosrif.ru magicdiet.gq
mountainstream.ms mailemedicinals.com
musicpro.monster maranbrinfo.com.br
myghillie.info master-muznachas.ru
mylovelibrabry.com matchpal-a.akamaihd.net
mypornfree.ru medizinreisen.de
myshopmatemac.com menetie.ru
naj-filmy24.pl metarip.ru
natali-forex.com militarysale.pro
networkad.net missis.top
mjchamonix.org
mlf.hordo.win
moneymaster.ru
moneyteam24.com
monsterdivx.tv
mouselink.co
mrvideospornogratis.xxx
myhitmp3.club
myiptest.com
newhairstylesformen2014.com
nikhilbahl.com nikhilbahl.com
november-lax.com niki-mlt.ru
o-dachnik.ru nonews.co
oivcvx.website nordstar.pro
oklogistic.ru onlineserialy.ru
one.net.in onlinewritingjobs17.blogspot.ru
online-sbank.ru oxford-book.com.ua
onlinemeetingnow.com ozas.net
ons-add.men partner-stop.men
ontargetseo.us pdn-4.com
ooo-olni.ru pensplan4u.com
open-odyssey.org perm.xrus.org
opinionreelle.com phpdevops.com
optikremont.ru picsearch.com
paccohichetoti.ml popup-fdm.xyz
parlament.biz pornhive.org
piccdata.com pornohub.me
picurams.pw pp-budpostach.com.ua
pornoelita.info prchecker.info
predominant-invent.tk prizestohandle.club
profit-opportunity.com prostitutki-kharkova.org
profitwithalex.info purplesphere.in
pussyspace.net quick-offer.com
puzo2arbuza.ru ranksignals.com
ranksonic.info rapevideosmovies.com
rarbg.to reklamuss.ru
rcb101.ru renewablewealth.com
rcpmda.ikan1080.xyz responsive-test.net
remmling.de rialp.getenjoyment.net
reversing.cc rn-to-bsn.com
rfd-split.hr rome2rio.com
rfserial.net ru-mediaget.ru
ringporno.com runofilms.ru
rus-pornuha.com rustic-quiver.win
rusprostitute.com salutmontreal.com
sawin.beth.webd.pl samoiedo.it
sc-specialhost.com schoolfiles.net
scanner-elena.top seemoreresultshu-a.akamaihd.net
scat.porn sethrollins.net
sdsjweb.com sex-sex-sex5.com
searchimpression.com sexkontakteao.info
seo-tools-optimizing.com sexpartygirls.net
serving.adbetclickin.pink
sex-znakomstva.online
sexyali.com sexyali.com
sexystrippe.info sharebutton.net
sexyteens.hol.es shop.xz618.com
shopcheermakeup.info slimcdn.com
shopwme.ru slonechka.ru
shtaketniki.kz snowplanes.com
siteripz.net soaksoak.ru
skladvaz.ru socialbuttons.xyz
slowmac.tech sport7777.net
soft1.ru spy-app.info
songplanet.ru stackthatbucks.com
sonnikforme.ru stretchingabuckblog.com
sotkal.lark.ru studentguide.ru
spammen.de sudexpert66.ru
sr-rekneskap.no superinterstitial.com
stockquotes.wooeb.com tacbibirfa.tk
strag-invest.ru td-l-market.ru
supervesti.ru teletype.in
surcentro.com testingads.pro
surffoundation.nl thegalerie.eu
taboola.com themestotal.com
taranerymagesswa.blogspot.com tizanidinemusclerelaxant.blogspot.com
tds-advert005.info tomatis.gospartner.com
terraclicks.com tongkatmadura.info
teslathemes.com toonfamilies.net
tizanidineformigraines.blogspot.com track-rankings.online
tizanidineonline.blogspot.com trafficjunky.net
topcasinoratings.ru treasuretrack-a.akamaihd.net
topkarkas.com tsstcorpcddvdwshbbdriverfb.aircus.com
trubywriting.com
trustl.life
tryrating.com
tubeoffline.com
tula.howotorg.ru
u555u.info u555u.info
unpredictable.ga uchetunet.su
urlopener.blogspot.com.au umityangin.net
usdx.us universals.com.ua
vabasa.inwtrade.com vigrx-original.ru
valmetrundan.se vitanail.ru
videospornogratisx.net vkonche.com
vitoriacabos.com vksaver-all.ru
vladimir.xrus.org walpaperlist.com
voloomoney.com webs.com
waysbetter.cn wedding0venues.tk
weburok.com wesharepics.site
wingsofrefuge.net whereiskentoday.com
witherrom55.eklablog.fr wholinkstome.com
woman-tampon.ru wnoz.de
word-vorlagen.xyz woodyguthrie.se
works.if.ua works.if.ua
wowas31.ucoz.ru woweb.com.ua
wygraj-skiny.win
wzgyyq.com
x-musics.com x-musics.com
xblog.in xaijo.com
xerox-douglas.cf
xjlottery.com xjlottery.com
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai xxxtube69.com
xtube.com yellowfootprints.com
xvideosbay.com yourserverisdown.com
yoshkarola.zrus.org yourtemplatefinder.com
youradexchange.com
youradulthosting.com
youtube-downloader.savetubevideo.com
za-fun-offer.com
zahvat.ru
zaim-pod-zalog-krasnodar.ru
zdesformula.ru
zebradudka.com
zero1.it zero1.it
zeroredirect11.com zocaparj.kz
znaniyapolza.ru zx6.ru

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.4 0; 10.1.0.43 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3517 ### Version: V4.2022.12.3518
### Updated: Wed Dec 14 10:03:40 UTC 2022 ### Updated: Thu Dec 15 10:04:06 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3517 ### Version: V4.2022.12.3518
### Updated: Wed Dec 14 10:03:40 UTC 2022 ### Updated: Thu Dec 15 10:04:06 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##