vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.03.2972 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-03-11 10:09:19 +00:00
parent e6214101f3
commit 408acd779c
47 changed files with 87590 additions and 87580 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.03.2971 #### Version: V4.2022.03.2972
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.03.2971 #### Version: V4.2022.03.2972
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.03.2971 #### Version: V4.2022.03.2972
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10812
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.03.2971 #### Version: V4.2022.03.2972
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10816
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10816
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10816
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10816
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

10816
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

10816
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

10816
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2971 2972

10816
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10816
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.217.71 0; 20.25.112.85 0;

10816
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

10816
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

10816
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

10816
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.217.71 0; 20.25.112.85 0;

10816
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.43.217.71 0; 20.25.112.85 0;

10816
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

16
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Buck
CazoodleBot CazoodleBot
Craftbot
Custo Custo
LieBaoFast DnBCrawler-Analytics
LinkWalker NetMechanic
MolokaiBot Nibbler
SISTRIX RankActive
Snake Thumbor
Spbot Voltron
WBSearchBot magpie-crawler

298
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
ALittle Client 404enemy
ASPSeek Aboundexbot
Abonti AllSubmitter
Acunetix AlphaBot
AiHitBot
Anarchy Anarchy
Anarchy99 Aspiegel
Anthill AwarioRssBot
Apexoo BLEXBot
Attach
BDCbot
BDFetch
BackStreet
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Bandit
Battleztar Bazinga
BetaBot
Bigfoot
Bitacle Bitacle
Blackboard Black Hole
Blow BlackWidow
BotALot BlowFish
Boardreader
Bolt
BuiltWith BuiltWith
CATExplorador CATExplorador
CCBot CazoodleBot
CODE87
Calculon
CensysInspect CensysInspect
CheTeam CheeseBot
CherryPicker
ChinaClaw
Chlooe Chlooe
Citoid Claritybot
Cloud mapping CopyRightCheck
Cocolyzebot Craftbot
Cogentbot CrazyWebCrawler
Collector CrunchBot
Copyscape DBLBot
Cosmos
CyotekWebCopy
DIIbot DIIbot
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli Devil
Demon Digincore
Dirbuster Dirbuster
DomCopBot Discobot
DomainAppender DnBCrawler-Analytics
Domains Project DnyzBot
Download Wonder DomainSigmaCrawler
DomainStatsBot
Dragonfly Dragonfly
Drip
ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf
EasyDL EasyDL
EirGrabber
Evil
Exabot Exabot
Express WebPictures Express WebPictures
Extractor ExtLinksBot
FemtosearchBot Ezooms
FlashGet FrontPage
Freeuploader Fuzz
G-i-g-a-b-o-t GT::WWW
GalaxyBot GalaxyBot
GetWeb GermCrawler
Getintent
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot
HEADMasterSEO
HMView HMView
HaosouSpider
Havij
Hloader Hloader
Humanlinks
HybridBot HybridBot
IDBot IDBTE4M
Iblog
Id-search Id-search
Image Fetch IlseBot
Image Sucker
InfoNaviRobot InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
Iskanie Internet Ninja
JOC Web Spider JOC Web Spider
Jbrofuzz Jbrofuzz
JennyBot JennyBot
Jetty
JikeSpider JikeSpider
Joomla JustView
Jyxobot Jyxobot
Kenjin Spider
Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot
Larbin Larbin
Leap Leap
LeechFTP LeechFTP
Lftp
LibWeb LibWeb
Libwhisker
Lightspeedsystems Lightspeedsystems
Likse Likse
LinkWalker
Linkdexbot Linkdexbot
LinkextractorPro
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey LinqiaScrapeBot
Lmspider MFC_Tear_Sample
MJ12bot
MTRobot MTRobot
Magnet Mag-Net
Mail.RU_Bot MauiBot
Masscan
Mb2345Browser Mb2345Browser
Meanpathbot Meanpathbot
MegaIndex.ru MegaIndex.ru
Metauri Metauri
MicroMessenger
Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Mister PiX Minefield
Moblie Safari Mojolicious
Morfeus Fucking Scanner NICErsPRO
Mozlila
NPbot NPbot
Name Intelligence Nameprotect
Needle NearSite
Net Vampire Net Vampire
NetAnts
NetLyzer NetLyzer
NetZIP NetZIP
Netcraft
Nettrack Nettrack
NextGenSearchBot NextGenSearchBot
Nikto Nikto
Nimbostratus
Ninja
Not Not
Octopus Nuclei
OrangeBot Offline Explorer
Openvas
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl PHPCrawl
Page Analyzer PageAnalyzer
PageScorer PageGrabber
Papa Foto Papa Foto
Petalbot Pavuk
Picscout Picscout
Pixray Picsearch
PleaseCrawl PictureFinder
Pimonster
Pockey Pockey
ProPowerBot ProPowerBot
Probethenet
Psbot Psbot
Pu_iN Pu_iN
Pump
PxBroker
PyCurl PyCurl
QueryN Metasearch
RankFlex
RankingBot
RankingBot2 RankingBot2
Rankivabot Re-re
ReGet ReGet
RealDownload RocketCrawler
Reaper Rogerbot
RebelMouse SBIder
RepoMonkey SEOlyticsCrawler
SEOkicks
SEOkicks-Robot
ScanAlert
Scanbot
ScreenerBot ScreenerBot
Searchestate Searchestate
SemanticJuice
SemrushBot
SeoSiteCheckup SeoSiteCheckup
Shodan Seomoz
Site Sucker
SiteSnagger
Sitebeam Sitebeam
Siteimprove Siteimprove
Sitevigil Sitevigil
Snake SmartDownload
Sociscraper Snapbot
Sottopop Sogou web spider
Spanner Spinn3r
SputnikBot SputnikBot
Sqlworm
Sqworm Sqworm
Stripper Stripper
Suzuran SuperBot
SuperHTTP
Surfbot
Szukacz Szukacz
T8Abot T0PHackTeam
TightTwatBot Teleport
Titan TeleportPro
Toata Telesphorep
Toweyabot
Trendiction
Trendictionbot
True_Robot True_Robot
TwengaBot Turingos
Twice Turnitin
URLy Warning
URLy.Warning
UnisterBot UnisterBot
Upflow
V-BOT V-BOT
VB Project VB Project
Vagabondo
Voil
WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WISENutbot
WPScan WPScan
WWW::Mechanize WWW-Collector-E
WWWOFFLE Wallpapers
Wallpapers/3.0 WeSEE
Web Auto Web Auto
Web Collage Web Sucker
Web Enhancer
Web Fuck
WebBandit
WebCollage
WebEnhancer
WebFetch WebFetch
WebFuck WebImageCollector
WebGo IS
WebPix WebPix
WebReaper WebReaper
WebSauger WebSauger
WebStripper
WebWhacker
WebZIP WebZIP
Webshag Webalta
WebsiteExtractor Website Quester
WebsiteQuester WebsiteQuester
Webster Whack
WinHTTrack Whatweb
WiseGuys Robot
Woobot Woobot
Xaldon WebSpider Wprecon
Xenu Zermelo
Zade ZmEu
Zauba
Zeus
ZumBot ZumBot
adscanner
archive.org_bot archive.org_bot
backlink-check arquivo-web-crawler
coccocbot arquivo.pt
com.plumanalytics autoemailspider
cah.io.community
crawler4j crawler4j
dataforseo.com eCatch
evc-batch
facebookscraper facebookscraper
isitwp.com gopher
iubenda-radar instabid
internetVista monitor
lwp-request lwp-request
lwp-trivial lwp-trivial
mediawords magpie-crawler
meanpathbot
muhstik-scan muhstik-scan
seocompany.store netEstate NE Crawler
seoscanners page scorer
plumanalytics
polaris version
scalaj-http
scan.lol
seobility
seostar seostar
serpstatbot serpstatbot
sysscan sitechecker.pro
trendiction.de siteripz
voyagerx.com sp_auditbot
webprosbot trendiction.com
xpymep1.exe webgains-bot
zauba.io webpros.com
zgrab x09Mozilla
x22Mozilla

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404checker
404enemy 404enemy
80legs 80legs
ADmantX ADmantX
Abonti AIBOT
ASPSeek
Aboundex Aboundex
Acunetix Acunetix
AhrefsBot AhrefsBot
AlphaBot Alexibot
Alligator
Anarchie
Anthill Anthill
Aspiegel Aspiegel
Attach Asterias
AwarioRssBot Atomseobot
AwarioSmartBot
BBBike BBBike
BDFetch BDFetch
BackDoorBot BLEXBot
BackStreet
BackWeb BackWeb
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
Bandit Bandit
Battleztar Bazinga BatchFTP
Bitacle
Black Hole Black Hole
BlackWidow BlackWidow
Blackboard Blackboard
BlowFish Blow
Boardreader
Brandprotect Brandprotect
Brandwatch Buck
Bullseye BuiltWith
CCBot
CSHttp
Calculon
CazoodleBot
CensysInspect
CheTeam CheTeam
CheeseBot CheeseBot
Cogentbot Cliqzbot
CrazyWebCrawler CrazyWebCrawler
Curious CrunchBot
Custo
CyotekWebCopy
DSearch DSearch
DTS Agent Demon
Deusu Devil
Dirbuster DigitalPebble
Dispatch Discoverybot
DittoSpyder DittoSpyder
DomCopBot DnBCrawler-Analytics
DomainSigmaCrawler DnyzBot
DomainStatsBot DomainCrawler
EMail Wolf Ebingbong
ExtLinksBot Ecxi
Ezooms Evil
Exabot
Extractor
ExtractorPro
Extreme Picture Finder
FemtosearchBot FemtosearchBot
Fimap Fimap
Firefox/7.0 Firefox/7.0
FlashGet
Flunky Flunky
Freeuploader
Fuzz
FyberSpider FyberSpider
Fyrebot GermCrawler
GalaxyBot Go-Ahead-Got-It
Genieo GoZilla
GetRight Gotit
Getintent
HEADMasterSEO
HTTrack
Haansoft Haansoft
HaosouSpider HaosouSpider
Harvest
Havij Havij
Heritrix
Humanlinks Humanlinks
IRLbot
Iblog Iblog
Id-search Image Sucker
Image Fetch
IndeedBot
Indy Library Indy Library
InfoNaviRobot InfoTekies
Intelliseek IstellaBot
InterGET
Internet Ninja
Iskanie
JOC Web Spider JOC Web Spider
JennyBot JennyBot
JikeSpider JetCar
Joomla Jetty
Jorgee Jorgee
Kenjin Spider JustView
Jyxobot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density Kinza
LNSpiderguy
LWP::Simple
Larbin Larbin
LeechFTP LeechFTP
LeechGet
Lftp Lftp
LibWeb LibWeb
Libwhisker
Lightspeedsystems Lightspeedsystems
LinkScan LinkScan
LinkWalker
Linkbot Linkbot
Linkdexbot LinksManager
LinkpadBot LinqiaScrapeBot
LinqiaMetadataDownloaderBot Lipperhey
Litemage_walker
Lmspider Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MQQBrowser MQQBrowser
MTRobot MSFrontPage
Mail.RU_Bot Mag-Net
Majestic SEO Magnet
Majestic12
MarkMonitor MarkMonitor
Mass Downloader
Masscan Masscan
Mata Hari
MauiBot MauiBot
MeanPath Bot
MicroMessenger
Microsoft Data Access
Microsoft URL Control
Minefield Minefield
Mister PiX Mister PiX
Moblie Safari
Mojolicious Mojolicious
MolokaiBot Musobot
Mr.4x3 NICErsPRO
NPbot NPbot
Name Intelligence Name Intelligence
NearSite Nessus
Net Vampire
NetAnts
NetLyzer NetLyzer
NetZIP Nettrack
Netcraft Netvibes
NextGenSearchBot NextGenSearchBot
Niki-bot Nibbler
Nikto NimbleCrawler
Ninja Offline Navigator
OpenLinkProfiler Openfind
OpenVAS OrangeSpider
OutfoxBot PageAnalyzer
PECL::HTTP Panscient
Papa Foto PeoplePal
Petalbot Petalbot
Pi-Monster
Picscout Picscout
Picsearch Picsearch
Pockey Piepmatz
Probethenet Pimonster
PyCurl Pu_iN
QueryN Metasearch Pump
RSSingBot
RankActive RankActive
RankFlex
RankingBot
RankingBot2 RankingBot2
RankurBot Recorder
Re-re RedesScrapy
RealDownload
Reaper
RebelMouse
Rogerbot Rogerbot
SBIder SEOkicks-Robot
SEOkicks SISTRIX
SMTBot SMTBot
ScoutJet SalesIntelligent
SemanticJuice ScanAlert
SemrushBot Scanbot
SeoSiteCheckup Screaming
Siphon ScrepyBot
Searchestate
SeobilityBot
SiteExplorer SiteExplorer
SiteLockSpider Siteimprove
SmartDownload Sitevigil
Snapbot Snake
Sogou web spider Snoopy
SpaceBison
Spammen Spammen
SpankBot SputnikBot
Spanner Sqlworm
Sqlmap Sqworm
Steeler
Stripper
Sucker Sucker
Sucuri Sucuri
Suzuran SuperBot
Swiftbot
Szukacz Szukacz
T0PHackTeam
T8Abot
Teleport
TeleportPro TeleportPro
Telesoft
The Intraformant The Intraformant
TheNomad
TightTwatBot TightTwatBot
Toweyabot
Tracemyfile
Trendiction Trendiction
Trendictionbot
True_Robot True_Robot
Turingos
Turnitin
TurnitinBot TurnitinBot
URLy Warning URLy.Warning
UnisterBot
V-BOT V-BOT
VB Project VB Project
Vacuum Vacuum
Vagabondo
VeriCiteCrawler
VidibleScraper
Voil Voil
WASALive-Bot Voltron
WEBDAV WEBDAV
WISENutbot WISENutbot
WPScan WPScan
WWW-Collector-E WWW-Mechanize
WWW::Mechanize Wallpapers/3.0
Wallpapers WallpapersHD
WeSEE WeSEE
Web Collage Web Fuck
Web Fetch Web Pix
WebAuto WebBandit
WebCollage WebCollage
WebCopier WebEnhancer
WebFetch WebFetch
WebGo IS WebFuck
WebPix WebImageCollector
WebSauger WebLeacher
WebSucker Webalta
WebZIP Webshag
WebmasterWorldForumBot
Website Quester Website Quester
Webster Whacker
Wonderbot Whatweb
Woobot Widow
WiseGuys Robot
Wprecon Wprecon
Zade Zade
Zauba
Zeus Zeus
ZmEu ZoominfoBot
ZoomBot adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
check1.exe
clark-crawler clark-crawler
coccocbot
crawl.sogou.com crawl.sogou.com
dataforseo.com
domainsproject.org
eCatch eCatch
facebookscraper facebookscraper
heritrix
internetVista monitor
isitwp.com isitwp.com
lwp-request magpie-crawler
lwp-trivial
meanpathbot
mediawords
netEstate NE Crawler
page scorer page scorer
polaris version
ripz
s1z.ru
scalaj-http
scan.lol scan.lol
serpstatbot seocompany.store
sitechecker.pro seostar
sogouspider sexsearcher
siteripz
sp_auditbot sp_auditbot
spyfu spyfu
trendiction.de tAkeOut
trendiction.com
voyagerx.com voyagerx.com
webmeup-crawler webpros.com
webprosbot zgrab
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
alfabot.xyz emerson-rus.ru
artefakct.com kinky-fetishes.com
compliance-brian.xyz lawyers.cafe
it-max.com.ua mobilierland.com
mall.uk sax-sex.com
matchpal-a.akamaihd.net searchengineranker.email
onlyforemont.ru sosdepotdebilan.com
pavlodar.xkaz.org theporndude.com
preparevideosafesystem4unow.site uptime.com
timetorelax.biz website-analytics.online

948
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 0n-line.tv
00it.com 14b.info
11pikachu.ru 1stat.ru
1688.com 2728fb936f0.com
178evakuator178.ru 2itech.ru
1webmaster.ml 55wmz.ru
2ads.co.uk 5u.com
4inn.ru a.pr-cy.ru
6hopping.com aa8780bb28a1de4eb5bff33c28a218a930.com
8xv8.com aad1f4acb0a373420d9b0c4202d38d94fa.com
991mostfm.co.id abcguru.xyz
aa08daf7e13b6345e09e92f771507fa5f4.com abwa.tk
aaronabel.com acc.eu.org
absugars.com adconscious.com
adclickthru.net addblueoff.com.ua
adpremium.org adobereader-free.ru
adultnet.in adtiger.tk
advancedmassagebysara.com advancedsoftwaresupport.com
advokateg.ru adverster.com
afslankpillen2017nl.eu agardomains.com
alert-fdm.xyz agreda.pluto.ro
alfa9.com airmaxshoes-2016.com
algarveglobal.com alf-img.com
all4invest.info alienwheel.es
alldezservice.kz alldownload.pw
alt-servis.ru alloysteel.ru
amoi.tn amehdaily.com
animal-drawings.com androids-store.com
animal36.com animali.deagostinipassion.it
animebox.com.ua anonymousfox.co
aosheng-tech.com apiallgeniusinfo-a.akamaihd.net
apiluckyleapnet-a.akamaihd.net apidiamondatanet-a.akamaihd.net
app-ready.xyz apparel-offer.com
appfixing.space apple.com-webbrowsing-security.review
arendatora.ru apxeo.info
arendovalka.xyz arcadepages.com
arkkivoltti.net arclk.net
arquapetrarca.info arius.tech
asscenihotosidea.blogspot.co.za arkartex.ru
asscenihotosidea.blogspot.com arvut.org
atelielembrancaqueencanta.com.br asmxsatadriverin.aircus.com
ausergrubhof.info audiofree.ru
avek.ru autoloans.com
ayerbo.xhost.ro autonew.biz
babyguns.ru autotrends.today
bag77.ru avon-novosib.ru
balitouroffice.com avramstroy.ru
balkanfarma.org babespcs.com
baikaleminer.com
balois.worldbreak.com balois.worldbreak.com
balook.com bamo.xsl.pt
banan.tv basisches-wasser.net
belstaffstore.online bazaronline24.ru
bensbargains.net bedcapdealers.com
best-coupon-offer.com
best-deals-products.com
best-seo-software.xyz best-seo-software.xyz
bestchoice.cf best-seo-solution.com
bestpriceninja.com bg6s0.com
bestsexyblog.com bget.ru
beta.hotkeys.com bidbuy.co.kr
betterscooter.com biglistofwebsites.com
bezlimitko.xyz bitcoinpile.com
biznesluxe.ru blog.axant.it
bmsco.co blogs.rediff.com
bochemit.com.ua bocoarchives.org
bosefux.esy.es boeuklad.com
bronzeaid-a.akamaihd.net bonvillan.com
brothers-smaller.ru bpro1.top
bus-offer.com braip.com.br
buzzzg.men buchananshardware.com
callmd5map.com buqayy0.livejournal.com
callstevens.com buy-meds24.com
calstaterealty.us buyhoverboard.com
canrioloadm.gq buyparajumpers.online
carstrends2015.com cah.io.community
cartierreplica.xyz candytiner.com
cheapcigarettesc.info celebsopics.com
children-learningreading.info cheapsergiorossi.online
chlooe.com chinaelitecheapjerseys.com
chocolateslim-en-espana.com christianlouboutinoutlet.win
cibpenonptib.flu.cc christianlouboutinsaleoutletonline.us
citizenclsdriveri7.pen.io cialgenisrx.com
clickhype.com ciekawinki.pl
clickpapa.com cineacademy.ru
clickprotects.com cjmc.info
clickzzs.nl clickcash.com
cocyq.inwtrade.com clicksor.com
coindirect.io clodo.ru
coinsspb.com club-samodelkin.ru
collegeessay19.blogspot.ru clubfashionista.com
compliance-barak.xyz cognitiveseo.com
compliance-checker-7.info com-onlinesupport.host
compliance-jess.top comissionka.net
construmac.com.mx compiko.info
copro.pw compliance-george.xyz
coslab.club conboy.us
crazyprotocol.com contentlook.co
credit-respect.ru counter.bloke.com
creditwell.ru creditnation.ru
csgo4.win csgo4.win
customcatchcan.com cupday.com
czeshop.info custom-electric-guitar.com
dailyrank.net customboxes4less.com
darkbooks.org damedingel.ya.ru
darmebel.com.ua dating-app.ru
data-mining.tk ddsoldes.fr
ddlmega.net de.zapmeta.com
degerlund.net dealighted.com
detective01.ru decorationspcs.com
detsky-nabytek.info deda-moroza-zakaz.ru
dev33.dioniqlabb.se dekorkeramik.ru
dikx.gdn deluxedumps.com
discountbarbour.online detalizaciya-tut.biz
djekxa.ru deutschland123.de
dms-sw.ru dev78.dioniqlabb.se
documentbase.net diminishedvalueoforegon.com
doktoronline.no dinosaurus.site
domaincrawler.com diusyawiga.tk
domainsatcost.com doeco.ru
dotnetdotcom.org domain2008.com
dpihatinh.gov.vn domainroam.win
draniki.org doreenblog.online
drillsaw.ru doska-vsem.ru
drugs-no-rx.info download-wallpaper.net
e-collantes.com downloadkakaotalk.com
e-kwiaciarz.pl doxysexy.com
easytuningshop.ru doyouknowtheword-flummox.ml
ec-file.info dynainbox.com
ed-shop01.ru e-commerce-seo.com
easyukraine.com
econom.co
ednorxmed.com
education-cz.ru
ekn-art.se
elektronischezigarettekaufen1.myblog.de elektronischezigarettekaufen1.myblog.de
enthuse.computernetworksonline.com elektrozigaretten1.tumblr.com
eonpal.com elitepcgames.com
erotische-geschichten-xxl.com eropho.net
espaceinventoristes.com erot.co
estibot.com eugenevaultstorage.com
expdom.com evrotekhservis.ru
expertblog.info exoclick.com
ezigarettekaufen.myblog.de eyelike.com.ua
eyes-on-you.ga
eyessurgery.ru
ezigaretteshop2.mywapblog.com
ezrvrentals.com ezrvrentals.com
factorynightclub.com fantasticpixcool.com
fakehandbags.xyz fashionindeed.ml
falcon-images.blogspot.com favornews.com
fetlifeblog.com fertilitetsradgivningen.se
filmci.pro filunika.com.ru
filmi-v.online findanysex.com
financehint.eu findsexguide.com
fiuxy.com flipper.top
floridahuntingfishingadventures.com floridamhca.org
forex-procto.ru fm-upgrade.ru
forexgb.ru focalink.com
fordsonmajbor.cf
forpostlock.ru
forum-engineering.ru
fotzen-ficken.com
freakycheats.com
free-fb-traffic.com
freefoto.ca
freevpn.space
frequiry.com
friendflnder.com friendflnder.com
frighteningremain.cf funnymama.com
frivgame250.com gameplexcity.com
frvo.alptandem.ru gate5.co.za
fxtips.ru gateway.zscalerone.net
galleryawesome.com gavuer.ru
garciniaxt.us
gay.adultgalls.com gay.adultgalls.com
ge0ip.net gaypornmovie.net
get-seo-domain.com geoads.com
getmiro.com gg-arena.ru
gg.zzyjxs.com ghostvisitor.com
ggiaro.com gilsonchiro.xyz
golaya.pw globatur.ru
goldadpremium.com goldandcard.ru
goo.ne.jp golfresa.lucania.se
good-mummy.ru gorodservis.ru
goodwinmetals.co gox.com.ua
goosefishpost.bid guidefs.ru
gpms.org.my guitar-master.org
gribkovye-zabolevaniya.com hamacapty.com
grtyi.com hayate.biz
grupografico-pilar.com.ar hchha.com
gsasearchengineranker.xyz hdhc.ru
handicapvansarea.com hdxnxxtube.mobi
hd-filmy.net healbio.ru
hexpilot.com hermesbirkinhandbagoutlets.com
heygidday.biz hitsbox.info
hitmuzik.ru hol.es
horoshieokna.com homemature.net
hostcritique.com hoporno.com
hotkeys.com hotblog.top
hotloans.ru hotelcrocenzi.sm
housediz.com howopen.ru
housekuba.org howtowhitenteethfast.xyz
houston-vikings.com hptwaakw.blog.fc2.com
hoztorg-opt.ru hulfingtonpost.com
huhn.altervista.org hully.altervista.org
hvd-store.com ido3.com
hyhj.info igrovyeavtomaty777.ru
i-hobot.ru imfamous.info
icaseclub.ru img.wallpaperstock.net
ifirestarter.ru imk.com.ua
iframe-toloka.com impotentik.com
igithab.com incolors.club
iklysha.ml instasexyblog.com
ikritikimou.gr int.search.mywebsearch.com
ilovevitaly.com integritylandscapeservices.com
indianmedicaltourismshop.com interferencer.ru
infobabki.ru interior-stickers.ru
infodocsportal.com
infowarcraft.ru
inspiring-desperate.tk
interesnie-faktu.ru
internetproviderstucson.com
iplogger.org
iptool.xyz iptool.xyz
irkutsk.zrus.org
ispaniya-costa-blanca.ru
istripper.com istripper.com
ix20.ru it-max.com.ua
janerikholst.se itronics.ca
javcoast.com izismile.com
jerseyssportsshop.com jabimgo.pw
jetsli.de jav-fetish.com
javatex.co.id
jerseychinabizwholesale.com
jmat.cn jmat.cn
jus0wil.pp.ua job.icivil.ir
kaliningrad.zrus.org jose.mulinohouse.co
karaganda.xkaz.org kamalsinha.com
kazan.zrus.org kandidos.com
kazinogames.lv keywords-monitoring-success.com
kchaxton.com kfon.eu
kevblog.top kimcurlrvsms.com
keywordblocks.com kinky-fetishes.com
kinozapas.org kinogonew.ru
kiwi237au.tk konkursowo-24.pl
kosmetyki.tm.pl kumuk.info
kotaku.com laapp.com
kraljeva-sutjeska.com lacloop.info
krasivye-devushki.net lampokrat.ws
ktotut.net laudit.ru
kw21.org leadwayau.com
labelwater.se lenvred.org
ladsup.com lignofix.ua
landinez.co livefixer.com
leadn.pl liverpool.gsofootball.com
ledpolice.ru liveshoppersmac.com
letto.by lostaruban.com
linkdebrideur.xyz louboutinshoes.xyz
littleberry.ru luckyday.world
littlesunraiser.com luxmagazine.cf
lodki-pvh.dp.ua machicon-akihabara.info
lombia.com maglid.ru
lordzfilmz.me magnetic-bracelets.ru
lulea-auktionsverk.se makis.nu
lyngdalhudterapi.no manifestation.betteroffers.review
m3gadownload.pl map028.com
marcogrup.com marcogrup.com
maslenka.kz mazda-roadsters.com
master-muznachas.ru mearns-tractors.co.uk
maxthon.com mebel-alait.ru
mebeldekor.com.ua
mediaoffers.click mediaoffers.click
medicine-4u.org
medicines-choice.com
mega-bony2017.pl
megahdporno.net
meget.co.za meget.co.za
mesbuta.info
metalonly.info metalonly.info
mexicosleevegastrectomy.com migente.com
middlerush-a.akamaihd.net minneapoliscopiers.com
mirtorrent.net mir-business-24.ru
mixx.com misslike.ru
mmgq.ru mnogabukaff.net
mobifunapp.weebly.com
mobilemedia.md mobilemedia.md
moesen-ficken.com mobilierland.com
moi-glazki.ru mobot.site
moneymaster.ru mobplayer.net
morpicert.pw mockupui.com
moviemail-online.co.uk moscow-region.ru
movpod.in moscow.online-podarki.com
mp3downloadhq.com moz.com
mrcsa.com.au mttwtrack.com
mulberryoutletonlineeu.com
mustwineblog.com
mvpicton.co.uk
my-bc.ru my-bc.ru
mypets.by mylovelibrabry.com
myphotopipe.com mzdish.site
niki-mlt.ru nakrutka.cc
nikkiewart.ru narkologiya-orel.ru
nl.netlog.com naughtyconnect.com
nlfjjunb5.ru neodownload.webcam
nofreezingmac.click netallergy.ru
nordstar.pro news-readers.ru
nosecret.com.ua nextlnk12.com
o-o-6-o-o.com nhl17coinsforps3.gratisblog.biz
o-o-6-o-o.ru noclegonline.info
o-o-8-o-o.com nottyu.xyz
obiavo.in nudo.ca
obiavo.uz oakleyglassesonline.us
oconto.ru obiavo.com
odesproperty.com offers.bycontext.com
ok.ru okroshki.ru
one-gear.com oksrv.com
one.net.in olvanto.ru
onlinefilmz.net omoikiri-japan.ru
onlineserialy.ru online-sbank.ru
onlyporno.ru onlythegames.com
openmultipleurl.com optom-deshevo.ru
panamaforbeginners.com orel-reshka.net
partnerline.men owathemes.com
patterntrader-en.com paparazzistudios.com.au
paydayonlinecom.com paperwritingservice17.blogspot.ru
pendelprognos.se parfusale.se
phormchina.com park.above.com
piccshare.com partners-ship.pro
picphotos.net paydayloanslocal.com
pipki.r.acdnpro.com pc-services.ru
pladform.ru pdns.cz
play-movie.pl pelfind.me
playtap.us petrovka-online.com
pokemon-go-play.online phobia.us
polcin.de pillscheap24h.com
popads.net piluli.info
poppen-nw.net pinsdaddy.com
popunder.net popmarker.com
pornhive.org pornofiljmi.com
pornohd1080.online pornosexrolik.com
pornohub.me pozdravleniya-c.ru
pornoklad.net privatamateure.com
pornolook.net prod2016.com
pornosee.info
pornosok.ru
posible.net
pregnant.guru
printingpeach.com
privatefx-in.ru
profitwithalex.info
profolan.pl
projectforte.ru projectforte.ru
prombudpostach.com.ua promoheads.com
prostitutki-astana.org prostitutki-yaroslavlya.org
pssucai.info puzzleweb.ru
pyrodesigns.com.au raavidesigns.com
quickloanbank.com rank-checker.online
qwarckoine.com rankscanner.com
rabot.host.sk ranksignals.com
rankingchart.de realitykings.com
ranksonic.org redirect.trafficreceiver.club
razvratnoe.org reeyanaturopathy.com
redhotfreebies.co.uk refudiatethissarah.info
reining.lovasszovetseg.hu reimageplus.com
reklama1.ru rentehno.ru
relayblog.com retailwith.com
remontvsamare.su revistaindustria.com
renhacklids.tk robotixix.com
rennlist.com rocketchange.ru
replica-watch.ru rosbalt.com.ua
restaurantlescampi.com
ring4rhino.com
ritlweb.com
rockprogblog.com
rubbed.us
rumamba.com rumamba.com
sadaholding.com russkie-gorki.ru
samoiedo.it saitevpatorii.com
sbricur.com salutmontreal.com
scanner-george.top sashagreyblog.ga
seansonline24.pl sbetodiodnye-lampy.ru
search-error.com scanner-jane.top
search.1and1.com search.pch.com
searchinterneat-a.akamaihd.net securityallianceservices.com
semxiu.com semrush.com
senger.atspace.co.uk seo-2-0.com
seo-platform.com seocdvig.ru
seolab.top seofirmreviewsus.info
sex-tracker.com seoholding.com
sexpornotales.com serpstat.com
sexspornotub.com seryeznie-znakomstva.ru
sextracker.de sexblog.pw
sexxdate.net sexs-foto.com
shillyourcoins.com sexsaoy.com
shodanhq.com sfd-chess.ru
sibtest.ru shitting.pro
sigmund-freud.co.uk shop.xz618.com
simply.net shoplvlv.us
site3.free-share-buttons.com shopwme.ru
site5.com sientalyric.co
small-games.biz sildenafil-tadalafil.info
sobecjvuwa.com.ru sispe.com.br
socas.pluto.ro skyway24.ru
sofit-dmd.ru sloopyjoes.com
softonicads.com smartshoppymac.com
solicita.info sneakyboy.com
songplanet.ru snip.to
soserfis.com snjack.info
spacash.com social-search.me
spamnuker.com socialbookmarksubmission.org
spravka130.ru sovetogorod.ru
spywarespy.com sovinsteel.ru
stairliftstrue.com spammen.de
stathat.com special-porn.com
statistici.ro sportbetfair.com
stats-collector.org sprttrack.com
sstroy44.ru
stair.registrydomainservices.com
startwp.org
starwars.wikia.com
stats-public.grammarly.io
stauga.altervista.org
stevemonsen.com
stop-gepatit.te.ua stop-gepatit.te.ua
stopnarco.ru
strangeduckfilms.com strangeduckfilms.com
streetfooduncovered.com sundrugstore.com
strfls.com superkanpo.com
stroimajor.ru superstats.com
studiofmp.com svbur.ru
suggest-keywords.com svensk-poesi.com
supernew.org takethatad.com
surcentro.com taranerymagesswa.blogspot.com
tacbibirfa.tk techart24.com
tarad.com teenfuck.tv
tastyfoodideas.com telesvoboda.ru
tattoo33.ru testbotprocessor44.com
tdsing.ru tfxiq.com
tech4master.com thaisamkok.com
telefonsexi.com thaismartloan.com
thebestphotos.eu theguardlan.com
thewebsitetemplate.info thenews-today.info
ticketsys.inetwd.com thepornsex.org
timer4web.com thewomenlife.com
timetorelax.biz tiandeural.ru
tizanidineorflexeril.blogspot.com tizanidineingredients.blogspot.com
tizanidinepills.blogspot.com tmtrck.com
tizanidinevssoma.blogspot.com tomatis.gospartner.com
tomck.com tongkatmadura.info
topclickguru.com topmira.com
torrents.cd tourismvictoria.com
touchmods.fr tozup.com
tracksz.co tracfone.com
trahic.ru tracklead.net
trkur.com trafficgenius.xyz
turbo-suslik.org traffictrade.life
traffique.net
trahvid.com
trailer.cinemaflix.website
truebeauty.cc
tversvet.ru
tvoystartup.ru
twittruth.com
tyumen.xrus.org tyumen.xrus.org
ukrtvir.com.ua udsgame.online
underthesite.com ugogo.info
unmaroll.ya.ru umityangin.net
userequip.com undergroundcityphoto.com
v24s.net uptimechecker.com
vacuumscleaner.com vacuumscleaner.com
varikoz24.com validccseller.com
venta-prom.ru valueclick.com
vepad.com vepad.com
viagengrarx.com versaut.xxx-cam.webcam
viagra-soft.ru video-chat.in
viagroid.ru videochat.tv.br
video-chat.cn vietimgy.pw
villakohlanta.nu vigrx-original.ru
vitanail.ru viveresaniesnelli.it
voloomoney.com vizitki.net
volume-pills.biz vsesubwaysurfers.com
voluumtracker1.com vzglyadriv.kg
voluumtrk.com vzubah.com
vpnmouse.com warningwar.ru
vzube.com website-analytics.online
w3data.co webstatsdomain.org
warning.or.kr weightatraining.com
webadvance.club welck.octopis.com
website-speed-check.site westum.se
weekes.biz.tc whatsappdownload10.com
wejdz-tu.pl wholinkstome.com
westermarkanjou.se wiosenny-bon-1500.pl
while.cheapwebsitehoster.com
whipme.yopoint.in
white-truck.life
whiteelephantwellington.com
wholesalejerseyscheapjerseys.us.com
wildworld.site
wineration.com
wnoz.de wnoz.de
wordkeyhelper.com womensterritory.ru
world-mmo.com www.get-free-traffic-now.com
worldhistory.biz wygraj-teraz.com
worldoffiles.ru xblognetwork.com
worst-sites.online xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
wowcasinoonline.ooo xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
wstroika.ru xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
ww1943.ru xn--80aikhbrhr.net
www.pinnacle-bets.com xn--h1ahbi.com.ua
x-mix.info xnxx699.com
x-true.info xtraffic.plus
xboxster.ru
xlovecam.com
xmronta.com
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn----dtbndd4ae7eub.top
xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn--80aafb2a.xn--p1ai
xn--80ak6aa92e.com
xn--80atua3d.xn--p1ai
xn--90acenikpebbdd4f6d.xn--p1ai
xn--b1addnj3cah.xn--p1ai
xn--e1afanlbnfckd7c3d.xn--p1ai
xpresscare.ru
xxart.ru
xxx-cam.webcam xxx-cam.webcam
xxxhdvideo.site yadro.ru
xxxtube69.com yaminecraft.ru
yeartwit.com yeartwit.com
yoluxuryevents.com yellowproxy.net
yourmovies.pl ynymnwbm.bloger.index.hr
yugk.net yogamatsexpert.com
yuweng.info youbloodyripper.com
zakonobosago.ru youdao.com
zarabotat-na-sajte.ru yourothersite.com
zeroredirect5.com yousense.info
zeroredirect9.com youtubologia.it
zigarettenonlinekaufen1.bloog.pl ytmnd.com
znakomstva-piter78.ru zaimhelp.ru
zoosexart.com zakazvzloma.com
zryydi.com zhacker.net
zvezdagedon.ru zigarettenonl.canalblog.com
zoolubimets.ru
zoominfo.com
zvuker.net

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
08800.top 100dollars-seo.com
23kw.ru 123kuma.com
2itech.ru 2drittel.de
3w24.com 4kpics.rocks
a1security.com.ua 6hopping.com
aa8780bb28a1de4eb5bff33c28a218a930.com 76brighton.co.uk
acheterviagrafr24.com acheterviagrafr24.com
acmebtn.ml addray.pro
addblueoff.com.ua adultgalls.com
admitad.com
adprotect.net
adultfriendfinder.com
adultmeetonline.info adultmeetonline.info
afora.ru adultwebhosting.info
animali.deagostinipassion.it alienwheel.es
animalrank.com allboard.xobor.de
animaltoplist.com allesohnegirls.net
apartment.ru allnews24.in
apelsinnik.website amazon-adsystem.com
artpicso.com an-donut.com
atleticpharm.org animal-drawings.com
backlinkwatch.com avitocars.ru
bannertesting.com aviva-limoux.com
bazaronline24.ru avon-ukraine.com
belinvestdom.by avto-oligarh.ru
bigpictures.club ayeartoforget.com
bioca.org azarttoto.com
bioinnovate.co azazu.ru
blogsfunky672.weebly.com azbukafree.com
bluerobot.info b00kmarks.com
businesxxl.com baditri.com
buyk.host.sk bang-hotties.com
bvlgariwallet.xyz best-games.today
callejondelpozo.es bestdraws.com
candy-glam-hp.com bestwaystogetpaid.us
cars-modification.net bitcoinremote.com
cartierreplica.win blog.remote-computer.de
cdncash.org brus-vsem.ru
cete.ru camdolls.com
cgi2.nintendo.co.jp casinorewards.info
charmstroy.info cayado.snn.gr
cheapestjerseys-wholesale.com cenokos.ru
citysecurity.nu chastnoeporno.com
club-samodelkin.ru chocolateslim-u-srbiji.com
codq.info
comissionka.net
compliance-elena.xyz compliance-elena.xyz
compliance-willy.top compliance-jack.top
coolbar.pro compliance-josh.top
coolingoods.com coolyarddecorations.com
country-chic.ru copenergo.ru
customcollegeessays.net cordstrap.cc
dinosaurus.site cranly.net
divku.ru customchocolate.business-for-home.com
djstools.com damedingel.ya.ru
docsportal.net derevesendeco.com
dojki-hd.com design-lands.ru
dollartree.info dev78.dioniqlabb.se
downvids.net doska-vsem.ru
dosugrostov.site
dotnetdotcom.org
drugs-no-rx.info drugs-no-rx.info
duplicashapp.com dscaas.website
earl-brown.info ekaterinburg.xrus.org
editors.choice6912650.hulfingtonpost.com ekn-art.se
ekto.ee electricwheelchairsarea.com
elektrozigaretten2.yn.lt electronicadirect.com
elvel.com.ua elektrischezigarette2.onsugar.com
eng-lyrics.com elektrischezigaretten1.blogse.nl
ertelecom.ru elektrischezigaretten2.beeplog.com
escortslet.net emerson-rus.ru
exchanges-bet.com eropho.net
exdocsfiles.com erotiktreff24.info
extremal-blog.com etotupo.ru
ezigarettekaufen2.yolasite.com everflora.ru
fakehandbags.xyz exhibitionplus.eu
faz99.com exoneration-project.us
fickblock18.com exportshop.us
financetip.eu extstat.com
findpik.com eyemagination.com
forex-procto.ru familyholiday.ml
forms-mtm.ru fanoboi.com
forsex.info forum69.info
foto-telok.net fotoxxxru.com
game300.ru free-today.com
gameprimary.com freewebs.com
gamezblox.com fugarif.ga
glavtral.ru gallerily.com
golaya.pw gamesprite.me
gosmeb.ru gate5.co.za
gratis-sexkontakte.com gdcentre.ru
gribokstop.com gilsonchiro.xyz
hchha.com girlspicsa.com
hexpilot.com glass-msk.ru
import-sales.com globatur.ru
inspiring-desperate.tk googlepositions.com
investingclub.ru gsou.cf
irunfar.com hahashka.ru
isoveti.ru hamilton.ca
itbc.kiev.ua headpharmacy.com
ivanstroi.ru hentai-manga.porn
jagg.info hermesbirkinhandbagoutlets.com
jovencitas.gratis hmmm.cz
home-task.com
hothot.ru
hully.altervista.org
ideoworld.org
ikearugs.xyz
ilikevitaly.com
imagui.eu
immobiliaremassaro.com
inclk.com
infowarcraft.ru
intelhdgraphicsgtdrive6w.metroblog.com
intellego.info
iphantom.com
iqoption-bin.com
iqs.biz.ua
ispaniya-costa-blanca.ru
itag.pw
iusstf.org
ivoiretechnocom.ci
james13prix.info
javcoast.com
jillepille.com
joyceblog.top
juliaworld.net
jyvopys.com jyvopys.com
kalandranis.gr kandidos.com
kazrent.com kevblog.top
kerei.ru keywordsuggests.com
kfon.eu kinofak.net
kino-rating.ru kongruan.com
kmd-pto.ru koversite.info
kosova.de lacapilla.info
lacloop.info lennatin.info
lancheck.net locksmith.jp
law-check-eight.xyz loginduepunti.it
le-clos-des-alouettes.com lubetube.com
loftdigital.eu malls.com
lomb.co manipulyator-peterburg.ru
lonely-mature.com
lumb.co
magazintiande.ru
magento-crew.net
marblestyle.ru
mebel-alait.ru mebel-alait.ru
medmajor.ru
medosmotr-ufa.ru
meetingrainstorm.bid meetingrainstorm.bid
metalonly.info memberty.com
mhtr.be meteocast.net
minecraft-rus.org mikozstop.com
mirobuvi.com.ua mir-limuzinov.ru
mlvc4zzw.space moi-glazki.ru
moinozhki.com mole.pluto.ro
monsterdivx.com more-letom.ru
monsterdivx.tv most.gov.iq
mriyadh.com moyakuhnia.ru
muycerdas.xxx mural.co
negral.pluto.ro na15.ru
nero-us.com netvouz.com
nfljerseysforsalewholesaler.com nikkiewart.ru
ningessaybe.me nl.netlog.com
npoet.ru nyfinance.ml
offer.wpsecurity.website ohmyrings.com
okayimage.com okayimage.com
ooo-olni.ru omsk.xrus.org
optom-deshevo.ru oynat.info
oxotl.com parajumpersstore.online
panouri-solare-acoperis.com parfusale.se
paradontozanet.ru partybunny.ru
phpdevops.com parvezmia.xyz
pinup-casino1.ru pateaswing.com
pio.polytopesexempt.com pb-dv.ru
pl.aasoldes.fr photokitchendesign.com
platezhka.net php-market.ru
play.leadzupc.com phuketscreen.com
plusnetwork.com picturesboss.com
poligon.com piratecams.com
polska-poezja.com pl-top.pl
polytopesexempt.com pojdelo.weebly.com
porn555.com porn-w.org
porno-raskazy.ru pornhubforum.tk
pornoxxx.com.mx pornobrazzers.biz
portside.cc pornotubexxx.name
pricheskaonline.ru pornzone.tv
princeadvantagesales.com powenlite24.ru
progonrumarket.ru producm.ru
progress-upakovka.ru proekt-gaz.ru
projefrio.com.br
promover.org
qpypcx.com qpypcx.com
quebec-bin.com radiogambling.com
randalljhoward.com razleton.com
rangoman.date razvratnoe.org
rebelmouse.com realresultslist.com
reklama-i-rabota.ru remmling.de
relax.ru remont-comp-pomosh.ru
remontgruzovik.ru remorcicomerciale.ro
rique.host.sk remybutler.fr
roofers.org.uk renecaovillasale.online
ruscopybook.com ringtonepartner.com
rutor.group ru-mediaget.ru
saecsa.co sbt-aqua.ru
sarm.tk scanner-mark.top
scanner-donald.top securityallianceservices.com
schalke04fc.info sel-hoz.com
se-welding.ru sextracker.be
search.pch.com sexyteens.hol.es
seasaltwithfood.com slim.sellany.ru
serptehnika.ru snyatie-lomki-v-stacionare.ru
sex-znakomstva.online sockshare.net
sexflirtbook.com soda.media
sextracker.com soheavyblog.com
shillyourcoins.com
shopwme.ru
simpoed.ufop.br
sispe.com.br
sitopreferito.it
skachat-besplatno-obrazcy.ru
sky-mine.ru
snsdeainavi.info
social-s-iii.xyz
solnplast.ru
spywarenuker.com
stairliftsarea.com stairliftsarea.com
tandvardshuset.net standardchartered-forex.com
tarad.com steebook.com
td-l-market.ru strongholdsb.ru
thefarmergame.com stroymonolit.su
theguardlan.com tackletarts.co
thin.me.pn taihouse.ru
tizanidineandgabapentin.blogspot.com tanieaukcje.com.pl
tizanidineforopiatewithdrawal.blogspot.com teletype.in
tizanidinestreetprice.blogspot.com theporndude.com
tn811.us tizanidineandcipro.blogspot.com
top-deal.com.pl tizanidineinfo.blogspot.com
track.deriv.com tizanidinemusclerelaxant.blogspot.com
trafficfactory.biz tizanidineshowupondrugtest.blogspot.com
tuberkulezanet.ru tootoo.to
totu.info
trafaret74.ru
tubeline.biz
turbodsp.com turbodsp.com
turist-strani.ru uamtrk.com
ua-company.ru ukrtvory.in.ua
ublaze.ru unimodemhalfduplefw.pen.io
unblocksit.es vacances-voyages.info
unpredictable.ga viberdownload10.com
uprour.com videochat.tv.br
uptimechecker.com vip-dom.in
venerologiya.com virtualbb.com
ventelnos.com vkonche.com
versaut.xxx-cam.webcam vykupavto-krasnodar.ru
video-woman.com
videogamesecrets.com
vostoktrade.info
vriel.batcave.net
vrnelectro.ru
vsdshnik.com
vse-pesni.com
wdfdocando.com wdfdocando.com
web-analytics.date webix.biz
website-stealer.nufaq.com webradiology.ru
worldoffiles.ru weburok.com
worldtraveler.world weclipart.com
www.labves.ru westen-v.life
xgftnlrt.bloger.index.hr whatsupinfoley.com
xmnb.net whengirlsgowild.com
xn----9sbebi2bvzr7h.xn--p1ai white-truck.life
xn----ctbigni3aj4h.xn--p1ai worldinternetauthority.com
xn--d1acah0c.xn--p1ai x-porno.video
xxxdatinglocal.us xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
yellowstonevisitortours.com xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
yginekologa.com xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
youraticles.pl yougotanewdomain.com
zaidia.xhost.ro yousense.info
zed21.net zeroredirect.com
zhorapankratov7.blogspot.com zetgie.com.pl
zx6.ru

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.57 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.03.2971 ### Version: V4.2022.03.2972
### Updated: Thu Mar 10 22:01:39 UTC 2022 ### Updated: Fri Mar 11 10:03:33 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.03.2971 ### Version: V4.2022.03.2972
### Updated: Thu Mar 10 22:01:39 UTC 2022 ### Updated: Fri Mar 11 10:03:33 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##