vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-07 21:20:02 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.12.3519 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-12-15 22:07:30 +00:00
parent 3a2eec4ff4
commit bebc0cfd19
47 changed files with 87128 additions and 87118 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3518 #### Version: V4.2022.12.3519
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3518 #### Version: V4.2022.12.3519
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.12.3518 #### Version: V4.2022.12.3519
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10752
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.12.3518 #### Version: V4.2022.12.3519
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10756
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10756
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10756
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10756
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 1; 10.1.0.242 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 0; 10.1.0.242 0;
127.0.0.1 0; 127.0.0.1 0;

10756
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 1; 10.1.0.242 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 0; 10.1.0.242 0;
127.0.0.1 0; 127.0.0.1 0;

10756
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 1; 10.1.0.242 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 0; 10.1.0.242 0;
127.0.0.1 0; 127.0.0.1 0;

10756
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3518 3519

10756
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10756
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.226.126.48 0; 20.245.230.164 0;

10756
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 1; 10.1.0.242 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 0; 10.1.0.242 0;
127.0.0.1 0; 127.0.0.1 0;

10756
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 1; 10.1.0.242 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 0; 10.1.0.242 0;
127.0.0.1 0; 127.0.0.1 0;

10756
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 1; 10.1.0.242 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 0; 10.1.0.242 0;
127.0.0.1 0; 127.0.0.1 0;

10756
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.226.126.48 0; 20.245.230.164 0;

10756
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.226.126.48 0; 20.245.230.164 0;

10756
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 1; 10.1.0.242 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
DigitalPebble 80legs
Getintent ADmantX
Leap BackStreet
Magnet DTS Agent
Nameprotect EyeNetIE
SEOkicks-Robot LinqiaScrapeBot
SEOstats Pi-Monster
Seomoz SiteSnagger
Siphon Wonderbot
Sitevigil xpymep1.exe

290
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404checker 404checker
80legs
ALittle Client ALittle Client
ASPSeek Abonti
Aboundexbot Aboundex
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Aipbot
Alexibot Alexibot
Alligator Ankit
AlphaBot Atomseobot
Anthill AwarioRssBot
Apexoo BBBike
Asterias
Attach
AwarioSmartBot
BDCbot BDCbot
Backlink-Ceck BDFetch
BacklinkCrawler BackStreet
BackWeb
Badass Badass
Bandit Bandit
Barkrowler Barkrowler
BatchFTP BatchFTP
BetaBot
Bigfoot Bigfoot
Blackboard Black Hole
BlackWidow
Blow Blow
Bolt Bolt
Buck
BuiltBotTough BuiltBotTough
BunnySlippers BuzzSumo
CATExplorador CCBot
CODE87 CODE87
Cegbfeieh
CensysInspect
CherryPicker CherryPicker
Chlooe ChinaClaw
Citoid
Cocolyzebot Cocolyzebot
Cogentbot Cogentbot
Collector Copier
CopyRightCheck Copyscape
Craftbot Craftbot
Crawling at Home Project
CrazyWebCrawler
CrunchBot CrunchBot
Curious Curious
Custo DIIbot
CyotekWebCopy DTS Agent
DBLBot Demon
DSearch
DataCha0s
Devil Devil
Digincore Dirbuster
DigitalPebble
Disco Disco
Discobot
DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Drip EasyDL
Ebingbong Ebingbong
EirGrabber Exabot
EroCrawler Express WebPictures
ExtLinksBot ExtLinksBot
ExtractorPro Extractor
Extreme Picture Finder
EyeNetIE EyeNetIE
FDM
FHscan FHscan
FemtosearchBot FemtosearchBot
Firefox/7.0 Foobot
Freeuploader
Fuzz Fuzz
FyberSpider FyberSpider
Fyrebot
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
Genieo Genieo
GetWeb GetRight
Getintent Go!Zilla
Gigabot
GoZilla
Grabber Grabber
Grafula
GridBot GridBot
HTTP::Lite HTTP::Lite
Haansoft Haansoft
Harvest
Havij Havij
HonoluluBot Hloader
Humanlinks
HybridBot
IDBTE4M
IRLbot IRLbot
Iblog
IlseBot IlseBot
Image Fetch Indy Library
InfoNaviRobot
InfoTekies InfoTekies
Internet Ninja Internet Ninja
Iria Iria
JOC Web Spider IstellaBot
JamesBOT JamesBOT
JennyBot
JetCar
Jetty
JikeSpider JikeSpider
Joomla Jorgee
Kenjin Spider Jyxobot
Keybot Translation-Search-Machine
Keyword Density Keyword Density
LNSpiderguy Kozmosbot
Lanshanbot Leap
Larbin LexiBot
LeechGet LieBaoFast
LibWeb LinkextractorPro
Libwhisker
Lightspeedsystems
LinkpadBot LinkpadBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lmspider Lipperhey
MFC_Tear_Sample Ltx71
MIIxpc
MJ12bot
MSFrontPage MSFrontPage
Mag-Net MTRobot
Majestic SEO Majestic SEO
MarkMonitor MarkWatch
Mass Downloader
Masscan Masscan
Mata Hari Mata Hari
MeanPath Bot Mb2345Browser
MegaIndex.ru MegaIndex.ru
Metauri
Microsoft Data Access Microsoft Data Access
Mister PiX Minefield
Moblie Safari
Mojeek
Mojolicious
MolokaiBot MolokaiBot
Musobot Morfeus Fucking Scanner
Mozlila
NICErsPRO
NPbot NPbot
Nameprotect
Navroad Navroad
NetMechanic NetMechanic
NetZIP NetZIP
Netvibes Netcraft
Nibbler Nettrack
Niki-bot
Nikto Nikto
NimbleCrawler NimbleCrawler
Ninja
Octopus Octopus
Offline Navigator Offline Navigator
OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openvas Openfind
OrangeSpider
OutfoxBot
PHPCrawl PHPCrawl
PageGrabber POE-Component-Client-HTTP
Page Analyzer
Panscient Panscient
Papa Foto
Pavuk
PeoplePal
Pi-Monster
Picsearch
PictureFinder PictureFinder
Pixray Pimonster
PleaseCrawl PleaseCrawl
Pockey Probethenet
ProPowerBot PyCurl
ProWebWalker RSSingBot
Pu_iN
Pump
QueryN Metasearch
Quick-Crawler
RankActiveLinkBot RankActiveLinkBot
RankingBot RankFlex
RankingBot2 Re-re
Reaper ReGet
Recorder Recorder
Ripper Ripper
RocketCrawler RocketCrawler
SEOkicks SBIder
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler
SMTBot SMTBot
Scanbot
Seekport Seekport
Semrush Semrush
SentiBot SeobilityBot
SeoSiteCheckup SiteCheckerBotCrawler
Site Sucker SiteExplorer
SiteLockSpider SiteLockSpider
Sitebeam Sitevigil
Siteimprove SlySearch
SmartDownload SmartDownload
SocialRankIOBot SocialRankIOBot
Sociscraper
Sogou web spider
Sosospider Sosospider
Spammen Spammen
SpankBot
Spbot
SputnikBot SputnikBot
Sqlmap Sqlmap
Sqlworm
Sqworm Sqworm
Steeler Sucker
Sucuri Sucuri
SuperBot SuperBot
SuperHTTP Surfbot
Suzuran Suzuran
T0PHackTeam Szukacz
Telesoft Thumbor
Telesphorep TightTwatBot
Titan Titan
Toata
Toweyabot Toweyabot
Trendiction Trendictionbot
Turingos Turingos
TurnitinBot TwengaBot
URLy.Warning Twice
VB Project Typhoeus
URLy Warning
UnisterBot
Vacuum Vacuum
Vagabondo VelenPublicWebCrawler
VeriCiteCrawler
Virusdie Virusdie
Voltron Voil
WPScan WASALive-Bot
WWWOFFLE Wallpapers
Wallpapers/3.0 Web Enhancer
WallpapersHD Web Pix
Web Sauger Web Sauger
WebBandit WebCopier
WebSucker WebFetch
Webshag WebLeacher
WebsiteQuester Whatweb
Whack WinHTTrack
Whacker
Woobot
Wprecon
Xaldon WebSpider Xaldon WebSpider
Xenu
YoudaoBot YoudaoBot
Zade
Zauba
Zermelo
Zitebot
ZmEu ZmEu
ZoomBot
ZoominfoBot ZoominfoBot
archive.org_bot ZyBorg
backlink-check arquivo-web-crawler
coccocbot cah.io.community
check1.exe
com.plumanalytics com.plumanalytics
dataforseo.com crawl.sogou.com
demandbase-bot domainsproject.org
evc-batch
facebookscraper facebookscraper
gopher ips-agent
heritrix iubenda-radar
internetVista monitor linkdexbot
oBot lwp-request
netEstate NE Crawler
pcBrowser
plumanalytics
polaris version
s1z.ru
satoristudio.net
scalaj-http
scan.lol scan.lol
seobility seobility
seoscanners seostar
sitechecker.pro
siteripz
sogouspider sogouspider
spyfu
sysscan sysscan
tAkeOut
trendiction.de trendiction.de
webgains-bot x09Mozilla
webpros.com zauba.io
zgrab

322
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404enemy 404enemy
ADmantX ADmantX
ALittle Client
ASPSeek
Aboundex Aboundex
Acunetix Aboundexbot
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
Alexibot
Anarchie Anarchie
Anarchy
Anarchy99
Ankit Ankit
Aspiegel Anthill
BackDoorBot Apexoo
BackStreet Asterias
Backlink-Ceck AwarioSmartBot
BacklinkCrawler BBBike
Bandit Bandit
BetaBot Battleztar Bazinga
Bigfoot
Black Hole
BlackWidow BlackWidow
Blackboard BlowFish
Blow Boardreader
Bolt Brandprotect
Buck Buck
Bullseye BuiltBotTough
BuiltWith
BunnySlippers BunnySlippers
CCBot
CODE87 CODE87
CazoodleBot CSHttp
Cegbfeieh Calculon
CensysInspect CheeseBot
CherryPicker Chlooe
ChinaClaw Cloud mapping
Cliqzbot Cocolyzebot
Collector CopyRightCheck
Copier
Copyscape
Cosmos Cosmos
Crawling at Home Project
Crescent Crescent
CrunchBot
Curious Curious
Custo DSearch
DBLBot
DataCha0s DataCha0s
DatabaseDriverMysqli Devil
Demon
DigitalPebble
Dirbuster Dirbuster
DittoSpyder Discoverybot
DnBCrawler-Analytics Dispatch
DomainStatsBot DomainAppender
EirGrabber DomainCrawler
ExtLinksBot Domains Project
Extractor Dotbot
EyeNetIE Download Wonder
Drip
ECCP/1.0
EasyDL
Ecxi
EroCrawler
Ezooms Ezooms
FHscan FDM
Firefox/7.0 Fimap
Fyrebot Flunky
GalaxyBot FyberSpider
Genieo G-i-g-a-b-o-t
GetWeb Gigabot
Go!Zilla Go-Ahead-Got-It
Grabber Grafula
GrapeFX GrapeFX
HEADMasterSEO GridBot
HTMLparser HTMLparser
Haansoft HonoluluBot
HybridBot
IRLbot IRLbot
IlseBot Iblog
IndeedBot Image Fetch
Image Sucker
InfoTekies InfoTekies
Internet Ninja
InternetSeer
Iria
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT JennyBot
JetCar Jetty
JikeSpider JikeSpider
Joomla Kinza
JustView
Kenjin Spider
Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple LWP::Simple
Larbin Linkbot
LibWeb
LieBaoFast
Lightspeedsystems
Likse
LinkWalker
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey Spider LinqiaScrapeBot
Litemage_walker
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc MIIxpc
MQQBrowser
MSFrontPage MSFrontPage
MSIECrawler
Majestic SEO Majestic SEO
Majestic-SEO
MarkWatch MarkWatch
Mb2345Browser Masscan
Mata Hari
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Minefield Microsoft Data Access
Mister PiX Mister PiX
Mojeek
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Msrabot NICErsPRO
Musobot Name Intelligence
NPbot NearSite
Navroad Needle
Net Vampire
NetAnts
NetMechanic NetMechanic
NetSpider NetZIP
Netcraft Niki-bot
Nimbostratus
Ninja Ninja
Nutch Nmap
Octopus Not
Offline Explorer OnCrawl
Offline Navigator Openfind
OrangeBot Openvas
OutfoxBot PECL::HTTP
PHPCrawl PHPCrawl
Page Analyzer Page Analyzer
PageThing.com PageAnalyzer
Pandalytics Pandalytics
Panscient Panscient
Papa Foto Papa Foto
Pavuk
Pi-Monster Pi-Monster
Picsearch Picscout
PictureFinder PictureFinder
Pixray Piepmatz
Pockey Pimonster
PleaseCrawl
ProPowerBot
ProWebWalker
Probethenet Probethenet
Pump Pump
PxBroker RSSingBot
PyCurl
Quick-Crawler
RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot2
RealDownload RankurBot
Reaper RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
Rogerbot Rogerbot
SEOkicks-Robot SBIder
SEOlyticsCrawler SEOkicks
SEOstats SMTBot
ScanAlert SalesIntelligent
Scanbot
ScreenerBot
ScrepyBot ScrepyBot
SearchmetricsBot Searchestate
Seekport SemanticJuice
SentiBot SentiBot
Shodan SeoSiteCheckup
SeobilityBot
Siphon
Site Sucker Site Sucker
SiteExplorer SiteCheckerBotCrawler
SiteSnagger Sitevigil
SiteSucker
Siteimprove
SlySearch
Snake
Snapbot
Sogou web spider
SpaceBison SpaceBison
SpankBot Spanner
Spbot Spbot
Sqlmap Steeler
Sqworm Stripper
SuperBot Surfbot
Swiftbot Szukacz
T0PHackTeam
T8Abot
Teleport Teleport
TeleportPro TeleportPro
Telesoft Telesoft
TightTwatBot Telesphoreo
Tracemyfile Telesphorep
True_Robot Titan
Turingos Toweyabot
Trendiction
TurnitinBot TurnitinBot
Typhoeus Twice
URLy Warning URLy Warning
V-BOT URLy.Warning
UnisterBot
VB Project VB Project
VCI VCI
Vacuum VeriCiteCrawler
Virusdie Virusdie
Voil VoidEYE
Voltron WASALive-Bot
WEBDAV
WISENutbot WISENutbot
WWW-Mechanize WWW::Mechanize
WallpapersHD Wallpapers
Web Auto WeSEE
Web Collage Web Pix
Web Fuck
Web Sucker Web Sucker
WebAuto
WebCopier WebCopier
WebEnhancer WebEnhancer
WebFuck WebGo IS
WebReaper WebImageCollector
WebWhacker WebLeacher
Webshag WebSauger
WebmasterWorldForumBot
Website Quester Website Quester
Whack Webster
Whatweb
Who.is Bot Who.is Bot
WinHTTrack
WiseGuys Robot
Wonderbot Wonderbot
Woobot Woobot
Wotbox Wotbox
Wprecon
YoudaoBot YoudaoBot
Zade Zade
Zauba Zauba
Zermelo Zeus
Zitebot
ZmEu ZmEu
ZoomBot ZoomBot
ZoominfoBot ZumBot
adscanner
arquivo-web-crawler
arquivo.pt
backlink-check backlink-check
cah.io.community cah.io.community
check1.exe coccocbot
clark-crawler com.plumanalytics
crawler4j crawl.sogou.com
demandbase-bot crawler.feedback
domainsproject.org dataforseo.com
facebookscraper gopher
heritrix heritrix
lwp-trivial isitwp.com
magpie-crawler lwp-request
mediawords
oBot oBot
pcBrowser page scorer
polaris version probe-image-size
satoristudio.net ripz
s1z.ru
scalaj-http scalaj-http
scan.lol
seobility seobility
seocompany.store seocompany.store
seoscanners
seostar
sexsearcher
sitechecker.pro sitechecker.pro
siteripz siteripz
sysscan sogouspider
tAkeOut ubermetrics-technologies.com
trendiction.de
voyagerx.com voyagerx.com
webgains-bot webprosbot
webmeup-crawler xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
1xbet4you.com bagsonsale.online
balkanfarma.ru bimatoprost-careprost.com
deplim.com check-host.net
heygidday.biz guge.io
krasnodar.ru images-graphics-pics.com
mastershef.club pdns.cz
mountainstream.ms pxhdwsm.com
nippon-bearings.ru validccseller.com
percin.biz.ly viel.su
takeflyte.com vucms.com

952
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
01apple.com
03e.info 03e.info
11pikachu.ru 123any.com
13tabs.com 178evakuator178.ru
1qingdao.com 2015god.org
2020iscoming.info 214jaluwobapef.cf
3-letter-domains.net 2girls1cup-free.com
4webmasters.org 3weekdiet.com
66cpwgln.space 4istoshop.com
4kplayer.pl
4senses.co
5ws.dating-app.ru
76brighton.co.uk
7minuteworkout.com 7minuteworkout.com
991mostfm.co.id aanapa.ru
aarbur.com acmebtn.ml
acunetix-referrer.com addblueoff.com.ua
addray.pro adelly.bg
adexprts.com adexprts.com
ads.tremorhub.com adimmix.com
adservme.com advancedmassagebysara.com
adult3dgames.com
adviceforum.info adviceforum.info
akvamaster.dp.ua ahrefs.com
alice.it aksonural.ru
allbizne.co.ua aktivator-windows10.blogspot.com
alldownload.pw alert-fdm.xyz
amoi.tn allblogroll.com
anaksma.info alldubai.biz
anniemation.com alltheviews.com
apimossnetinfo-a.akamaihd.net allwomen.info
apimountainbikei-a.akamaihd.net alta-realestate.com
aramaicmedia.org alyeskaresort.com
anal-acrobats.com
apishops.ru
apparel-offer.com
appfixing.space
arcadeplayhouse.com
arcteryxstore.online
arquivo.pt arquivo.pt
artdeko.info asscenihotosidea.blogspot.com
auto-zapchasti.org atofilms.com
audiobangout.com
audiofree.ru
auto.rusvile.lt auto.rusvile.lt
autobrennero.it autorn.ru
autodo.info avek.ru
avto-oligarh.ru azartclub.org
avtolombard-krasnodar.com bad-stars.net
axbocz.net balla-boo.se
azbukadiets.ru bamo.xsl.pt
azlex.uz
backlinkwatch.com
balans.shahterworld.org
bang-hotties.com bang-hotties.com
bank.uz bannertesting.com
best-seo-offer.com bedandbreakfast.com
bestbrainprod.win beget.tech
bestcoin.cc bestmarriages.com
bestcurs.org bestprofits-there.com
bestdraws.com bg6s0.com
bidbuy.co.kr bioinnovate.co
bigames.online
bitcoinremote.com bitcoinremote.com
bjetjt.com bitnote.co
blackplanet.com biznesluxe.ru
bloggers.nl bloggen.be
bnt-team.com bluejays-jerseys.us
boanonihaca.tk bluesalt.co
bolegapakistan.com bongiornos.info
bolezniorganov.ru bonky.biz
bolitgorlo.net boom-celebs.com
bombla.org bosman.pluto.ro
bookmark4you.com.biz brainxs.us
brianhenry.co breget74.ru
buybest1.biz brk-rti.ru
buyviagraa.blogspot.com brus.city
callawaygolfoutlet.online builtwith.com
candycrushshop.com buketeg.xyz
candytiner.com bumble.cheapwebsitehoster.com
capsderfudd.tk bumskontakte.org
cartierlovestore.xyz buyantiviralwp.com
casite-513049.cloudaccess.net bytimedance.ru
chastnoeporno.com cah.io.community
cheapbelstaff.online californianews.cf
cherrypointplace.ca canrole.com
choice-credit.ru carsplate.com
chorus.terakeet.com cartierlovebracelet.xyz
civilwartheater.com cartierlovebraceletreplica.xyz
cleanallspyware.com casablancamanor.co.za
clickbank.net cashonads.com
clothesforcash.com casinos4dummies.co.uk
clubfashionista.com caulderwoodkennels.com
cmd.kz ccbill.com
colehaanoutlet.store censys.io
compliance-alexa.top chatseo.com
compliance-andrew.top cheapbarbour.online
consorzioilmosaico.org chelnytruck.ru
contentsexpress.com chinaelitecheapjerseys.com
cookie-law-enforcement-hh.xyz chiptuninger.com
coop-gamers.ru citetick.com
copyrightclaims.org clickpapa.com
corta.co clodo.ru
cosmeticswomens-womensports.rhcloud.com compliance-ivan.top
costablue.xyz compliance-jane.top
cougfan.info cosmetic.donna7753191.ru
counter.bloke.com counter.yadro.ru
counterbot.com counterbot.com
cowlmash.com craftburg.ru
crackzplanet.com creditnation.ru
crynet.cc crest-poloski.ru
cupday.com customchocolate.business-for-home.com
cymbaltaandtizanidine.blogspot.com cymbaltaandtizanidine.blogspot.com
dailyrank.net d-black.bz
datract.com daisye.top
dayibiao.com darodar.com
dealwifi.com dataloading.net
delayreferat.ru ddlmega.net
demian.kz decorazilla.com
detskie-zabavi.ru design-lands.ru
dichvuvesinhhanoi.com dev78.dioniqlabb.se
discountbarbour.online dignitasdata.se
dktr.ru dimfour.com
dolohen.com diminishedvalueoforegon.com
domain-tracker.com discovertreasure-a.akamaihd.net
domainsatcost.com divci-hry.info
domcran.net dneprsvet.com.ua
donna7753191.ru documentsite.net
doreenblog.online doeco.ru
download-of-the-warez.blogspot.com domain-submit.info
draniki.org domaincdn.xyz
dverimegapolis.ru domik-derevne.ru
dvr.biz.ua dostavimvdom.ru
dwomlink.info dosug-lux.ru
elektrischezigarette2.onsugar.com downloads-whatsapp.com
elektrischezigaretten2.beeplog.com eachdayisagift.review
elektroniksigaraankara.info edudocs.net
elektrozigaretten1.postbit.com efkt.jp
elkacentr.ru egvar.net
electricwheelchairsarea.com
electro-prom.com
elektir.ru
elektrischezigarettekaufen2.cowblog.fr
elektronischezigarettekaufen1.beeplog.com
elektrozigarettekaufen1.devhub.com
emediate.eu emediate.eu
erot.co en.home-task.com
event-tracking.com englate.com
extstat.com eropornosex.ru
f0815.de eshop4u.jp
facebook-mobile.xyz euronis-free.com
famix.xyz exoneration-project.us
faptitans.com expertblog.info
fastfixing.tech extrabot.com
favornews.com eyelike.com.ua
fba-mexico.com ezigarettekaufen2.blox.pl
feargames.ru ezigarettenkaufen1.pagina.gr
fellowshipoftheminds.com ezigarettenkaufen2.dreamwidth.org
fertilitetsradgivningen.se falcoware.com
fickkontakte.org familyholiday.ml
fitness-video.net fandlr.com
flashahead.info fastcrawl.com
floridamobilebillboards.com fealq.com
flowwwers.com ferieboligkbh.dk
footbalive.org fickluder69.com
forex-indextop20.ru filerockstar298.weebly.com
foto-weinberger.at filmi-v.online
fotosfotos.eu financehint.eu
frameimage.org finstroy.net
frankofficial.ru fishingwholesale.us
free-deals.faith focalink.com
free-traffic.xyz fullfileaccess.com
freegamesplay.online fxtips.ru
freelinkbuilding.website.tk games.kolossale.ru
froggytube.com
fxgallery.com
galeria-zdjec.com
gameonasia.com
gameplexcity.com
gamingspark.com
gateway.zscalertwo.net gateway.zscalertwo.net
gay-file.com gay-site.store
gazobeton-p.com.ua gdzkurokam.ru
genericviagrasildenafiled.net generousdeal-a.akamaihd.net
get-free-social-traffic.com gentamicineyedrops.blogspot.com
getaclueamerica.com geoads.com
gilbertbanda.net girlgamerdaily.com
glasof.es goldpanningtools.com
goldadpremium.com golmau.host.sk
golfresa.lucania.se goodwriterssales.com
gomusix.com gourcy.altervista.org
gopro-online.info greatdealshop.com
goroda-vsego-mira.ru greatgrace.ru
gosarhivrt.ru gstatey.net
gpirate.com gtopstats.com
gsasearchengineranker.space hamacapty.com
gyffu.com hamilton.ca
handicapvantoday.com handicapvansarea.com
hasshe.com handsandlegs.ru
havepussy.com hdhc.ru
highspeed5.net hesteel.pl
hobbyhuren24.net highstairs-a.akamaihd.net
hobild.net hitmuzik.ru
holidaypics.org holidaypics.org
hollywoodactress.info homik.pw
homedecoguide.info hotel-mkad.ru
hotblognetwork.com hotshoppymac.com
hoverboard360.de hotsocialz.com
husky-shop.cz hptwaakw.blog.fc2.com
hystersister.com huimin764128.com
i-hobot.ru humanorightswatch.org
iideaidekonkatu.info hut1.ru
ilovevitaly.com iblogpress.xyz
imgata.com imgata.com
immigrational.info imguramx.pw
imperiafilm.ru indiakino.net
impotentik.com infostatsvc.com
in-tandem.co intellego.info
inboundlinks.win internet-apteka.ru
incanto.in.ua isitpaleo.info
increasewwwtraffic.info itronics.ca
inmate-locator.us janettabridal.com
int.search.mywebsearch.com javcoast.com
iomoio.net jumptap.com
ispac.org kaac.ru
itag.pw kambasoft.com
iwanttodeliver.com kazka.ru
izhevsk.zrus.org keywordhouse.com
janavibekken.no kidskunst.info
javspace.net kinoflux.net
jerseywholesaleelitestore.com kiwe-analytics.com
jolic2.com klin3952.ru
jose.mulinohouse.co
kaidalibor.de
kalandranis.gr
kamorel.com
kartiniresto.com
ketrzyn.pl
keywesthideaways.co
kimcurlrvsms.com
kino-key.info
kollesa.ru
komp-pomosch.ru
konturkrasoty.ru
koronirealestate.gr
krasnodar-avtolombards.ru
kreativperlen.ch kreativperlen.ch
kustanay.kz kupit-adenu.ru
landreferat.ru
laptoper.net
larger.io larger.io
layola.biz.tc lashstudia.ru
lightinghomes.net leadwayau.com
linerdrilling.com legalrc.biz
littlesexdolls.com lego4x4.ru
liupis.com letmacwork.world
live-cam6.info lion.bolegapakistan.com
locustdesign.co livingcanarias.com
lookingglassemb.com loadingpages.me
lunchrock.co locatellicorretor.com.br
lomza.info
longadventure.com
louboutinreplica.pw
love-baby.cz
lyrics.home-task.com
m4ever.net m4ever.net
mafa-free.com mactechinfo.info
magicdiet.gq madot.onlinewebshop.net
magicplayer-s.acestream.net mansiondelrio.co
magnetpress.sk mantramusic.ru
mainlinehobby.net marktforschung-stuttgart.com
manipulyator-peterburg.ru masterseek.com
mapquestz.us mature.free-websites.com
masthopehomes.com max-eclat.men
matchpal-a.akamaihd.net mebel-alait.ru
matrixalchemy.com menetie.ru
medkletki.ru
meget.co.za
menhealed.net
mexicotravelnet.com
mhtr.be
michaelkorsoutlet.store
michellblog.online
middlerush-a.akamaihd.net middlerush-a.akamaihd.net
milblueprint.com mjchamonix.org
mindeyegames.com mmstat.com
minecraft-neo.ru mobicover.com.ua
minecraft-rus.org mobot.site
miniads.ca monetizer.com-01.site
mirzonru.net moneyviking-a.akamaihd.net
missclub.info montredemarque.nl
mlf.hordo.win morocco-nomad-excursions.com
mmgq.ru moviezbonkerssk.cf
mmofreegames.online moyakuhnia.ru
mockupui.com mrbojikobi4.biz
mojpregled.com mriyadh.com
mole.pluto.ro msfsaar.de
mp3films.ru mtmtv.info
muzaporn.com mttwtrack.com
mydownloadengine.com music7s.me
mygameplus.ru mwtpludn.review
myhitmp3.club my-bc.ru
myonlinepayday.co my-floor.in.ua
mywallpaper.top myliveblog.ru
naj-filmy24.pl na-telefon.biz
narco24.me netcheckcdn.xyz
naturalpharm.com.ua newhairstylesformen2014.com
naval.jislaaik.com newstraveller.ru
neodownload.webcam newyorkhotelsmotels.info
netallergy.ru nicefloor.co.uk
nibbler.silktide.com no-fuel.org
notasprensa.info norththeface.store
novodigs.com noscrapleftbehind.co
nzfilecloud.weebly.com nrjmobile.fr
o-o-8-o-o.ru o.light.d0t.ru
oakleyglassesonline.us oballergiya.ru
ochistka-stokov.ru obsessionphrases.com
olvanto.ru odesproperty.com
omgtnc.com odoratus.net
onblastblog.online officedocuments.net
online-sbank.ru oil-td.ru
online7777.com okout.ru
onlinemegax.com oneshotdate.com
onlinetvseries.me online-hit.info
optom-deshevo.ru online-x.ru
panchro.xyz onlyforemont.ru
paperwritingservice17.blogspot.ru orion-v.com
parajumpersoutlet.online ozoz.it
parfusale.se paintingplanet.ru
partner-high.men partner-host.men
pay2me.pl patol01.pw
pc-test.net pawli.eu
pechikamini.ru pdn-4.com
pelfind.me pensplan.com
pensplan4u.com pepperstyle.ru
perfectpracticeweb.com photostudiolightings.com
perl.dp.ua pijoto.net
pewit.pw pinapchik.com
pic2fly.com pirateday.ru
picswe.com pixgood.com
picturesfrom.com pl-top.pl
pinsdaddy.com
pinwallpaper.xyz
piski.top
pl-vouchers.com
platesauto.com
platezhka.net platezhka.net
ploenjitmedia.azurewebsites.net polimga.pw
popup-jdh.xyz polybuild.ru
poquoson.org poptool.net
pornhive.org porno-chaman.info
pornogratisdiario.com porno-raskazy.ru
pornokorol.com pornoreino.com
pornosok.ru pornotubs.com
potoideas.us portal-eu.ru
predominant-invent.tk pospr.waw.pl
preparevideosafesystem4unow.site pourvous.info
pretty-mart.com powenlite24.ru
primedice.com privhosting.com
printie.com prizefestival.mobi
privatamateure.com prodvigator.ua
prod2016.com projectforte.ru
profitfx.online prostitutki-tyumeni.org
prom23.ru proxyradar.com
prostitutki-astana.org qexyfu.bugs3.com
pulse33.ru questionmarque.ch
qcstrtvt.bloger.index.hr quit-smoking.ga
quick-offer.com
quickloanbank.com
rada.ru
rangoman.date
rankingchart.de
rankscanner.com
ranksonic.info ranksonic.info
refads.pro rcb101.ru
rialp.getenjoyment.net realtytimes.com
rniaeba.ga redirect.trafficreceiver.club
robertefuller.com redirectme.net
rockprogblog.com replicaclub.ru
royal-investments.net rezeptiblud.ru
rp9.ru ribieiendom.no
runstocks.com ric.info
rus-teh.narod.ru rimedia.org
sabizonline.com ringporno.com
sc-specialhost.com rn-to-bsn.com
scanner-julia.top robotixix.com
sdi-pme.com rocis.site
sdsjweb.com royal-betting.net
search-goo.com royalads.net
search.xtconnect.com roznica.com.ua
searchinterneat-a.akamaihd.net rusoft-zone.ru
searchtooknow-a.akamaihd.net russkie-gorki.ru
sack.net
safe-app.net
samara.rosfirm.ru
sammyweaver.com
sanyuprojects.com
sarahmilne.top
savetubevideo.com
scanner-jessica.top
scansafe.net
sdelatmebel.ru
seasaltwithfood.com seasaltwithfood.com
selectads.men securesmrt-dt.com
seofirmreviewsus.info seo-smm.kz
seogadget.ru seo18.su
servisural.ru seokicks.de
sevendays.com.ua seonetwizard.com
sexblog.pw sexfreepornoxxx.com
sexphoto.site sexkrasivo.net
sexy.babes.frontend-stack.top sexkvartal.com
sexyteens.hol.es sexyebonyteen.com
sfd-chess.ru shiksabd.com
shinikiev.com.ua shoppingmiracles.co.uk
ship-marvel.co.ua simplesite.com
shopcheermakeup.info simply.net
sibvitr.ru skytraf.xyz
sim-service.net sladkoevideo.com
siteexpress.co.il small-games.biz
sk.golden-praga.ru smsactivator.ru
skanninge.se snegozaderzhatel.ru
skuteczna-dieta.co.pl snip.tw
skylta.com soaksoak.ru
smartpet.ru social-buttons.xyz
social-vestnik.ru softxaker.ru
sogimlecal.tk souvenirua.com
solitaire-game.ru speeddream.xyz
soundfrost.org speedup-my.site
souvenir.cc spin2016.cf
soviet-portal.do.am spinnerco.ca
spanking.to sport-video-obzor.ru
spb.afora.ru spylog.com
sprttrack.com stackthatbucks.com
spy-app.info star61.de
squidoo.com stardevine.com
sr-rekneskap.no statoutlook.info
steebook.com
stefanbakosab.se stefanbakosab.se
stopnarco.ru stevemonsen.com
strag-invest.ru strana-solnca.ru
studworks.org studiofaca.com
superinterstitial.com stuff-about-money.com
surcentro.com stuffpride.com
sukarame.net
supernew.org
surintech.ac.th
svetlotorg.ru svetlotorg.ru
sweepstakes.rewardit.com tackletarts.co
swinger-mobil.net tecnoteakviareggio.it
swsociety.se telegramdownload10.com
sygraem.com terraclicks.com
tastyfoodideas.com teslathemes.com
technika-remont.ru theguardlan.com
the-torrent-tracker.blogspot.com thetattoohut.com
thedownloadfromwarez.blogspot.com thin.me.pn
thegioixekhach.com tiandeural.ru
theheroes.ru time-japan.ru
thetoiletpaper.com timecrimea.ru
thiegs.reco.ws tizanidine4mg.blogspot.com
tiens2010.ru tizanidineformigraines.blogspot.com
tilido.com tizanidineforopiatewithdrawal.blogspot.com
tizanidine4mgtablets.blogspot.com tizanidinehcl2mgsideeffects.blogspot.com
tizanidinemusclerelaxant.blogspot.com tizanidinepills.blogspot.com
tizanidinenarcotic.blogspot.com tizanidinewithdrawalsymptoms.blogspot.com
tizanidineorflexeril.blogspot.com tmtrck.com
tizanidinesleepaid.blogspot.com tobeyouday.win
tizanidinewithdrawal.blogspot.com tonerbox.kz
tkanorganizma.ru tooplay.com
tonivedu.it top-deal.com.pl
totu.info topdownloads.ru
totu.us toproadrunner5.info
tpu.ru toy-shop.top
tracxn.com tri-slona.org
trahvid.com trkdf.com
trainoffend.ml truemfilelj.gq
triplepanda.xyz tupper-shop.ru
tubeline.biz turizmus.us
tuberkulezanet.ru tvnewsclips.info
turizm.bz u17795.netangels.ru
ukrtvory.in.ua um-razum.ru
unrealcommander.org unisexjewelry.org
urlcut.ru
urll.eu
urlopener.com urlopener.com
usacasino.com uvozdeckych.info
v-doc.co valmetrundan.se
v24s.net vapomnoncri.tk
varikozdok.ru viel.su
video-production.com vikistars.com
videochat.bz vinylvault.co.uk
videospornogratisx.net vip-file.com
vilingstore.net virtuagirl.com
villakohlanta.nu virus-respirators.com
vitalads.net viteonlusarezzo.it
viveresaniesnelli.it vodkoved.ru
vkontaktemusic.ru vonradio.com
vladhistory.com vzglyadriv.kg
voronezh.xrus.org vzlom-na-zakaz.com
vse-pesni.com walkme.com
warezaccess.com warning.or.kr
webenlace.com.ar watchmyfb.pl
webmasterhome.cn wdfdocando.com
webnode.me webcamtalk.net
westen-z.life webix.biz
while.cheapwebsitehoster.com westum.se
win-spy.com whipme.yopoint.in
wnoz.de withstandingheartwarming.com
womens-journal.net wwwadultcheck.com
wonderfulflowers.biz x-rates.ru
woodyguthrie.se xaijo.com
wrona.it xaylapdiendanang.com
www.ehscloud.cn xdoza.com
www.kabbalah-red-bracelets.com xmlinde.com
www.timer4web.com xmnb.net
x-stars.ru xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
x-true.info xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xn--80ajbshivpvn2i.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xn--l1aengat.xn--p1ai
xn----9sbubg3ambdfl1j.xn--p1ai xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xn--80aaagvmjabrs1aoc9luc.xn--p1ai xtremeeagles.net
xn--80atua3d.xn--p1ai xxart.ru
xtube.com xxxhdvideo.site
xxxdatinglocal.us xxxrus.org
xxxkaz.org yadro.ru
yaminecraft.ru yogamatsexpert.com
yellowstonevisitortours.com youporn-forum.ga
yun56.co youporn-ru.com
yunque.pluto.ro yourmovies.pl
zagadki.in.ua yourothersite.com
zaimite.ru yoursite.com
zapatosenventa.info youtube-downloader.savetubevideo.com
zeroredirect6.com yuarra.pluto.ro
zetgie.com.pl za-music.mymobiplanet.com
zetmaster.ru zebradudka.com
zigarettenonlinekaufen2.drupalgardens.com zero1.it
zionstar.net zeroredirect11.com
zkjovpdgxivg.ga zeroredirect7.com
zolotoy-lis.ru zhongwenlink.com
zvooq.eu zigarettenonlinekaufen.tumblr.com
znaniyapolza.ru
zoompegs.com
zophim.me
zscaler.net
zvetki.ru

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123cha.com 4kpics.rocks
3weekdiet.com 7fon.ru
83net.jp 8kisses.eu
aanapa.ru aaronabel.com
abiente.ru abclauncher.com
adpremium.org adloads.com
ads.gold adobereader-free.ru
adtiger.tk adsloads.com
am15.net agadelha.com.br
amateurlivechat.org agahinameh.com
anatomiy.com alfa-medosmotr.ru
angel1777.kz all4invest.info
animalia-life.club all4invest.ru
apple.com-webbrowsing-security.review alveris.ru
atleticpharm.org amazon-seo-service.com
aviav.eu ampower.me
avtolombard-krasnodar.ru apibetweenlinesn-a.akamaihd.net
backiee.com app-ready.xyz
balla-boo.se apple.com-webbrowsing-security.science
bangla-chat-uk-paralud.ga aproposde.com
bankiem.pl arcadeplayhouse.com
baoxaydung.com.vn arenda-yeisk.ru
best-seo-software.xyz arendovalka.xyz
bharatdefencekavach.com asiengirls.net
bidr.trellian.com audiofree.ru
blackcurranthumidifiers.site autochoixspinelli.com
blogsfunky672.weebly.com avtochehli.by
bluerobot.info bala.getenjoyment.net
bmw-ark.ru beetpics.pw
bolezniorganov.ru bestwebsiteawards.com
boo-arts.com betune.onlinewebshop.net
boole.onlinewebshop.net bimatoprost-careprost.com
bronzeaid-a.akamaihd.net bloggerads.net
brothers-smaller.ru blogs.rediff.com
brus-vsem.ru bongiornos.info
buyessay3.blogspot.ru brus.city
buynorxx.com buttons-for-your-website.com
buzzonclick.com camdenmemorials.com
byme.se camsex.xxx-cam.webcam
cakemediahq.com.au cartierreplica.win
callawaygolfoutlet.online cashonads.com
camdolls.com cdncash.org
canyougethighofftizanidine.blogspot.com chatroulette.video
cascadelink.org cheapbelstaff.online
casinopinup-wins.com citetick.com
celebsopics.com claytransformations.info
centrdebut.ru clickso.com
cheapsergiorossi.online cometorussia.net
chocolateslim-en-espana.com compliance-elena.top
clickzzs.nl cookie-law-enforcement-ee.xyz
coinsspb.com czat.wp.pl
com-onlinesupport.site czeshop.info
compliance-george.xyz d2jsp.org
compliance-ivan.top darodar.com
compliance-nelson.top
comsysnet.com
concordexoticrentals.com
credit.co.ua
dafatiri.com
daptravel.com
daretodonate.co
dawlenie.com dawlenie.com
day-news.info db.speedup.gdn
dcj-nn.ru dcj-nn.ru
digitalassetmanagement.site delitime.info
domaincdn.xyz diesel-parts28.ru
domashneeruporno.com dominateforex.ml
dpihatinh.gov.vn eksprognoz.ru
e-biznes.info elvenar.com
e90post.com emergencyneeds.org
ecookna.com.ua erotik0049.com
ekspertmed.com escortplius.com
elektroniksigaraankara.info etm-consult.de
elektrozigaretten1.webs.com exchanges-bet.com
expolicenciaslatam.co exoneration-project.us
ezaz.info ezigarettekaufen2.blox.pl
ezigarettekaufen2.yolasite.com fickkontaktehobbyhuren.com
ezigarettenkaufen1.deviantart.com fidelityfunding.com
fickenprivat.info filesdatabase.net
fighrofacciufreesig.ga filmbokep69.com
fil.ru flashlarevista.com
filerockstar298.weebly.com fototravel.eu
film-one.ru freegamesplay.online
filmgo.ru galaxy-family.ru
fiverr.com gallerylisting.com
foxydeal.com games.kolossale.ru
franecki.net gavuer.ru
free-today.com ge0ip.com
freelinkbuilding.website.tk gemgrab-a.akamaihd.net
freewhatsappload.com gettpromos.com
ftns.ru getyourimage.club
globatur.ru gidonline.one
gsasearchengineranker.site gk-atlant.info
gungamesz.com global-ics.co.za
hawaiielectriclight.com gombita.info
hdwalls.xyz googlemare.com
healing-dysplasia.ru gourcy.altervista.org
heliko.no gsmlab.pl
hobbyhuren24.net hausfrauensex18.com
hotgirlhdwallpaper.com haveinc.xyz
houseofrose.com hdhc.ru
i-hobot.ru hifidesign.ru
iblogpress.xyz homedecoguide.info
ideoworld.org homemade.gq
igtools.club hop.clickbank.net
interior-stickers.ru horoshieokna.com
hostsshop.ru
housemilan.ru
hyiphunter.org
imagerydatabase.com
img.wallpaperstock.net
inbabes.sexushost.com
incep.imagine-ex.co
iradiology.ru
islandminingsupply.wordpress.com islandminingsupply.wordpress.com
ispaniya-costa-blanca.ru
istanbulit.com istanbulit.com
jerseywholesaleelitestore.com johannesburgsingles.co.za
johnrobertsoninc.com jolic2.com
jovencitas.gratis
justkillingti.me
kevblog.top
keyword-suggestions.com keyword-suggestions.com
keywordspay.com keywordglobal.co.za
kinofak.net keywordsdoctor.com
kndxbkdx.bloger.index.hr kino-fun.ru
kometars.xyz koral.se
konpax.com koronirealestate.gr
kosynka-games.ru kraljeva-sutjeska.com
landscapebackgrounds.blogspot.com lego4x4.ru
layola.biz.tc lenpipet.ru
lerporn.info letolove.ru
lifeinsurancekb.com letto.by
liupis.com lion.playtap.us
livefixer.com lisque.batcave.net
livingcanarias.com live-sexchat.ru
locustdesign.co lookingglassemb.com
longgreen.info lost-alpha.ru
louboutinreplica.xyz madot.onlinewebshop.net
m.b00kmarks.com magnetpress.sk
magicdiet.gq mainhp.net
mailemedicinals.com medmajor.ru
maranbrinfo.com.br metafilter.com
master-muznachas.ru moncleronline.xyz
matchpal-a.akamaihd.net montazhnic.ru
medizinreisen.de moscow.xrus.org
menetie.ru mtmtv.info
metarip.ru my-cash-bot.co
militarysale.pro mydownloadengine.com
missis.top myonlinepayday.co
mjchamonix.org mzdish.site
mlf.hordo.win nastydollars.com
moneymaster.ru nicefloor.co.uk
moneyteam24.com nmrk.ru
monsterdivx.tv nofreezingmac.click
mouselink.co nootrino.com
mrvideospornogratis.xxx nuup.info
myhitmp3.club oklogistic.ru
myiptest.com okout.ru
newhairstylesformen2014.com onclkads.com
nikhilbahl.com onlainbesplatno.ru
niki-mlt.ru optibuymac.com
nonews.co orgasmatrix.com
nordstar.pro pammik.ru
onlineserialy.ru panchro.co.uk
onlinewritingjobs17.blogspot.ru partnerads.men
oxford-book.com.ua partnerline.men
ozas.net partners-ship.pro
partner-stop.men pateaswing.com
pdn-4.com phrcialiled.com
pensplan4u.com
perm.xrus.org
phpdevops.com
picsearch.com picsearch.com
popup-fdm.xyz pigrafix.at
pornhive.org pix-hd.com
pornohub.me pl-vouchers.com
pp-budpostach.com.ua platezhka.net
prchecker.info porno-play.net
prizestohandle.club predmety.in.ua
prostitutki-kharkova.org privhosting.com
purplesphere.in prospekt-st.ru
quick-offer.com prostitutki-rostova.org
ranksignals.com pukaporn.com
rapevideosmovies.com radiodigital.co
reklamuss.ru rentalcarnavi.info
renewablewealth.com rethinkwasteni.info
responsive-test.net scrapy.org
rialp.getenjoyment.net sdrescher.net
rn-to-bsn.com seadragonherbery.com
rome2rio.com searchtooknow-a.akamaihd.net
ru-mediaget.ru semrush.com
runofilms.ru seo-traffic-ranking.info
rustic-quiver.win sexfreepornoxxx.com
salutmontreal.com sexobzor.info
samoiedo.it sexreliz.com
schoolfiles.net sexsaoy.com
seemoreresultshu-a.akamaihd.net slim.sellany.ru
sethrollins.net smartshoppymac.com
sex-sex-sex5.com socas.pluto.ro
sexkontakteao.info softomix.net
sexpartygirls.net softwaretrend.net
sexyali.com soviet-portal.do.am
sharebutton.net spb.afora.ru
shop.xz618.com spidtest.space
slimcdn.com stauga.altervista.org
slonechka.ru steamoff.net
snowplanes.com sunflowerdrawingpaintings.blogspot.com
soaksoak.ru tastyfoodideas.com
socialbuttons.xyz telegraf.by
sport7777.net thebitcoincode.com
spy-app.info thepokertimer.com
stackthatbucks.com time-japan.ru
stretchingabuckblog.com titelhelden.eu
studentguide.ru tizanidine4mgprice.blogspot.com
sudexpert66.ru tizanidinehcl4mgisitanarcotic.blogspot.com
superinterstitial.com tizanidinepills.blogspot.com
tacbibirfa.tk tjkckpytpnje.com
td-l-market.ru tksn.ru
teletype.in tobeyouday.win
testingads.pro topcasinoratings.ru
thegalerie.eu topquality.cf
themestotal.com topsy.com
tizanidinemusclerelaxant.blogspot.com torrentred.games
tomatis.gospartner.com tourcroatia.co.uk
tongkatmadura.info tracksurf.daooda.com
toonfamilies.net trubywriting.com
track-rankings.online tube8.com
trafficjunky.net ukrtvir.com.ua
treasuretrack-a.akamaihd.net umekana.ru
tsstcorpcddvdwshbbdriverfb.aircus.com umg-stroy.ru
u555u.info underthesite.com
uchetunet.su unrealcommander.biz
umityangin.net uroffer.link
universals.com.ua urzedowski.eu
vigrx-original.ru utiblog.fr
vitanail.ru vacuumcleanerguru.com
vkonche.com versaut.xxx-cam.webcam
vksaver-all.ru vgoloveboli.net
walpaperlist.com videonsk.com
webs.com vip2ch.com
wedding0venues.tk visionwell.com.cn
wesharepics.site vita.com.hr
whereiskentoday.com vpnmouse.com
wholinkstome.com vrnelectro.ru
wnoz.de vsesubwaysurfers.com
woodyguthrie.se vzlomtw.com
works.if.ua web-analytics.date
woweb.com.ua webix.biz
wygraj-skiny.win website-analytics.online
wzgyyq.com wejdz-tu.pl
x-musics.com ws.ampower.me
xaijo.com x-diesel.biz
xjlottery.com xingzi-vision.com
xxxtube69.com xn----ctbbcjd3dbsehgi.xn--p1ai
yellowfootprints.com xtrafficplus.com
yourserverisdown.com yellowads.men
yourtemplatefinder.com zakonobosago.ru
zero1.it zastroyka.org
zocaparj.kz zerocash.msk.ru
zx6.ru zog.link

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.43 0; 10.1.0.242 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3518 ### Version: V4.2022.12.3519
### Updated: Thu Dec 15 10:04:06 UTC 2022 ### Updated: Thu Dec 15 22:01:24 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3518 ### Version: V4.2022.12.3519
### Updated: Thu Dec 15 10:04:06 UTC 2022 ### Updated: Thu Dec 15 22:01:24 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##