vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 02:59:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2724 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-19 10:09:03 +00:00
parent 62e4342259
commit 958b61eca7
47 changed files with 64470 additions and 64460 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2723 #### Version: V4.2021.11.2724
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2723 #### Version: V4.2021.11.2724
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2723 #### Version: V4.2021.11.2724
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7920
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2723 #### Version: V4.2021.11.2724
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7924
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7924
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

7924
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7924
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 1; 10.1.0.249 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 0; 10.1.0.249 0;
127.0.0.1 0; 127.0.0.1 0;

7924
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 1; 10.1.0.249 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 0; 10.1.0.249 0;
127.0.0.1 0; 127.0.0.1 0;

7924
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 1; 10.1.0.249 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 0; 10.1.0.249 0;
127.0.0.1 0; 127.0.0.1 0;

7924
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2723 2724

7924
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

7924
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.124.229.231 0; 51.143.41.154 0;

7924
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 1; 10.1.0.249 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 0; 10.1.0.249 0;
127.0.0.1 0; 127.0.0.1 0;

7924
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 1; 10.1.0.249 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 0; 10.1.0.249 0;
127.0.0.1 0; 127.0.0.1 0;

7924
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 1; 10.1.0.249 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 0; 10.1.0.249 0;
127.0.0.1 0; 127.0.0.1 0;

7924
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.124.229.231 0; 51.143.41.154 0;

7924
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.124.229.231 0; 51.143.41.154 0;

7924
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 1; 10.1.0.249 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AwarioSmartBot Brandprotect
LibWeb Disco
Download Wonder
Fyrebot
Nessus Nessus
SpaceBison Octopus
Trendictionbot PxBroker
Zauba SalesIntelligent
arquivo-web-crawler Web Sucker
clark-crawler tAkeOut
seobility
sexsearcher

320
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com ADmantX
404checker
AIBOT AIBOT
ASPSeek ALittle Client
Abonti
Aboundex Aboundex
AhrefsBot Aboundexbot
Aipbot Aipbot
Alexibot
AllSubmitter
Alligator Alligator
AlphaBot
Anarchy Anarchy
Aspiegel Apexoo
Asterias
Attach Attach
AwarioRssBot BDFetch
BDCbot
BLEXBot BLEXBot
BackStreet BackStreet
Bandit BacklinkCrawler
BatchFTP
Battleztar Bazinga Battleztar Bazinga
Black Hole Bigfoot
BlackWidow BlackWidow
Blackboard Blackboard
BlowFish Boardreader
BotALot Bolt
Buddy Brandprotect
BuiltWith Buck
BunnySlippers
BuzzSumo
CODE87 CODE87
CSHttp Calculon
Cegbfeieh CensysInspect
ChinaClaw ChinaClaw
Chlooe
Claritybot Claritybot
Cliqzbot Cogentbot
Cloud mapping Copier
Cocolyzebot CopyRightCheck
Copyscape
Cosmos Cosmos
CrazyWebCrawler
Crescent
Curious Curious
Custo DBLBot
DIIbot
DSearch
DTS Agent
DataCha0s DataCha0s
DatabaseDriverMysqli Disco
Devil DomainStatsBot
Dirbuster Dotbot
Discobot Download Wonder
Dispatch
DittoSpyder
DnBCrawler-Analytics
DomainAppender
DomainSigmaCrawler
Dragonfly
Drip Drip
EasyDL ECCP/1.0
Ebingbong Ebingbong
Exabot Ecxi
EirGrabber
EroCrawler
ExtLinksBot ExtLinksBot
ExtractorPro Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FDM FHscan
FlashGet FemtosearchBot
FyberSpider Flunky
GT::WWW Foobot
GalaxyBot FrontPage
GermCrawler GermCrawler
GetRight GetRight
GetWeb
Getintent
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit
GrabNet GrabNet
Grabber
Grafula
GrapeFX
HEADMasterSEO HEADMasterSEO
HMView
HTTP::Lite
HTTrack HTTrack
Haansoft Haansoft
HaosouSpider
Heritrix Heritrix
Hloader
Humanlinks Humanlinks
IDBTE4M HybridBot
IDBot IRLbot
Image Sucker Iblog
Intelliseek Id-search
InterGET IlseBot
InternetSeer InfoNaviRobot
Iria Internet Ninja
Iskanie IstellaBot
JOC Web Spider
JamesBOT JamesBOT
Jyxobot Jbrofuzz
Kenjin Spider JetCar
Leap Jetty
LeechFTP JikeSpider
LexiBot Joomla
Lftp Jorgee
Lightspeedsystems JustView
Keyword Density
Kozmosbot
LNSpiderguy
Lanshanbot
Larbin
LeechGet
Libwhisker
Likse
LinkScan LinkScan
Linkbot Linkbot
LinqiaMetadataDownloaderBot Linkdexbot
Litemage_walker LinkextractorPro
Lmspider LinkpadBot
Ltx71 LinksManager
LinqiaRSSBot
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc MQQBrowser
MJ12bot Mag-Net
MSFrontPage
Magnet Magnet
Majestic-SEO Mail.RU_Bot
Mass Downloader Majestic SEO
MarkMonitor
Mata Hari
Mb2345Browser
MeanPath Bot
Mediatoolkitbot
Metauri Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Mister PiX Mojeek
Moblie Safari Mr.4x3
Morfeus Fucking Scanner
Msrabot
Musobot
NICErsPRO
Name Intelligence
Net Vampire
NetLyzer NetLyzer
NetSpider
NextGenSearchBot NextGenSearchBot
NimbleCrawler Nibbler
Nimbostratus Niki-bot
Nikto
Ninja
Nmap
Nutch Nutch
Octopus OnCrawl
Openfind OpenLinkProfiler
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PECL::HTTP
PageGrabber Page Analyzer
PageScorer PageScorer
PageThing.com
Pandalytics Pandalytics
PeoplePal PeoplePal
Petalbot Pi-Monster
PictureFinder Pimonster
PleaseCrawl ProPowerBot
Probethenet Probethenet
Pump Pu_iN
QueryN Metasearch PyCurl
Quick-Crawler Quick-Crawler
RSSingBot RankActiveLinkBot
RankingBot RankFlex
RankingBot2 RankingBot2
Re-re
ReGet ReGet
RealDownload
RedesScrapy
RepoMonkey
Ripper Ripper
Rogerbot SBIder
SEOkicks
SEOlyticsCrawler
SEOprofiler
SMTBot
SalesIntelligent
ScanAlert ScanAlert
ScoutJet
Scrapy
ScreenerBot
ScrepyBot ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
Siphon SentiBot
SiteExplorer SeobilityBot
SiteSucker SiteSucker
Sitebeam
Sitevigil Sitevigil
Snake SlySearch
Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper Sottopop
Sogou web spider
Sosospider
SpankBot
Spanner
Spbot Spbot
Sqlmap Spinn3r
Sqlworm
Sqworm Sqworm
SuperBot Steeler
Sucuri
Surfbot Surfbot
SurveyBot
Suzuran Suzuran
Swiftbot
Szukacz Szukacz
TeleportPro Thumbor
Telesoft TightTwatBot
Telesphorep
TheNomad
Toweyabot Toweyabot
Trendiction
Trendictionbot Trendictionbot
Turingos TwengaBot
Twice URLy Warning
Typhoeus
URLy.Warning URLy.Warning
UnisterBot
Upflow Upflow
VB Project VB Project
Vacuum Vacuum
VeriCiteCrawler Vagabondo
VelenPublicWebCrawler
Virusdie Virusdie
VoidEYE Voil
Voltron WBSearchBot
WWW-Collector-E WWW-Collector-E
WallpapersHD WallpapersHD
Web Auto
Web Collage Web Collage
Web Enhancer
Web Fetch Web Fetch
WebCopier WebBandit
WebFetch WebCollage
WebFuck WebFuck
WebImageCollector WebStripper
WebLeacher
WebPix
WebSucker
WebWhacker
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Webster
Whack Whack
Whacker Whatweb
WinHTTrack Widow
Wonderbot Wonderbot
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider Zeus
Xenu Zitebot
Zade
Zauba
Zermelo
ZmEu ZmEu
ZoominfoBot backlink-check
archive.org_bot coccocbot-web
autoemailspider
check1.exe
cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com dataforseo.com
demandbase-bot
domainsproject.org domainsproject.org
facebookscraper facebookscraper
gopher instabid
ips-agent lwp-trivial
isitwp.com meanpathbot
mediawords
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
pcBrowser
probe-image-size probe-image-size
ripz scalaj-http
satoristudio.net
scan.lol scan.lol
seoscanners seocompany.store
serpstatbot seostar
siteripz sitechecker.pro
sp_auditbot sp_auditbot
spyfu ubermetrics-technologies.com
tAkeOut voyagerx.com
trendiction.com
webmeup-crawler
webpros.com webpros.com
zauba.io webprosbot
x09Mozilla
x22Mozilla
zgrab zgrab

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404enemy 404checker
AIBOT ADmantX
ALittle Client ALittle Client
Abonti
Aboundex
Aboundexbot
Acunetix
Alexibot Alexibot
AllSubmitter Alligator
AlphaBot AlphaBot
Anthill Anarchy99
Ankit
Apexoo
Asterias Asterias
AwarioRssBot AwarioRssBot
BDFetch AwarioSmartBot
BLEXBot
BackDoorBot BackDoorBot
BackStreet BackStreet
Badass
Barkrowler
Battleztar Bazinga Battleztar Bazinga
Black Hole Bigfoot
BotALot Blackboard
Buck Bolt
BuiltBotTough Buddy
BuiltWith
Bullseye Bullseye
BunnySlippers BuzzSumo
CSHttp CATExplorador
CazoodleBot CCBot
CherryPicker CODE87
ChinaClaw Calculon
Cliqzbot Cegbfeieh
Cloud mapping Copier
Cogentbot Craftbot
CopyRightCheck CrazyWebCrawler
Copyscape
CrunchBot CrunchBot
Curious
Custo Custo
CyotekWebCopy DataCha0s
DBLBot
Demon Demon
Deusu Discobot
Devil
DigitalPebble
Dirbuster
Discoverybot Discoverybot
DnyzBot DittoSpyder
DomCopBot
DomainAppender
DomainSigmaCrawler
Dotbot Dotbot
Download Wonder Download Wonder
Dragonfly
Drip
ECCP/1.0 ECCP/1.0
EMail Wolf
EasyDL
Ebingbong Ebingbong
EirGrabber Ecxi
EroCrawler Evil
Exabot Express WebPictures
ExtractorPro Extreme Picture Finder
Ezooms Ezooms
FHscan FDM
Fimap Fimap
Firefox/7.0
Flunky
Freeuploader
FrontPage FrontPage
Fuzz Fuzz
FyberSpider
Fyrebot
G-i-g-a-b-o-t
GalaxyBot GalaxyBot
Genieo Genieo
Getintent Go!Zilla
Go-Ahead-Got-It GoZilla
Gotit Gotit
GrapeFX Grabber
Grafula
GrapeshotCrawler GrapeshotCrawler
HMView GridBot
HTMLparser HaosouSpider
HTTrack Harvest
Havij Heritrix
HybridBot
IDBTE4M IDBTE4M
IDBot
Id-search Id-search
IlseBot IlseBot
IndeedBot Image Sucker
InfoNaviRobot InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
Iria Internet Ninja
Iskanie InternetSeer
JamesBOT JamesBOT
JennyBot JennyBot
Jetty Jetty
JikeSpider JikeSpider
Joomla JustView
Jyxobot
Kenjin Spider
Kozmosbot Kozmosbot
Lanshanbot
Larbin
LeechFTP LeechFTP
LexiBot LeechGet
Lftp LieBaoFast
Libwhisker Likse
LinkScan
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinksManager
LinqiaScrapeBot LinqiaScrapeBot
MJ12bot Lipperhey
Lipperhey Spider
MFC_Tear_Sample
MIIxpc
MSFrontPage
MSIECrawler
MTRobot
Mag-Net
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mass Downloader
Mata Hari Masscan
Mb2345Browser MegaIndex.ru
MeanPath Bot
Metauri
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Mojeek
Mister PiX Mojolicious
Moblie Safari
Mozlila
Mr.4x3 Mr.4x3
Msrabot
Musobot Musobot
NICErsPRO
NearSite NearSite
Needle Needle
NetZIP Nessus
NetLyzer
NetMechanic
NetSpider
Netcraft Netcraft
Nettrack Nettrack
Netvibes
Nibbler
Nikto
NimbleCrawler NimbleCrawler
Nmap Nimbostratus
Nutch Nuclei
OpenVAS OnCrawl
Openvas OpenLinkProfiler
OutfoxBot Openfind
PHPCrawl PHPCrawl
PageGrabber PageAnalyzer
PageThing.com PageThing.com
Pandalytics Pandalytics
Panscient Petalbot
Papa Foto Picsearch
Pi-Monster
PictureFinder PictureFinder
Pimonster PleaseCrawl
Pockey
ProPowerBot ProPowerBot
ProWebWalker ProWebWalker
RankActive Psbot
RankActiveLinkBot PyCurl
RankingBot QueryN Metasearch
RSSingBot
Rankivabot
RankurBot RankurBot
RealDownload Re-re
RebelMouse
Ripper
RocketCrawler RocketCrawler
Rogerbot
SBIder
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOstats
SISTRIX
SalesIntelligent SalesIntelligent
Scanbot Scanbot
ScreenerBot ScoutJet
Scrapy
Screaming
Searchestate
SearchmetricsBot SearchmetricsBot
Shodan Semrush
Site Sucker SentiBot
SiteSnagger SeoSiteCheckup
Sitebeam
SlySearch SlySearch
SmartDownload Sosospider
Snake
Snapbot
SocialRankIOBot
Sogou web spider
SpankBot SpankBot
Spanner
Spinn3r Spinn3r
SputnikBot Sqlmap
Sqlworm Stripper
Steeler
Sucker
Surfbot Surfbot
Suzuran
Swiftbot Swiftbot
Szukacz Szukacz
T0PHackTeam
T8Abot
Teleport
TeleportPro TeleportPro
The Intraformant Telesoft
TheNomad Telesphorep
Thumbor Thumbor
TightTwatBot TightTwatBot
Titan Titan
URLy Warning Toweyabot
URLy.Warning Turnitin
UnisterBot Twice
Upflow Upflow
V-BOT V-BOT
VB Project VB Project
Vacuum
Vagabondo Vagabondo
Voltron VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
Virusdie
VoidEYE
WEBDAV WEBDAV
WPScan WISENutbot
WWW-Collector-E WWWOFFLE
WWW::Mechanize Wallpapers/3.0
Web Auto WallpapersHD
Web Sucker Web Collage
WebBandit Web Enhancer
Web Fuck
WebCollage
WebFuck
WebGo IS WebGo IS
WebImageCollector
WebSauger WebSauger
WebStripper
WebSucker WebSucker
WebsiteExtractor Website Quester
WebsiteQuester
Webster Webster
Whack Widow
Whacker
Whatweb
WinHTTrack WinHTTrack
Wonderbot Wprecon
Wotbox Xaldon WebSpider
YoudaoBot
Zade Zade
Zauba ZyBorg
Zermelo
ZmEu
ZoomBot
archive.org_bot
arquivo-web-crawler
arquivo.pt arquivo.pt
backlink-check cah.io.community
check1.exe
coccocbot-web coccocbot-web
com.plumanalytics com.plumanalytics
crawl.sogou.com crawler4j
dataforseo.com
eCatch eCatch
facebookscraper
gopher gopher
heritrix
isitwp.com isitwp.com
lwp-request lwp-request
lwp-trivial lwp-trivial
magpie-crawler meanpathbot
mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
pcBrowser
plumanalytics plumanalytics
polaris version
s1z.ru s1z.ru
scan.lol scan.lol
seocompany.store
seoscanners
seostar
serpstatbot serpstatbot
siteripz
sogouspider sogouspider
tAkeOut spyfu
trendiction.com sysscan
ubermetrics-technologies.com ubermetrics-technologies.com
webmeup-crawler voyagerx.com
webpros.com webpros.com
x09Mozilla x09Mozilla
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
abctoppictures.net advocatemsk.ru
ibmdatamanagement.co ahrefs.com
mokrayakiska.com bizru.info
opinionreelle.com cialisndbrx.com
playfortuna-play.ru dptaughtme.com
pskcijdc.bloger.index.hr gateway.zscalerone.net
twinderbella.com sildenafil-tadalafil.info
typer.one softtor.com
viagroid.ru voltrknc1.com
www.timer4web.com xn----ctbbcjd3dbsehgi.xn--p1ai

944
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03p.info 0daymusic.org
1008.su 101billion.com
178evakuator178.ru 12masterov.com
1adult.com 1bet.com
1xbet4you.com 1j7740kd.website
202ch.com 1millionusd.xyz
404.mn 2daytrendingnews.com
7search.com 3-letter-domains.net
8jn.dating-app.ru 45en.ru
aab94f698f36684c5a852a2ef272e031bb.com 4kepics.com
aad1f4acb0a373420d9b0c4202d38d94fa.com 5000-cotydzien.com
abusora.com 7a2rnuey1tw9ar.ru
acmebtn.ml 888.com
acunetix-referrer.com 8kisses.eu
adservme.com 991mostfm.co.id
afmuseum.com a.pr-cy.ru
afora.ru aa08daf7e13b6345e09e92f771507fa5f4.com
afslankpillen2017nl.eu aac500b7a15b2646968f6bd8c6305869d7.com
ageofclones.com abctoppictures.net
ahrefs.com adamoads.com
air-edem.ru adexprts.com
aksonural.ru adrunnr.com
akvamaster.dp.ua ads.gold
alarmobninsk.ru adserver-e7.com
adtech.us
adultgalls.com
akusherok.ru
alldezservice.kz
allproblog.com allproblog.com
alta-realestate.com alphacarolinas.com
amazon-seo-service.com amanda-porn.ga
android-systems.ru amoi.tn
anmysite.com an-donut.com
arcteryxstore.online apioasisspacenet-a.akamaihd.net
ausergrubhof.info apioutoboxnet-a.akamaihd.net
autonew.biz apiwoodensealcom-a.akamaihd.net
aversis.net arenda-yeisk.ru
artdestshop.eu
asdfz.pro
asrv-a.akamoihd.net
autoloans.com
avkzarabotok.info avkzarabotok.info
avtolombard-krasnodar.ru avtoarenda.by
avtovolop.ru backuperwebcam.weebly.com
b00kmarks.com bannerconnect.net
baditri.com bazaronline24.ru
bannerswap.com beachtoday.ru
bashtime.ru beremenyashka.com
belinvestdom.by bestprofits-there.com
besthomemadepornsites.com bestwebsiteawards.com
bestwebsitesawards.com bestwrinklecreamnow.com
bezsporno.ru
bgviagrachrx.com bgviagrachrx.com
binaryoptionscops.info bidbuy.co.kr
bio-optomarket.ru biographiya.com
blagovest-med.ru bitcoin-ua.top
bloglag.com biznesluxe.ru
bmw-ark.ru blue-square.biz
bonvillan.com bmwhighperformers.com
bonzbuddy.com bobinoz.com
bookmark4you.com boostmyppc.com
bpro1.top brainxs.us
brakehawk.com brakehawk.com
brandehk.dk brandedleadgeneration.com
brateg.xyz bravegirlsclub.com
brk-rti.ru brokergid.ru
browserprotecter.com buqyxa.rincian.info
btnativenav.com buttons-for-your-website.com
buehne-fuer-menschenrechte.de buybest1.biz
bum.com.ru buyessay3.blogspot.ru
buqayy0.livejournal.com candiceloves.us
businescoop.men candlewooddentalcentre.com.au
buttons-for-website.com
buzzurl.jp
bycontext.com
callmd5map.com
cam-kontakte.org
candytiner.com candytiner.com
carmuffler.net canoncdriverq3.pen.io
cartierlove.xyz cardsharp1.ru
cartierreplica.win cars-modification.net
cartechnic.ru
cartierring.xyz cartierring.xyz
catalogs-parts.com cayado.snn.gr
cbozhe.com centrumcoachingu.com
cctva.tv chatseo.com
cejewelry.xyz cheapestjerseys-wholesale.com
centre-indigo.org.ua chocolateslim-en-france.com
cgi2.nintendo.co.jp choice-credit.ru
chastnoeporno.com cialischsrx.com
cheapmarmot.online cialischstgerts.com
chee-by.biz cialisndbrx.com
chinese-amezon.com cigarpass.com
cialischmrx.com cinemaflix.website
cleanallvirus.com citysecurity.nu
clmforexeu.com cl.s7.exct.net
club-lukojl.ru clarithromycin500mg.com
clean-virus-mac.com
clickso.com
coderstate.com coderstate.com
coldfilm.ru
commoncrawl.org commoncrawl.org
compliance-alexa.top compliance-donald.xyz
compliance-fred.xyz confib.ifmo.ru
compliance-mary.top cookielawblog.wordpress.com
conciergegroup.org coolyarddecorations.com
conocer-sanabria.com copypaste.traderzplanet.in
corta.co cordstrap.cc
cvety24.by cougfan.info
cyclobenzaprinevstizanidine.blogspot.com countercrazy.com
daneshetabiat.com crackzplanet.com
decoratinghomes.ga craftinsta.ru
delta-line.men credit-online.ws
detmebel.su crystalslot.com
dichvuvesinhhanoi.com customerguru.in
dildofotzen.net czat.wp.pl
dinkolove.ya.ru dame-ns.kz
divci-hry.info darkages.info
doctormakarova.ru data-mining.tk
dogsrun.net de.zapmeta.com
domain-tracker.com design-lands.ru
domcran.net dikx.gdn
download-of-the-warez.blogspot.com directrev.com
downloaddy.net djstools.com
drev.biz dnepropetrovsk.mistr-x.org
drivotracker.com dominterior.org
dwomlink.info dostavka-v-krym.com
e705.net downloads-whatsapp.com
eandsgallery.com doxysexy.com
ebooknovel.club doyouknowtheword-flummox.ml
ed-shop01.ru dverimegapolis.ru
ege-essay.ru dynainbox.com
electrik-avenue.com e2click.com
elektrischezigarette1.blog.pl easytuningshop.ru
en.altezza.travel ednorxmed.com
enbersoft.com eduinfosite.com
enhand.se eleimgo.pw
eralph.tk elektrischezigaretten1.blogse.nl
erotikchat-24.com elektrozigaretten2.yn.lt
errorfixing.space elitedollars.com
eugenevaultstorage.com eloconcream.blogspot.com
euronis-free.com emailaccountlogin.co
emediate.eu
esnm.ru
evening-dating-club.info evening-dating-club.info
everflora.ru excaliburfilms.com
exdocsfiles.com exoneration-project.us
extremepornos.net exonline.info
ezigarettenkaufen2.dreamwidth.org failingmarriege.blogspot.com
ezigaretteshop.webs.com fakehandbags.xyz
f0815.de fanrto.com
factorynightclub.com
faithe.top
falopicm.pw
farm26.ru farm26.ru
fetishinside.com fast-wordpress-start.com
financetip.eu faz99.com
findacheaplawyers.com fickkontaktehobbyhuren.com
fitness-video.net fickluder69.com
fikasound.tk
filesclub.net
finteks.ru
flash4fun.com flash4fun.com
florida-tourism.net
flowwwers.com
footballfarrago.com
forexunion.net
forms-mtm.ru forms-mtm.ru
fortwosmartcar.pw foxinsocks.ru
forum69.info
foto-sisek.porngalleries.top
foxtechfpv.com foxtechfpv.com
free-share-buttons.blogspot.com free-deals.faith
galaxyflowers.ru free-floating-buttons.com
gameprimary.com fun-mobi.pl
gdzkurokam.ru fx-brokers-review.com
ge0ip.net gabeshop.ru
geckoandfly.com gagrasector.ru
genericlowlatencyasiodriverhq.aircus.com galaxys6manual.info
generousdeal-a.akamaihd.net gambarkatabaru.com
getpopunder.com gaming-journal.com
gheus.altervista.org gateway.zscalertwo.net
ghostvisitor.com gay-site.store
globalscam.ga gaytube.com
globetrotting-culture.ru gazporno.com
golfresa.lucania.se gentamicineyedrops.blogspot.com
golmau.host.sk gifspics.com
googlemare.com gigapeta.com
graphics8.info gloverid.site
greatdealshop.com gobongo.info
greenidesign.co gogps.me
gribkovye-zabolevaniya.com goldadpremium.com
growboxbl.ru googst2.ru
hauleddes.com gosarhivrt.ru
hdpixion.com gotwebsite1.com
headpress.ru gsbs.com.ua
healbio.ru guiadeserraazul.com
hermesreplica.win guigyverpo.cf
hikesearch.net gwebtools.com.br
homedo.fabpage.com hackers-crackers.tk
homegardenlova.com hamilton.ca
hotblog.top handicapvansarea.com
hotgirlhdwallpaper.com hdfreeporno.net
hothot.ru helvetia.com.ua
houseofgaga.ru hermesbirkinhandbagoutlets.com
hoverboardforsaledirect.com historichometeam.com
hyhj.info hitcpm.com
hyiphunter.org holidaypics.org
iamsport.org hotel-mkad.ru
ictizanidinehcl4mg.blogspot.com hotelcrocenzi.sm
id-forex.com hothor.se
iflycapetown.co.za hreade.com
iframe-toloka.com humanelydrew.com
ikearugs.xyz humbmosquina.tk
ilikevitaly.com ilovevitaly.ru
immobiliaremassaro.com imagui.eu
infektsii.com imfamous.info
infobanks.ru img.wallpaperstock.net
informatiecentro.be imguramx.pw
imk.com.ua
imperia31.ru
impisr.edunsk.ru
impresagaia.it
indianmedicaltourismshop.com
infobabki.ru
infokonkurs.ru
infostatsvc.com
infowarcraft.ru
interferencer.ru interferencer.ru
internet-apteka.ru iphantom.com
intimshop-fantasy.ru iplogger.org
investzalog.ru iqoption.pro
ip-guide.com irkutsk.online-podarki.com
iqbazar.ru irunfar.com
istizanidineanopiate.blogspot.com itservicesthatworkforyou.com
izismile.com ix20.ru
janettabridal.com jackpotchances.com
jerseysbizwholesalecheap.com janerikholst.se
jerseywholesalechinabiz.com japan-bearings.ru
jerseywholesaleelitestore.com jillepille.com
johnnyhaley.top jimmychoosale.online
journeydownthescale.info jongose.ninja
juliaworld.net junketjuice.blogspot.com
kakadu-interior.com.ua jus0wil.pp.ua
kanimage.com kabbalah-red-bracelets.com
karaganda.xkaz.org kalb.ru
kasino-money.pw kambasoft.com
kazan.zrus.org kinoflux.net
kfon.eu kochanelli.com
kihi.gdn koptims.tiu.ru
kinky-fetishes.com koronirealestate.gr
kino2018.cc kotaku.com
kollekcioner.ru krafte.ru
komputers-best.ru kvartiry-remont.ucoz.ru
krasnodar-avtolombards.ru lacasamorett.com
ks1234.com lalalove.ru
kupiproday.com.ua
lanasshop.ru
landinez.co landinez.co
lashstudia.ru landscapebackgrounds.blogspot.com
laudit.ru laptoper.net
lecbter-relationships.ga letto.by
lerporn.info life-instyle.com
letmacworkfaster.world likesdesign.com
letsart.ru liveshoppersmac.com
limtu.ifmo.ru
listiseltemournlan.gq
lizace.com lizace.com
loadopia.com
lollypopgaming.com
lonerangergames.com
lookingglassemb.com lookingglassemb.com
louboutinbooties.xyz macfix.life
louboutinreplica.pw magda-gadalka.ru
lsex.xyz magiadinamibia.blogspot.com
ltvperf.com mansparskats.com
lyrster.com marktforschung-stuttgart.com
macnewtech.com masterseek.com
magicdiet.gq mcar.in.ua
mail.allnews24.in meccadumps.net
makis.nu medi-fitt.hu
maladot.com meget.co.za
maranbrinfo.com.br metarip.ru
marinetraffic.com metaxalonevstizanidine.blogspot.com
mecash.ru michellblog.online
meddesk.ru mikozstop.com
medicine-4u.org minet.club
medicines-choice.com mixed-wrestling.ru
medispainstitute.com.au mmofreegames.online
mega-polis.biz.ua mmog-play.ru
members.ghanaweb.com mncrftpcs.com
metabar.ru mnogabukaff.net
microsearch.ru mobile.ok.ru
mister-shop.com moesen-ficken.com
modenamebel.ru monarhs.info
mompussy.net monclerboots.xyz
monclerjacketsoutlet.pw monclercheap.xyz
monclerjacketsoutlet.win moneyteam24.com
monetizer.com-01.site morpicert.pw
moneyviking-a.akamaihd.net moscow-clining.ru
moomi-daeri.com moskva.nodup.ru
morepoweronmac.trade moz.com
mouselink.co mrbitsandbytes.com
msfsaar.de muizre.ru
music.utrolive.ru muz-shoes.ru
musicas.baixar-musicas-gratis.com myliveblog.ru
mybackgroundlandscape.blogspot.com myonlinepayday.co
mycouponizemac.com nagdak.ru
mydownloadengine.com narkologiya-peterburg.ru
myftpupload.com nastroyke.net
mylesosibirsk.ru needtosellmyhousefast.com
mymercy.info neks.info
mypornfree.ru networkad.net
nailsimg.com
narkologiya-orel.ru
new-post.tk new-post.tk
newhairstylesformen2014.com next-dentists.tk
news-readers.ru nicovideo.jp
nikhilbahl.com nippon-bearings.ru
november-lax.com nodup.ru
nrv.co.za nordvpn.com
northfacestore.online
notfastfood.ru
novodigs.com
ntic.fr
nudejapan.net nudejapan.net
nw-servis.ru olvanto.ru
o-o-6-o-o.ru openfrost.net
obiavo.com optom-deshevo.ru
ok.ru
okroshki.ru
oktube.ru
onclkads.com
onemactrckr.com
onetravelguides.com
onlinemegax.com
optikremont.ru
orion-v.com orion-v.com
ostroike.org page2rss.com
oz-offers.com paighambot.com
paydayonlinecom.com patterntrader-en.com
pekori.to pawli.eu
pensplan.com pcads.ru
perl.dp.ua pestomou.info
phobia.us phuketscreen.com
phrcialiled.com picsforkeywordsuggestion.com
picturesify.com picturesfrom.com
pillscheap24h.com picurams.pw
pinapchik.com pintattoos.com
piulatte.cz pinwallpaper.top
piuminiita.com pistonclasico.com
pivka.xyz
pixell.club pixell.club
pkr1hand.com pizza-imperia.com
plastgran.com play.leadzupc.com
platesauto.com pons-presse.com
poiskzakona.ru
polimga.pw
poptool.net poptool.net
poquoson.org pornohd1080.online
pornogig.com pornoinn.com
pornolook.net
pornosok.ru pornosok.ru
porodasobak.net pornowarp.info
potoideas.us potolokelekor.ru
pourvous.info pp-budpostach.com.ua
praisong.net privat-girl.net
preconnubial.usuby.site proekt-gaz.ru
princevc.com profitfx.online
priora-2.com progress-upakovka.ru
privetsochi.ru prokotov.com
prizesbook.online proprostatit.com
prostitutki-rostova.org prosmibank.ru
pulseonclick.com pussyspace.net
qcstrtvt.bloger.index.hr qualitymarketzone.com
radiogambling.com queerspace.com
rank-checker.online quit-smoking.ga
rankexperience.com r-e-f-e-r-e-r.com
rapidgator-porn.ga randki-sex.com
rednise.com rangjued.com
registrydomainservices.com rczhan.com
reimageplus.com recinziireale.com
remmling.de reelheroes.net
retreatia.com regdefense.com
rossanasaavedra.net remont-comp-pomosh.ru
ru-dety.ru remont-fridge-tv.ru
rus-pornuha.com remontgruzovik.ru
ruscoininvest.company research.ifmo.ru
rusexy.xyz rezeptiblud.ru
ryetaw.com riralmolamsaca.tk
s-iwantyou.com royalcar-ufa.ru
sa-live.com runofilms.ru
saitevpatorii.com rusenvironmental.net
samoiedo.it rutor.vip
s1z.ru
sabaapress.com
sady-urala.ru
samo-soznanie.ru
samolet.fr
sbprabooks.com
sbricur.com
scanner-fred.top scanner-fred.top
scanner-george.top scanner-josh.top
scanner-julianna.top scrapinghub.com
scanner-willy.top scripted.com
schoolfiles.net search.pch.com
search.1and1.com searchimage.co
seo-2-0.com sell-fb-group-here.com
seo-smm.kz seo-smm.kz
seotoolsagency.com seozoom.it
serptehnika.ru servethis.com
service.adtech.fr sexcamamateurchat.com
serw.clicksor.com sexkrasivo.net
sex-tracker.de sextracker.be
sexreliz.com sexxdate.net
sexvporno.ru sexystrippe.info
shoppingjequiti.com.br sharebutton.net
sharebutton.org
shinikiev.com.ua
shtora66.ru shtora66.ru
sibtest.ru sisiynas.ru
sierraapps.com site-speed-check.site
silvercash.com skincrate.net
silvermature.net skylta.com
sims-sims.ru skypasss.com
skachat-besplatno-obrazcy.ru social-search.me
sloopyjoes.com socialsignals24.com
sluganarodu.ru soserfis.com
soblaznu.net sovinsteel.ru
sohoindia.net space2019.top
solinf.co speeddream.xyz
spaceshipad.com sports-supplements.us
spasswelt.net sr-rekneskap.no
spb.ru sribno.net
sps-shop.com stackthatbucks.com
steame.ru start.myplaycity.com
sterva.cc stauga.altervista.org
stevemonsen.com store-rx.com
stretchingabuckblog.com stroicol.net
strfls.com success-seo.com
superoboi.com.ua sunflowerdrawingpaintings.blogspot.com
support.nopeas.sk supermesta.ru
svolze.com supernew.org
syndicate.fun taaaak.com
tam-gde-more.ru tacbibirfa.tk
taqywu51.soup.io taximytishi.ru
taraz.xkaz.org telegraf.by
tasteidea.com terraclicks.com
telefonsexsofort.tk teslathemes.com
telsis.com thaismartloan.com
textads.men the-trader.net
thefds.net thebestweightlosspills.ovh
timecrimea.ru thecoolimages.net
timhost.ru thedownloadfromwarez.blogspot.com
tivolibasket.it thegameriders.com
tizanidine4mgstreetprice.blogspot.com thepantonpractice.co.uk
tizanidineduringpregnancy.blogspot.com tilido.com
tizanidinehcl4mgtab.blogspot.com tizanidinehydrochloride2mg.blogspot.com
tizanidinepain.blogspot.com tizanidinenarcotic.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com tnctrx.com
tizanidinesleepaid.blogspot.com tonerbox.kz
topads.men top10-online-games.com
toy-shop.top topmira.com
tracklead.net torrnada.ru
traderzplanet.in totu.us
tradgardspartner.se tourcroatia.co.uk
trafficmonetize.org track.deriv.com
trafficshaper.com tracksurf.daooda.com
trahvid.com traffic2cash.org
treasuretrack-a.akamaihd.net trafficmania.com
truck-addzilla.life traffictrade.life
trucri.me trustaffs.com
turbo-suslik.org tryrating.com
turvgori.ru tsstcorpcddvdwshbbdriverfb.aircus.com
tvoystartup.ru turizmus.us
twojebook.pl twittrading.com
tytoona.com typer.one
u-cheats.ru ua.tc
uggbootsoutletsale.us ukrup.com
ugogo.info ultimatesetnewfreeallsoftupgradesystems.pw
uni.me underthesite.com
uptime-as.net unitexindia.com
usiad.net unitygame3d.com
uvozdeckych.info unmaroll.ya.ru
uzporno.mobi uptimechecker.com
vancleefreplica.pw urlcut.ru
vclicks.net vegan-foods.us
veles.shop velen.io
venta-prom.ru vepad.com
videogamesecrets.com vezuviy.su
videospornogratisx.net videochat.cafe
villakohlanta.nu vipms.ru
vip-file.com virtuagirl.com
vipsiterip.org virus-respirators.com
viveresaniesnelli.it visa-pasport.ru
vremya.eu voloomoney.com
wallpaperaccess.com vpnmouse.com
website-audit.com.ua vse-pesni.com
website-datenbank.de vtcdns.com
wedding0venues.tk vucms.com
westen-z.life vzlomsn.org
westsextube.com watsonrealtycorp.com
westum.se webshoppermac.com
wfb.hatedriveapart.com wesharepics.info
whiteproduct.com westen-v.life
wholesalenfljerseys.us.com wieseversa.no
withstandingheartwarming.com wikes.20fr.com
wnoz.de wildworld.site
wonderfulflowers.biz windowssearch-exp.com
world-mmo.com wowas31.ucoz.ru
wovis.site wurr.voila.net
ws.ampower.me www.bookmaker-bets.com
ww2awards.info
www.888.com
wyniki-lista.pl wyniki-lista.pl
x-diesel.org xn----dtbndd4ae7eub.top
xingzi-vision.com xn--80aikhbrhr.net
xn----ctbbcjd3dbsehgi.xn--p1ai xolodremont.ru
xn----itbeirbjbi7bc6bh2d.xn--p1ai xpresscare.ru
xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--80abgj3a5acid6ghs.top
xn--80atua3d.xn--p1ai
xn--lifehacer-1rb.com
xnxx699.com
xsion.net xsion.net
xtube.com yaderenergy.ru
yebocasino.com yellocloud.be
youandcredit.ru yogamatsexpert.com
youradulthosting.com yopoint.in
youthreaders.com zaim-pod-zalog-krasnodar.ru
yun56.co zarepta.com
yur-p.ru zastenchivosti.net
zakazvzloma.com zastroyka.org
zeg-distribution.com zhongwenlink.com
zigarettenonlinekaufen1.blox.pl zigarettenonlinekaufen2.bloog.pl
zigarettenonlinekaufen2.drupalgardens.com zirondelli.it
zigzog.ru zoogdinsney.com
zooggames.com zoomovies.org
zoominfo.com zophim.me
zreloeporno.tv zscalerone.net
zverokruh-shop.cz
zvooq.eu

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
08800.top 0c47f8422d3f.com
14b.info 1001desert.com
1bet.com 273-fz.ru
1stat.ru 3th.co.in
45en.ru 5i2.net
4pp13.com 5ws.dating-app.ru
4rent.ru 999webdesign.com
98oi.ru aa625d84f1587749c1ab011d6f269f7d64.com
a96527gi.beget.tech absolute-s.ru
abiente.ru adbetclickin.pink
abwa.tk adtech.us
adelly.bg agapovdg.ru
ads.tremorhub.com air-edem.ru
adservme.com allforminecraft.ru
akita.kz animali.deagostinipassion.it
aludecor.info anniemation.com
am-se.com apessay.com
apimossnetinfo-a.akamaihd.net api.stathat.com
arcteryxsale.online apibrowsesmartne-a.akamaihd.net
avon-novosib.ru arendakvartir.kz
avtointeres.ru artdeko.info
axbocz.net artel116.ru
azartniy-bonus.com atlant-auto.info
baixar-musicas-gratis.com aviav.ru.com
baltgem.com azazaporn.com
bannerswap.com backgroundpictures.net
battle.net bankofthewext.com
bayareaaeroclub.org bestmarriages.com
becuo.com bioca.org
bestwebsiteawards.com biteg.xyz
bg6s0.com biznesschat.net
bio.trade-jp.net blog-hits.com
bizfly.info bloggerads.net
bjgugu.net.ua blpmovies.com
blog.f00kclan.de boeuklad.com
blog.koreadaily.com bonzybuddy.com
bluesalt.co brk-rti.ru
bmsco.co brothers-smaller.ru
bolitgorlo.net browserprotecter.com
boltalko.xyz buckspar.gq
bonkers.name businesn.men
bookmark4you.com buyclomidonlaine.com
bradleylive.xyz buyfriend.ru
bristolhotel.com canrole.com
butstrap.space carina-sy.de
buy-meds24.com carsplate.com
buyessay3.blogspot.ru cartujano-pre.de
calc-for-credit.ru cascadelink.org
cardiosport.com.ua celejihad.info
carloans.com christianlouboutinsaleonline.us
clayvasedesigns.tk cialisndbrx.com
colehaanoutlet.store civilwartheater.com
compliance-fred.top classicakuhni.ru
compliance-walter.top cleanmypc.com
cookie-law-enforcement-aa.xyz com-secure.download
copblock.org compliance-checker.info
coprofam.org compliance-olga.top
crazyboost.pro conboy.us
cutt.us country-chic.ru
cxpromote.com crackguru.tk
dailyfinancefix.com cymbaltaandtizanidine.blogspot.com
data.vtc.pw dame-ns.kz
de.zapmeta.com davidovic.info
deda-moroza-zakaz.ru demian.kz
deniven.1bb.ru dignitasdata.se
dfwu1019.info dogbreedspicture.net
diesel-parts28.ru ecxtracking.com
dildofotzen.net
dimaka.info
docstoc.com
domain2008.com
drunkmoms.net
ege-essay.ru
ekspertmed.com
elektrozigaretten1.postbit.com
elitepcgames.com elitepcgames.com
elmacho.xyz eluxer.net
epicbrogaming.com enews.tech
essay-edu.biz essaypro.com
eu-cookie-law.blogspot.com etotupo.ru
eu-cookie-law.info facebook-mobile.xyz
eyemagination.com farmprofi.net
fermersovet.ru fettefrauen.net
fillmewithhappiness.com fickkontaktehobbyhuren.com
financepoints.eu filesdatabase.net
florida-tourism.net finteks.ru
footballfarrago.com fishtauto.ru
forexgb.ru flowwwers.com
freakycheats.com franklinfire.co
free-social-buttons.xyz fym.com.ru
freshmac.space gambarkataku.co
freza-sverlo.ru getprismatic.com
froggytube.com getyourimage.club
gael-s.ru gg.zzyjxs.com
game-top.su ggiaro.com
gk170.ru golaya.pw
gomusix.com gomusix.com
hatdc.org gsasearchenginerankerhelp.com
gwebtools.com
hasfun.com
hdseriale.pl
healing-dysplasia.ru healing-dysplasia.ru
helpmymacfaster.trade hem.passagen.se
highstairs-a.akamaihd.net hermesbirkinhandbagoutlets.com
hit-men.men hostnow.men
hoste.octopis.com hotchatdate.com
houseofrose.com houdom.net
id-forex.com hoverboard360.nl
ilte.info hybrid.ru
investsuccess.org iamsport.org
inzn.ru ilovevitaly.ru
istizanidineanopiate.blogspot.com imguramx.pw
iwantmyfreecash.com inome.com.ua
janettabridal.com interior-stickers.ru
jerseysbizwholesalecheap.com irkutsk.online-podarki.com
karachev-city.ru islandminingsupply.wordpress.com
karadene.com itronics.ca
kinoplen.ru jetsli.de
komputernaya-pomosh-moscow.ru johnrobertsoninc.com
kareliatobacco.ru
karusel-market.ru
kichenaid.ru
kinohall.ru
klin3952.ru
koptims.tiu.ru koptims.tiu.ru
kvartira-sutochno.com kursy-ege.ru
labelwater.se ladsblue.com
lawyersinfo.org ladsup.com
letmacworkfaster.site landreferat.ru
liran-locks.com laptoper.net
liverpool.gsofootball.com ledpolice.ru
lotto6888.com legalrc.biz
lottospring.com limads.men
mactechinfo.info lineavabit.it
medkritika.ru linkbolic.com
medosmotr-ufa.ru linkbux.ru
milblueprint.com lol-smurfs.com
mnogolok.info lombia.com
mobile.ok.ru lonfon.xyz
mozello.ru longgreen.info
mttwtrack.com lsp-awak-perikanan.com
musicvidz.ru manve.info
muycerdas.xxx maximilitary.ru
muz-baza.net members.ghanaweb.com
mylida.org micasainvest.com
mzdish.site mini.7zap.com
nalogovyykodeks.ru modabutik.ru
nash-krym.info mojowhois.com
neobux-bg.info monclercheap.xyz
novgorod.xrus.org moncleroutletonline.win
nrv.co.za monetizer.com-01.site
nudejapan.net moz.com
nuit-artisanale.com mrwhite.biz
nullrefer.com musicpro.monster
oil-td.ru myphotopipe.com
ok.ru na-telefon.biz
omsk.xrus.org net-radar.com
onescreen.cc nmrk.ru
onetravelguides.com o333o.com
onlinadverts.com obiavo.uz
ops.picscout.com online-hit.info
ozoz.it online-templatestore.com
piccshare.com ooo-gotovie.ru
pio.polytopesexempt.com optikremont.ru
pistonclasico.com paccohichetoti.ml
playfortuna-play.ru paydayonlinecom.com
poligon.com picsearch.com
picurams.pw
piski.top
play-movie.pl
pobeiranie.pl
pokemon-go-play.online
pornhubforum.tk
pornoreino.com pornoreino.com
pornosemki.info powitania.pl
pornosexrolik.com primedice.com
potoideas.us prosmibank.ru
pp-budpostach.com.ua ranksignals.com
pro-okis.ru reining.lovasszovetseg.hu
quickbuck.com remmling.de
quizzitch.net rewardpoll.com
rapidgator-porn.ga rff-cfal.info
realting-moscow.ru rfid-locker.co
reeyanaturopathy.com rrutw.com
ritlweb.com rubbed.us
rockma.se ruf777.com
rollercoin.com ruspdd.com
runofilms.ru rustic-quiver.win
russian-postindex.ru ryetaw.com
rvtv.ru sa-rewards.co.za
s-iwantyou.com sad-torg.com.ua
saitevpatorii.com sbf441.com
saltspray.ru screpy.com
sc-specialhost.com search.pch.com
scanmarine.info seccioncontrabajo.com
semrush.com sexkontakteao.info
seo-prof1.xyz sexvideo-sex.com
sexblog.pw sexxdate.net
sexs-foto.com sexy-screen-savers.com
shell-pmr.ru sfd-chess.ru
site-auditor.online sharebutton.to
skanninge.se shareyards.com
sloopyjoes.com shopcheermakeup.info
slowmac.tech site-speed-check.site
softxaker.ru sitebeam.net
songoo.wz.cz smstraf.ru
songplanet.ru soserfis.com
spasswelt.xyz speechfoodie.com
spy-app.info spylog.com
stair.registrydomainservices.com spywarenuker.com
steebook.com starpages.net
store-rx.com stefanbakosab.se
streetfire.net strana-solnca.ru
sugvant.ru studiokamyk.com.pl
support.nopeas.sk suchenindeutschland.com
swinon.site superstats.com
tacbibirfa.tk sygraem.com
tackletarts.co tattooreligion.ru
teastory.co teenforporn.com
telegraf.by telemetryverification.net
thegameriders.com theheroes.ru
thelottosecrets.com thejournal.ru
thesmartsearch.net timdreby.com
thewebsitetemplate.info titelhelden.eu
tizanidineformigraines.blogspot.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
tour-line.net tootoo.to
tradgardspartner.se top-l2.com
trafficgenius.xyz tracfone.com
trustaffs.com ttsq.fr
trustedhealthtips.com turbabitload.weebly.com
tuberkulezanet.ru twittrading.com
tuberkuleznik.ru u17795.netangels.ru
tubo360.com umaseh.com
turbo-suslik.org unece.org
turist-strani.ru v24s.net
validccseller.com videonsk.com
vashsvet.com vip-parfumeria.ru
vclicks.net virus-schutzmasken.de
venta-prom.ru viveresaniesnelli.it
videositename.com vkak.ru
vip.51.la vse-pesni.com
virus-respirators.com waycash.net
viteonlusarezzo.it westvilletowingservices.co.za
vizitki.net winterclassichockeyjerseys.com
vonradio.com worldis.me
vozbujdenie.com x-diesel.com
w7s.ru x5market.ru
watchmyfb.pl xchangetrak.com
wheelchairliftsarea.com xn--1-8sbcpb0bdm8k6a.xn--p1ai
winwotgold.pl xn--lifehacer-1rb.com
wordpresscore.com yellowstonevisitortours.com
ws.ampower.me youdao.com
xingzi-vision.com youradexchange.com
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai zeg-distribution.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai zeikopay.com
xpresscare.ru zigarettenonlinekaufen1.bloog.pl
youtubologia.it zone-kev717.info
zagadki.in.ua zscaler.net
zaobao.com.sg
zastenchivosti.net
zed21.net
zigarettenonlinekaufen1.blox.pl
zona-aqua.ru
zoogdinsney.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.251 0; 10.1.0.249 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2723 ### Version: V4.2021.11.2724
### Updated: Thu Nov 18 22:01:26 UTC 2021 ### Updated: Fri Nov 19 10:03:12 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2723 ### Version: V4.2021.11.2724
### Updated: Thu Nov 18 22:01:26 UTC 2021 ### Updated: Fri Nov 19 10:03:12 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##