vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 06:15:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.02.2908 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-02-09 22:07:03 +00:00
parent 7f291f67c4
commit e8ab943e48
47 changed files with 83130 additions and 83120 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2907 #### Version: V4.2022.02.2908
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.02.2907 #### Version: V4.2022.02.2908
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.02.2907 #### Version: V4.2022.02.2908
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10254
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.02.2907 #### Version: V4.2022.02.2908
#### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7089](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [636](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10258
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10258
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10258
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10258
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 1; 10.1.1.32 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 0; 10.1.1.32 0;
127.0.0.1 0; 127.0.0.1 0;

10258
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 1; 10.1.1.32 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 0; 10.1.1.32 0;
127.0.0.1 0; 127.0.0.1 0;

10258
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 1; 10.1.1.32 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 0; 10.1.1.32 0;
127.0.0.1 0; 127.0.0.1 0;

10258
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2907 2908

10258
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10258
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.187.3.24 0; 52.234.213.132 0;

10258
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 1; 10.1.1.32 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 0; 10.1.1.32 0;
127.0.0.1 0; 127.0.0.1 0;

10258
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 1; 10.1.1.32 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 0; 10.1.1.32 0;
127.0.0.1 0; 127.0.0.1 0;

10258
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 1; 10.1.1.32 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 0; 10.1.1.32 0;
127.0.0.1 0; 127.0.0.1 0;

10258
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.187.3.24 0; 52.234.213.132 0;

10258
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.187.3.24 0; 52.234.213.132 0;

10258
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 1; 10.1.1.32 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Black Hole Backlink-Ceck
Collector LinksManager
DIIbot Lipperhey
Digincore Magnet
DittoSpyder Microsoft URL Control
G-i-g-a-b-o-t Msrabot
JustView Netvibes
Spanner Spbot
VidibleScraper netEstate NE Crawler
Wprecon zgrab

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 404checker
80legs 80legs
ADmantX
Aboundex Aboundex
Alexibot Aboundexbot
AlphaBot AfD-Verbotsverfahren
AhrefsBot
Aipbot
Alligator
Anarchie
Anarchy Anarchy
Ankit Ankit
Anthill Anthill
Asterias Asterias
Atomseobot Attach
AwarioRssBot BLEXBot
AwarioSmartBot
BBBike
BackWeb BackWeb
Backlink-Ceck Badass
Bigfoot
Bitacle
BlackWidow
BlowFish
BotALot BotALot
Brandprotect Brandprotect
Brandwatch Buck
BunnySlippers Buddy
BuzzSumo CATExplorador
Calculon Calculon
CazoodleBot CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect
CheTeam
CheeseBot CheeseBot
ChinaClaw Chlooe
Citoid
Claritybot
Cloud mapping Cloud mapping
Collector Cocolyzebot
Cogentbot
CopyRightCheck CopyRightCheck
CrazyWebCrawler Copyscape
Cosmos
Craftbot
Crescent Crescent
DIIbot CrunchBot
DTS Agent Curious
DataCha0s DatabaseDriverMysqli
Demon
Deusu
Devil Devil
DigitalPebble DigitalPebble
Disco Disco
DittoSpyder Discoverybot
DnBCrawler-Analytics DomCopBot
DnyzBot
DomainAppender
Download Wonder Download Wonder
Drip
ECCP/1.0 ECCP/1.0
ExtractorPro EasyDL
EroCrawler
Evil
Express WebPictures
Extractor
FDM
FHscan FHscan
Fimap
Firefox/7.0 Firefox/7.0
FrontPage Flunky
Freeuploader
Fuzz Fuzz
FyberSpider FyberSpider
Fyrebot Fyrebot
GT::WWW
GalaxyBot
Genieo Genieo
GetRight GetRight
Getintent GetWeb
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
Gotit
GrabNet GrabNet
GrapeshotCrawler Grabber
GrapeFX
GridBot
HEADMasterSEO HEADMasterSEO
HMView
HTMLparser
Haansoft
HaosouSpider HaosouSpider
Harvest Harvest
HonoluluBot IDBTE4M
IDBot
IRLbot
Iblog
Id-search Id-search
IlseBot Image Fetch
IndeedBot
Indy Library Indy Library
InfoTekies InfoNaviRobot
InterGET InternetSeer
Iskanie
IstellaBot IstellaBot
JOC Web Spider
JamesBOT
JennyBot
Jetty Jetty
Joomla Jorgee
Jyxobot
Kenjin Spider Kenjin Spider
Kozmosbot Kozmosbot
LWP::Simple LNSpiderguy
LeechFTP Lanshanbot
LexiBot
LibWeb LibWeb
Libwhisker LinkScan
LieBaoFast Linkbot
Likse
LinkWalker
Linkdexbot
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
Lmspider
MFC_Tear_Sample MFC_Tear_Sample
MJ12bot
MSFrontPage MSFrontPage
MTRobot Magnet
Mag-Net Majestic12
Mail.RU_Bot
MarkMonitor MarkMonitor
Masscan Mass Downloader
MauiBot MeanPath Bot
Microsoft URL Control Meanpathbot
Mojolicious Mediatoolkitbot
MolokaiBot MegaIndex.ru
Morfeus Fucking Scanner Metauri
Mr.4x3 Minefield
Nameprotect NPbot
Navroad NearSite
Needle NetSpider
Net Vampire
NetAnts
NetLyzer
NetZIP NetZIP
NextGenSearchBot NimbleCrawler
Niki-bot
Nikto
Nimbostratus Nimbostratus
Ninja Nmap
Nuclei
Nutch Nutch
Offline Explorer
Offline Navigator
OnCrawl OnCrawl
Openfind OpenLinkProfiler
OrangeBot OpenVAS
OrangeSpider
OutclicksBot
OutfoxBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageGrabber Page Analyzer
Pandalytics
Panscient
Pavuk Pavuk
Petalbot
Pi-Monster Pi-Monster
PictureFinder Pimonster
Pixray Pixray
ProWebWalker Pockey
Pu_iN Psbot
QueryN Metasearch PxBroker
Quick-Crawler
RankActiveLinkBot
RankFlex RankFlex
RankurBot RankingBot2
Re-re ReGet
Reaper
RebelMouse RebelMouse
Recorder Recorder
SBIder SEOkicks-Robot
SEOkicks SMTBot
SEOprofiler SalesIntelligent
SEOstats Scanbot
SeoSiteCheckup ScoutJet
Shodan Scrapy
Site Sucker ScreenerBot
Searchestate
SearchmetricsBot
SemanticJuice
SemrushBot
SiteCheckerBotCrawler SiteCheckerBotCrawler
Siteimprove SiteExplorer
Sitevigil
SlySearch
Snake Snake
Snapbot Snapbot
SpaceBison Sociscraper
SpankBot SpankBot
Spbot
Spinn3r Spinn3r
SputnikBot Sqworm
SuperHTTP Sucuri
Suzuran Surfbot
Swiftbot
Szukacz Szukacz
T0PHackTeam
TeleportPro TeleportPro
Telesoft
Telesphoreo Telesphoreo
The Intraformant TheNomad
TightTwatBot Tracemyfile
Titan Trendiction
Toata Turnitin
Toweyabot
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Twice
Typhoeus
URLy.Warning URLy.Warning
UnisterBot Upflow
VB Project
Vacuum Vacuum
VeriCiteCrawler VelenPublicWebCrawler
Voil VidibleScraper
Voltron Voltron
WASALive-Bot WASALive-Bot
WBSearchBot
WISENutbot WISENutbot
WPScan WWW::Mechanize
WWW-Collector-E WWWOFFLE
WWW-Mechanize
Web Fetch
Web Collage Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Fetch
WebFuck
WebImageCollector
WebPix
WebReaper
Web Sauger Web Sauger
Web Sucker Web Sucker
WebEnhancer
WebFetch
WebGo IS
WebPix
WebSauger
Webalta Webalta
WebmasterWorldForumBot
Webshag
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Whacker Xaldon_WebSpider
WinHTTrack YoudaoBot
Wprecon
Xaldon WebSpider
Xenu
Zade
Zauba
Zermelo Zermelo
Zeus
ZmEu ZmEu
adscanner ZumBot
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt autoemailspider
backlink-check backlink-check
cah.io.community cah.io.community
dataforseo.com check1.exe
demandbase-bot clark-crawler
crawler4j
domainsproject.org domainsproject.org
facebookscraper facebookscraper
gopher internetVista monitor
instabid isitwp.com
ips-agent
iubenda-radar
lwp-request lwp-request
mediawords lwp-trivial
muhstik-scan magpie-crawler
netEstate NE Crawler
oBot oBot
plumanalytics pcBrowser
probe-image-size ripz
s1z.ru
satoristudio.net satoristudio.net
scalaj-http scalaj-http
scan.lol scan.lol
seocompany.store seocompany.store
seostar seostar
serpstatbot sexsearcher
sitechecker.pro sitechecker.pro
sogouspider siteripz
sp_auditbot
sysscan sysscan
tAkeOut trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com
webgains-bot webgains-bot
x22Mozilla x22Mozilla
zauba.io
zgrab

314
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404enemy 404checker
ALittle Client
ASPSeek
Abonti
Aboundexbot Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
Aipbot Aipbot
Alexibot AllSubmitter
Alligator Apexoo
Anarchie
Anthill
Aspiegel Aspiegel
Asterias
AwarioRssBot AwarioRssBot
AwarioSmartBot
BBBike BBBike
BDFetch
BackDoorBot BackDoorBot
BackStreet
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
BetaBot BetaBot
Black Hole Blow
BlowFish Bolt
Boardreader BotALot
Brandwatch CCBot
BuiltWith CensysInspect
CATExplorador CheTeam
CazoodleBot
CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Chlooe Citoid
Claritybot Claritybot
Cocolyzebot CopyRightCheck
Collector Crescent
Copier
Copyscape
Cosmos
Craftbot
CrazyWebCrawler
CrunchBot CrunchBot
Custo
CyotekWebCopy
DBLBot
DIIbot DIIbot
DTS Agent Demon
DataCha0s Digincore
Deusu Discobot
Devil Discoverybot
Disco
Dispatch
DomainAppender DomainAppender
DomainStatsBot DomainSigmaCrawler
Domains Project
Dotbot Dotbot
Download Wonder Dragonfly
ECCP/1.0
EMail Wolf EMail Wolf
EirGrabber EasyDL
EroCrawler
Evil Evil
Express WebPictures Exabot
ExtLinksBot EyeNetIE
Ezooms
Fimap
Flunky
Foobot
Freeuploader Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GalaxyBot GermCrawler
Gigabot Getintent
Go!Zilla
Gotit Gotit
Grabber GrabNet
Grafula Grafula
HEADMasterSEO GrapeshotCrawler
HTTP::Lite HMView
HTTrack HTMLparser
Haansoft Haansoft
HaosouSpider
Harvest
Havij Havij
Iblog Heritrix
HonoluluBot
Humanlinks
HybridBot
Id-search
Image Fetch
IndeedBot IndeedBot
Indy Library
InfoTekies
InterGET
InternetSeer InternetSeer
Iskanie JamesBOT
Jbrofuzz Jbrofuzz
JennyBot JetCar
Jetty
JikeSpider JikeSpider
Jorgee Jorgee
JustView JustView
Kenjin Spider Jyxobot
Keyword Density Keyword Density
LNSpiderguy Kinza
Leap Kozmosbot
LeechGet LibWeb
Libwhisker
LieBaoFast LieBaoFast
Lightspeedsystems Linkbot
Likse
LinkScan
LinkWalker
Linkdexbot Linkdexbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaScrapeBot
Lipperhey
Lmspider
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MIIxpc
MJ12bot MJ12bot
MSFrontPage
Mag-Net
Magnet Magnet
Mail.RU_Bot
Majestic SEO
Majestic-SEO
MarkMonitor
Mass Downloader Mass Downloader
Masscan
Mata Hari Mata Hari
MauiBot MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot MeanPath Bot
Metauri MegaIndex.ru
Minefield Mojeek
Moblie Safari Morfeus Fucking Scanner
Mojolicious Mozlila
Mr.4x3 Navroad
Msrabot NetLyzer
NPbot
Name Intelligence
NearSite
Needle
Nessus
NetMechanic NetMechanic
NetZIP NetSpider
Netcraft Netcraft
Nettrack Netvibes
Nibbler NextGenSearchBot
Nmap
Not
Nutch Nutch
Octopus
Offline Explorer Offline Explorer
OpenLinkProfiler OrangeBot
Openfind PECL::HTTP
OutfoxBot PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
Panscient PageThing.com
PeoplePal
Picsearch Picsearch
Piepmatz Pimonster
Probethenet Pixray
PxBroker Psbot
Pump
PyCurl PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankingBot2 RankFlex
Re-re Reaper
RebelMouse
RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
Rogerbot
SBIder SBIder
SEOkicks-Robot SEOstats
SEOlyticsCrawler Searchestate
Scanbot
Scrapy
ScrepyBot
SearchmetricsBot
Seekport
SemanticJuice
Semrush Semrush
SemrushBot
Shodan Shodan
Siphon SiteCheckerBotCrawler
SiteExplorer
SiteSucker SiteSucker
SiteLockSpider SlySearch
Sitebeam SmartDownload
Siteimprove Snoopy
Snapbot SocialRankIOBot
Sociscraper
Sogou web spider Sogou web spider
Spammen Sosospider
Sottopop
SpankBot SpankBot
Spanner
SputnikBot
Sqworm
Steeler
Sucker Sucker
Sucuri
SuperBot SuperBot
SuperHTTP
SurveyBot SurveyBot
Suzuran Suzuran
Szukacz Swiftbot
T0PHackTeam T0PHackTeam
Telesoft Teleport
Telesphoreo Titan
Telesphorep
The Intraformant
Toata Toata
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot
TurnitinBot
Typhoeus Typhoeus
URLy Warning URLy Warning
UnisterBot V-BOT
Upflow VB Project
VCI VidibleScraper
Virusdie VoidEYE
Voil Voltron
WISENutbot WISENutbot
WWW-Collector-E WWWOFFLE
Wallpapers/3.0
WallpapersHD
Web Fetch
Web Sauger
Web Auto Web Auto
Web Fetch
Web Fuck
Web Pix
Web Sucker
WebBandit
WebCollage WebCollage
WebCopier WebCopier
WebFetch
WebFuck
WebImageCollector
WebLeacher WebLeacher
WebPix
WebReaper
WebSauger
WebStripper
WebSucker WebSucker
WebWhacker
Webalta Webalta
Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester Whacker
Who.is Bot Whatweb
Wonderbot WiseGuys Robot
Woobot Wprecon
Wotbox
Xaldon WebSpider Xaldon WebSpider
YoudaoBot YoudaoBot
Zade Zade
Zeus Zeus
ZoomBot ZoominfoBot
adscanner ZumBot
archive.org_bot ZyBorg
arquivo.pt
autoemailspider autoemailspider
clark-crawler check1.exe
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
demandbase-bot
eCatch eCatch
instabid facebookscraper
magpie-crawler gopher
mediawords heritrix
ips-agent
isitwp.com
lwp-request
lwp-trivial
meanpathbot
netEstate NE Crawler
oBot oBot
polaris version
probe-image-size probe-image-size
ripz scalaj-http
s1z.ru scan.lol
satoristudio.net seocompany.store
seostar
serpstatbot serpstatbot
sexsearcher sitechecker.pro
sogouspider siteripz
sp_auditbot spyfu
voyagerx.com tAkeOut
webgains-bot webpros.com
webmeup-crawler webprosbot
x22Mozilla x09Mozilla
xpymep1.exe zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
cartierreplica.top angkortours.vn
christianlouboutinoutlet.win deanmoore.ie
cienum.fr div.as
heygidday.biz gobongo.info
hobbyhuren24.net mezaruk.info
lolnex.us promoforum.ru
pufip.com shop.xz618.com
reklama-i-rabota.ru solution4u.com
scrnet.biz.ua strangeduckfilms.com
website-datenbank.de strfls.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com
00webcams.com 00webcams.com
0lovespells0.blogspot.com 03p.info
101raccoon.ru 0c47f8422d3f.com
123locker.com 1008.su
14b.info 100dollars-seo.com
404.mn 100searchengines.com
66cpwgln.space 123cha.com
7a2rnuey1tw9ar.ru 1adult.com
7makemoneyonline.com 1stat.ru
7xc4n.com 1xbet4you.com
256bit.by
2drittel.de
2x2fan.ru
3w24.com
4webmasters.org
5000-cotydzien.com
7yue.org
8lufu.com
a.pr-cy.ru a.pr-cy.ru
adclickservice.com aad1f4acb0a373420d9b0c4202d38d94fa.com
adclickthru.net absolutelyanalog.com
adconscious.com abtasty.com
adrunnr.com adktrailmap.com
adtech.fr ads-seo.men
advokateg.ru adsloads.com
afftrack001.com adult-shop.com.ua
aihelen.net adultmeetonline.info
aimayangzhi.com adzpower.com
air-edem.ru aerogo.com
akvamaster.dp.ua afslankpillen2017nl.eu
amazon-seo-service.com ahhjf.com
ameblo.top akusherok.ru
amoi.tn albamargroup.com
amung.us all-number.com
angigreene.com allproblog.com
animal36.com allstatesugarbowl.org
anmysite.com alphaforum.ru
apilinkswiftco-a.akamaihd.net alvtank.se
apispringsmartne-a.akamaihd.net amateurmatch.com
apple.com-cleaner.systems animenime.ru
appmsr.org apessay.com
atmovs.com apiadanaknet-a.akamaihd.net
auto-moto-elektronika.cz apibetweenlinesn-a.akamaihd.net
autodo.info apioasisspacenet-a.akamaihd.net
apisecretsaucebi-a.akamaihd.net
app-ready.xyz
appiq.mobi
apple.com-webbrowsing-security.review
arabgirls.adultgalls.com
arcadepages.com
asophoto.com
asscenihotosidea.blogspot.com
atofilms.com
aussie-prizes.com
aviav.ru aviav.ru
ayodhya.co avon-severozapad.ru
azadnegar.com azarttoto.com
babespcs.com azbuka-mo.ru
babieca.com azlex.uz
bala.getenjoyment.net balois.worldbreak.com
batanga.net bangla-chat-uk-paralud.ga
beachdriveblog.com best-seo-solution.com
beachfix.co bestcoin.cc
berlininsl.com bestdraws.com
best-way.men bestpriceninja.com
bestcalovebracelet.cn betteroffers.review
bestserials.com bget.ru
bestsexyblog.com bigcareer.info
bharatdefencekavach.com bingoporno.com
biglistofwebsites.com biteg.xyz
bitcoin-ua.top bizcheapjerseyswholesalechina.com
blackbabesporn.com
blog.axant.it
bloglag.com
blogos.kz
blogs.rediff.com blogs.rediff.com
bnt-team.com blue-square.biz
bmw.afora.ru
boazpower.com boazpower.com
bookmark4you.com bobba.dzaba.com
bronzeaid-a.akamaihd.net bonzibuddi.com
businesxxl.com book-bets.com
buypuppies.ca bracketsmackdown.com
buzz-porno.info breget74.ru
buzzurl.jp browserprotecter.com
call-of-duty.info btc4u.ru
callmd5map.com btnativenav.com
camsex.xxx-cam.webcam buehne-fuer-menschenrechte.de
cah.io.community
cakesplus.com.au
calc-for-credit.ru
camdolls.com
candycrushshop.com candycrushshop.com
canrioloadm.gq capsderfudd.tk
canvas.gloverid.site carloans.com
carabela.com.do cartierlovestore.com
cartierlove.xyz casinos4dummies.co.uk
cdncash.com cbb1smartlist12.click
cdncash.net cdncash.org
cfacarrosserie74.com cellfun.mobi
changde.58.com cenokos.ru
cheapjerseysa.com centrumcoachingu.com
cheatcode-lita12.rhcloud.com ceotrk.com
chimiver.info christianlouboutinreplica.pw
chocolateslim-en-espana.com christianlouboutinreplica.win
chocoslim.pro chuckguilford.com
ci.ua cialis-samples.com
cigarpass.com clothesforcash.com
citetick.com compliance-fred.top
clean-virus-mac.com compliance-ivan.xyz
clickhype.com compliance-mary.top
collegerentals.com compliance-viktor.top
community.allhiphop.com conocer-sanabria.com
compliance-barak.xyz coprofam.org
concordexoticrentals.com credit-online.ws
construmac.com.mx credit-respect.ru
cookie-law-enforcement-ee.xyz culmimg.pw
copenergo.ru d2jsp.org
cosmeticswomens-womensports.rhcloud.com dailystrength.org
costablue.xyz daneshetabiat.com
country-chic.ru data1.scopich.com
covadhosting.biz decenttools.com
crazyboost.pro depositfiles-porn.ga
customerguru.in
cypernhuset.se
dame-ns.kz
daptravel.com
dcj-nn.ru
deda-moroza-zakaz.ru
defenderxtactical.com
dentalpearls.com.au
design-lands.ru
designdevise.com
detskie-konstruktory.ru detskie-konstruktory.ru
dichvuvesinhhanoi.com dfwu1013.info
diffbot.com diminishedvalueoforegon.com
dignitasdata.se discovertreasure-a.akamaihd.net
discovertreasurenow.com dktr.ru
distver.ru doctormakarova.ru
djihispano.com dolohen.com
djstools.com domainanalyzing.xyz
domaineaneblanc.com dosug-lux.ru
dpihatinh.gov.vn doyouknowtheword-flummox.ml
drupa.com drupa.com
dvr.biz.ua
dvrlists.com
e2click.com
earthmagic.info
easyukraine.com easyukraine.com
ecookna.com.ua ebonyporn.site
efkt.jp ecommerce-seo.org
eets.net
ege-essay.ru
electricwheelchairsarea.com electricwheelchairsarea.com
elektrischezi.canalblog.com
elektrischezigarette1.blog.pl
elektronischezigarette2.mex.tl
elektronischezigarettekaufen1.beeplog.com
elektrozigarette1.dreamwidth.org elektrozigarette1.dreamwidth.org
englishdictionaryfree.com elektrozigarettekaufen1.devhub.com
erotag.com elektrozigaretten1.tumblr.com
estelight.ru eorogo.top
eugenevaultstorage.com erotikchat-24.com
europeanwatches.ru everflora.ru
ezigarettenshop2.myblog.de exhibitionplus.eu
fable.in.ua exonline.info
faithe.top extractorandburner.com
fanrto.com ezigaretteshop2.vefblog.net
fapgon.com f00kclan.de
fickblock18.com fakehandbags.xyz
filesmonster.porn familyholiday.ml
filesvine.com fast-torrent.ru
find1friend.com fatmaelgarny.com
fickenbumsen.net
film-one.ru
filmi-onlain.info
findanysex.com findanysex.com
findthe.pet flagcounter.me
finstroy.net fok.nl
flipper.top folowsite.com
flowersbazar.com forex.osobye.ru
footbalive.org forodvd.com
forminecrafters.ru foto-weinberger.at
fotzen-ficken.com fototravel.eu
franecki.net funtoonez.com
free-gluten.ru fxund.us
free-share-buttons.blogspot.com gael-s.ru
free-share-buttons.top gallerylisting.com
free-social-buttons.xyz gamblingnerd.com
freevpn.space gaming-journal.com
freshdz.com gateway.zscalerone.net
front.ru gavuer.ru
fullfileaccess.com ge0ip.net
fxtips.ru gg.zzyjxs.com
gambarkatabaru.com ghostvisitor.com
game-mmorpg.net girlspicsa.com
gameprimary.com globatur.ru
gate5.co.za golden-praga.ru
generic-pills-online.com gorodservis.ru
getresponse.com greendream.com.ua
ghernnqr.skyrock.com groupmoney.ru
gilsonchiro.xyz grupografico-pilar.com.ar
glall.ru gsasearchengineranker.pw
globetrotting-culture.ru gsmlab.pl
goatse.ru gta-top.ru
golden-catalog.pro guidefs.ru
goodly.pro guitar-master.org
goodnightjournal.com gwebtools.com.br
gotomontenegro.net gz2.bbsoldes.fr
gpms.org.my hamilton.ca
growboxbl.ru harmonyglen.us
habermetre.com hccoder.info
handsandlegs.ru healthcarestore.info
hatedriveapart.com help.tpu.ru
haveinc.xyz high-speed1.net
hchha.com highland-homes.com
helvetia.com.ua
hidemyass.com
highspeed5.net
historichometeam.com historichometeam.com
hitmuzik.ru hit-men.men
homedecoguide.info hledejvshopech.cz
hostcritique.com hollywoodactress.info
hosting-tracker.com home.myplaycity.com
hotcore.info hostnow.men
hotelcrocenzi.sm hotsocialz.com
houdom.net huhn.altervista.org
hoverboard360.at hunterboots.online
https-legalrc.biz hustoon.over-blog.com
iloveitaly.ru i-midias.net.br
ilte.info ideibiznesa2015.ru
importchinacoach-teach.com ie.57883.net
iflycapetown.co.za
iideaidekonkatu.info
ilovevitaly.com
imperia31.ru
import-sales.com
in-tandem.co in-tandem.co
incomekey.net incanto.in.ua
integritylandscapeservices.com intellego.info
interfucks.net investsuccess.org
intermesh.net ivoiretechnocom.ci
iplogger.org james13prix.info
iplusbit.blogspot.co.za jeremyeaton.co
ipostroika.ru job.icivil.ir
iradiology.ru jobgirl24.ru
isitpaleo.info jofucipiku.tk
istmira.ru joy-penguin.com
itbc.kiev.ua kalandranis.gr
itrevolution.cf kareliatobacco.ru
iwanttodeliver.com kartiniresto.com
jav-way.site kenaba.su
javcoast.com keyword-suggestions.com
je7.us kidd.reunionwatch.com
johannesburgsingles.co.za kinocccp.net
johnrobertsoninc.com kndxbkdx.bloger.index.hr
junketjuice.blogspot.com kollekcioner.ru
justdating.online kostenlos-sexvideos.com
kellyonline.xyz kostenloser-sex.com
kerch.site krasnodar.zrus.org
keyhantercume.com
keywordspay.com
kinohit1.ru
kladrus.ru
klosetkitten.com
kolotiloff.ru
krafte.ru
kupit-adenu.ru kupit-adenu.ru
kursy-ege.ru landinez.co
kw21.org lescinq.com
landscaping.center letmacworkfaster.site
letslowbefast.site lider-zhaluzi.kiev.ua
lifehacĸer.com
liffa.ru
limtu.ifmo.ru limtu.ifmo.ru
linkbolic.com linkdebrideur.xyz
linkrr.com lion.bolegapakistan.com
lion.misba.us lion.playtap.us
lisque.batcave.net lockerz.com
live-cam6.info losangeles-ads.com
live-sexcam.tk louboutinbooties.xyz
lmrauction.com lubetube.com
locustdesign.co luckyday.world
lyrster.com m0r0zk0-krava.ru
m.b00kmarks.com mafa-free.com
m5home.ru mail.allnews24.in
mabdoola.blogspot.com make-money-online.com
mac-shield.com maladot.com
mackeeper-land-672695126.us-east-1.elb.amazonaws.com
magiadinamibia.blogspot.com
mamasuper.prom.ua mamasuper.prom.ua
map028.com mansiondelrio.co
marketingvici.com manualterap.roleforum.ru
marmotstore.online manyresultshub-a.akamaihd.net
mebelcomplekt.ru massage-info.nl
medicinacom.ru masterseek.com
merryhouse.co.uk medispainstitute.com.au
metarip.ru menhealed.net
mielec.pl michellblog.online
mirmedinfo.ru militarysale.pro
mmstat.com minharevisao.com
mobifunapp.weebly.com mobstarr.com
mobplayer.net moi-glazki.ru
modnie-futbolki.net monclercoats.xyz
moneymaster.ru monsterdivx.tv
moneytop.ru msk.afora.ru
motherboard.vice.com musezone.ru
mrsdalloways.com mustat.com
mrvideospornogratis.xxx myonigroup.com
mttwtrack.com
mug-na-chas-moscow.ru
mwtpludn.review
mybackgroundlandscape.blogspot.com
mybinaryoptionsrobot.com
myhydros.org
myperiod.club myperiod.club
mysexpics.ru myplaycity.com
navalwiki.info myra.top
netallergy.ru myshopmatemac.com
newsperuse.com mzdish.site
nhl17coinsforps3.gratisblog.biz nalogovyy-kodeks.ru
nina.az nalogovyykodeks.ru
nippon-bearings.ru narkologiya-belgorod.ru
nosecret.com.ua netvouz.com
notfastfood.ru new-apps.ru
nudepatch.net new7ob.com
nullrefer.com nikkiewart.ru
o333o.com nomuos.it
obiavo.ru nottyu.xyz
oktube.ru novgorod.xrus.org
onblastblog.online nvformula.ru
onlinedomains.ru nw-servis.ru
optitrade24.com ofanda.com
orion-code-access.net office-windows.ru
ossmalta.com okel.co
ostroike.org okout.ru
ostrovtaxi.ru oksrv.com
oyster-green.com olgacvetmet.com
pacificair.com omoikiri-japan.ru
pagesense.com onefilms.net
palocco.it onemactrckr.com
partners-ship.pro online-hd.pl
partybunny.ru online-x.ru
onlyforemont.ru
orel-reshka.net
orhonit.com
orsonet.ru
outclicks.net
outrageousdeal-a.akamaihd.net
outshop.ru
p-business.ru
paceform.com
paramountmarble.co.uk
partner-cdn.men
partner-host.men
patol01.pw
paydayloanslocal.com paydayloanslocal.com
paydayonlinecom.com
pdns.download
pekori.to pekori.to
perfectplanned.com perl.dp.ua
pestomou.info picphotos.net
phelissota.xyz
picsearch.com picsearch.com
pisanieprac.info pix-hd.com
pistonclasico.com placid-rounded-coast.glitch.me
plastgranar.nu
playboyfiles.xblog.in playboyfiles.xblog.in
playtap.us plusnetwork.com
pogodnyyeavarii.gq pobeiranie.pl
pons-presse.com poisk-zakona.ru
popup.matchmaker.com poligon.com
pornophoto.xyz pooleroadmedicalcentre.co.uk
poptool.net
pornobrazzers.biz
pornogad.com
pornotubs.com pornotubs.com
powc.r.ca.d.sendibm2.com portnoff.od.ua
pozvonim.com preconnubial.usuby.site
praisong.net printie.com
pravoholding.ru privat-girl.net
privatamateure.com progress-upakovka.ru
privatbank46.ru pyrodesigns.com.au
prizeestates.cricket qcstrtvt.bloger.index.hr
prod2016.com quebec-bin.com
prostitutki-astana.org quick-offer.com
proxyelite.biz quit-smoking.ga
pukaporn.com rankingchart.de
putevka24.ru
puzzleweb.ru
quickchange.cc
rangjued.com
ratemodels.net
rating-casino2021.ru
razleton.com
razorweb-a.akamaihd.net
rcb101.ru rcb101.ru
redbottomheels.xyz relatodelpresente.com.ar
redirectme.net
registratciya-v-moskve.ru
relayblog.com
restaurantlescampi.com restaurantlescampi.com
rewardpoll.com
rezeptiblud.ru rezeptiblud.ru
riders.ro ricorsogiustizia.org
ring4rhino.com ringporno.com
robotixix.com robertefuller.com
roll123.com rockprogblog.com
royalvegascasino.com rogervivierforsale.com
rozalli.com rome2rio.com
ru-mediaget.ru roznica.com.ua
rubbed.us rucrypt.com
runstocks.com ruex.org.ua
rukino.org
ruscoininvest.company ruscoininvest.company
rutor.vip rusexy.xyz
saltspray.ru russintv.fr
ryetaw.com
saddiechoua.com
saecsa.co
sanatorrii.ru sanatorrii.ru
sankt-peterburg.nodup.ru
satellite.maps.ilovevitaly.com
sbdl.no sbdl.no
scanner-mary.top sbwealthsolutions.ca
scanner-willy.top sc-specialhost.com
school-diplomat.ru scanner-mark.top
searchengineranker.email searchencrypt.com
searchimpression.com see-your-website-here.com
seccioncontrabajo.com serving.adbetclickin.pink
securesmrt-dt.com sexyteens.hol.es
segol.tv sherlock.se
semalt.com shillyourcoins.com
seocheckupx.com
servethis.com
service.adtech.fr
servisural.ru
sex-tracker.com
sexkontakteao.info
sexyali.com
sexyebonyteen.com
sexytrend.ru
sfj-ror.no
shijian.ac.cn
shoesonlinebuy.cn shoesonlinebuy.cn
shop.acim.org
shop.xz618.com
shopfishing.com.ua shopfishing.com.ua
sientalyric.co shymkent.xkaz.org
sildenafil-tadalafil.info
sims-sims.ru sims-sims.ru
sinel.info sitebeam.net
sitevaluation.org sitopreferito.it
sitevalued.com smailik.org
sneakyboy.com smartpet.ru
snowplanes.com snsdeainavi.info
soft-terminal.ru snts.shell-pmr.ru
social-buttons.com
socialbookmarksubmission.org
softxaker.ru softxaker.ru
solmarket.by sotechco.co
space2019.top spb.ru
special-porn.com speechfoodie.com
spidtest.space sptslmtrafms.com
stard.shop spywareit.com
staynplay.net ssconstruction.co
stefanbakosab.se sticken.co
strigkaomsk.ru stmassage.ru
stroydetali.ru stpicks.com
stroyhelp-dv.ru streetfooduncovered.com
strv.se sugarlyflex.pw
sudexpert66.ru surgut.zrus.org
suhanpacktech.com
sukarame.net
superstats.com
swingerseiten.com
taaaak.com taaaak.com
taqplayer.info tam-gde-more.ru
tecnoteakviareggio.it tech4master.com
teencastingporn.com testbotprocessor44.com
teenporn18.net textads.men
telefonsexi.com thaismartloan.com
telegraf.by theallgirlarcade.com
tfxiq.com thegioixekhach.com
the-trader.net timdreby.com
theheroes.ru tizanidinehclsideeffects.blogspot.com
themestotal.com tizanidineorflexeril.blogspot.com
thepornsex.org tizanidinesideeffectsweightloss.blogspot.com
titan-cloud.life tizanidinesleepaid.blogspot.com
tivolibasket.it tmm-kurs.ru
tizanidinenarcotic.blogspot.com top-l2.com
tizanidinepain.blogspot.com torrent-newgames.com
tizanidinerestlesslegsyndrome.blogspot.com tracfone.com
tizanidinewithdrawal.blogspot.com trafficjunky.net
tn811.us trafficmp.com
top-study.work traffictrade.life
top250movies.ru
topads.men
topdownloads.ru
tourismvictoria.com
tradedeals.biz
trkur.com
trucri.me trucri.me
trumpetedextremes.com
tube8.com
tuberkulezanet.ru tuberkulezanet.ru
turizm.bz turbodsp.com
turn-up-life.life ua-company.ru
twitlinks.com ublaze.ru
ua.tc ukkelberg.no
udayavani.com ukr-lit.com
ukkala.xyz uni.me
upupa.net upupa.net
url-img.link ussearche.cf
v720hd.ru uytmaster.ru
valmetrundan.se uzungil.com
velobikestock.com v24s.net
venerologiya.com vduplo.ru
veselokloun.ru vedomstvo.net
vgoloveboli.net venta-prom.ru
videochaty.ru vietimgy.pw
vip-dom.in vigrx-original.ru
vipps.com.my vip2ch.com
vodaodessa.com vipromoffers.com
voloomoney.com volume-pills.biz
vseigru.one
vzglyadriv.kg
vzlom-na-zakaz.com
vzlomfb.com
vzlomsn.org
vzube.com
wallpaperaccess.com
wallpapers-best.com wallpapers-best.com
wallpapersist.com watch-movies.ru
watchmyfb.pl watchinf.com
waterefficiency.co weclipart.com
wdfdocando.com wemedinc.com
webadvance.club wesharepics.info
webcamdevochka.com whatistizanidinehclusedfor.blogspot.com
webscouter.net wholesalecheapjerseysfree.com
webshoppermac.com wholinkstome.com
weprik.ru wineitudes.wordpress.com
wleuaprpxuvr.ga witherrom55.eklablog.fr
woman-h.ru wowas31.ucoz.ru
worldinternetauthority.com www.888.com
worldtraveler.world www.event-tracking.com
www.get-free-traffic-now.com
xblog.in
xblognetwork.com
xboxster.ru
xchangetrak.com
xcombear.ru
xmronta.com
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn----9sbebi2bvzr7h.xn--p1ai xn----9sbebi2bvzr7h.xn--p1ai
xn----ctbigni3aj4h.xn--p1ai xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80aodinpgi.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--oogle-wmc.com xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai xn--b1agm2d.net
xn--l1aengat.xn--p1ai
xportvusbdriver8i.snack.ws xportvusbdriver8i.snack.ws
xtremeeagles.net xtrafficplus.com
xvideosbay.com xwatt.ru
xxxnatelefon.ru yellowfootprints.com
yachts-cruise.info yellowstonevisitortours.com
yebocasino.com yogamatsexpert.com
youradexchange.com youresponsive.com
youtubedownload.org youthreaders.com
yuarra.pluto.ro ypmuseum.ru
zahvat.ru yun56.co
zeikopay.com yur-p.ru
zigarettenonlinekaufen.tumblr.com zarepta.com
znakom.sibtest.ru zazagames.org
znakomstva-moskva77.ru zdesoboi.com
zetmaster.ru
zmoda.hostreo.com
znakomstva-piter78.ru znakomstva-piter78.ru
zrus.org
zskdla.site
zvetki.ru
zvuker.net

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 1314dh.com
108shot.com 34.gs
11235813.webzdarma.cz 4rent.ru
1j7740kd.website 5u.com
2020iscoming.info 91abcw.com
21h2o.com abiente.ru
24x7-server-support.site actulite.com
2kata.ru acunetix-referrer.com
4kepics.com ads-cool.pro
57883.net adultmeetonline.info
adanih.com aibolita.com
adktrailmap.com alta-realestate.com
adrunnr.com amateurgalls.com
alibestsale.com amazingpic.net
amehdaily.com android-systems.ru
amoi.tn api.stathat.com
anonymous-redirect.com applicationg29.com
apartamentwroclaw.eu appsaurus.com
apidigidocketnet-a.akamaihd.net aquarium-pro.ru
apilinkswiftco-a.akamaihd.net arius.tech
app-ready.xyz asiavirtualsolutions.com
appfastplay.com atelielembrancaqueencanta.com.br
appfixing.space atley.eu.pn
baksman.com autorn.ru
bala.getenjoyment.net balkanfarma.ru
balla-boo.se belstaffstore.online
bannerspace.com bgviagrachrx.com
bbsoldes.fr bioca.org
besthomemadepornsites.com bookmaker-bet.com
beyan.host.sk brateg.xyz
bfz.biz breget74.ru
bibys.com brokergid.ru
bjgugu.net.ua brus.city
bkns.vn btvn.ru
blagovest-med.ru burn-fat.ga
blogrankers.com businesxxl.com
boleznikogi.com butstrap.space
bonzbuddy.com buyantiviralwp.com
buynorxx.com buyessaynow.biz
carina-sy.de buzzonclick.com
casablancamanor.co.za bwlx.prepedu.cn
cascadelink.org cascadelink.org
catalogs-parts.com chatroulette.world
cementaresearch.se chiptuninger.com
cfsrating.sonicwall.com christianlouboutinsaleoutletonline.us
chcu.net cialisndbrx.com
checkpagerank.net click2pawn.com
cookingmeat.ru club-musics.ru
cottageofgrace.com coinsspb.com
cpabegins.ru compliance-checker.info
creditnation.ru compliance-elena.top
dailyrank.net compliance-george.xyz
dbmkfhqk.bloger.index.hr counterbot.com
covadhosting.biz
crafthubs.com
creams.makeforum.eu
creativehutindia.com
cutalltheshit.com
cxpromote.com
daooda.com
decorazilla.com decorazilla.com
delayreferat.ru delayreferat.ru
derevesendeco.com dentfidemountpreach.tk
dfwu1019.info detoxmed24.ru
djihispano.com dichvuvesinhhanoi.com
doctissimo.fr digitalassetmanagement.site
domaincrawler.com diusyawiga.tk
dorratex.tn dnepr-avtospar.com.ua
duawitchrarato.tk download-wallpaper.net
enternet.ee dustyorate.com
eropho.com ebonyporn.site
eropho.net econom.co
erotik-kostenlos.net ednorxmed.com
estibot.com efkt.jp
evidencecleanergold.com elektronischezigarette2.mex.tl
extstat.com elektronischezigarettekaufen1.beeplog.com
exct.net
expertblog.info
eyes-on-you.ga eyes-on-you.ga
ezigarettenshop1.yolasite.com family1st.ca
f0815.de fanpagerobot.com
fetlifeblog.com feargames.ru
findclan.org fickluder69.com
flauntyoursite.com filmci.pro
floridahuntingfishingadventures.com firma-legion.ru
flowersbazar.com freshdz.com
frameimage.org gaytube.com
free411games.com geileweiber.tk
freenode.info girlsfuckdick.com
game-mmorpg.net gne8.com
gazobeton-p.com.ua gotomontenegro.net
generalporn.org greendream.com.ua
getyourimage.club greenshop.su
goldandcard.ru gwebtools.com
googlefeud.com hausfrauensex18.com
guardlink.com hdseriale.pl
gz2.bbsoldes.fr healbio.ru
handicapbathtubarea.com hentaiheroes.com
hasshe.com home-task.com
hd-filmy.net ibb.com.ua
help.tpu.ru iblogpress.xyz
hobbyhuren24.net indiakino.net
hotblognetwork.com indianmedicaltourismshop.com
hoverboard360.es isistaylorporn.info
husky-shop.cz ix20.ru
idealtits.net jabimgo.pw
ido3.com james13prix.info
imicrovision.com javcoast.com
impisr.edunsk.ru jrpmakati.com
interferencer.ru kerwinandcariza.com
ipostroika.ru kfon.eu
iqupdatetmz.win kollekcioner.ru
izhevsk.xrus.org kovrenok.ru
javspace.net labafydjxa.su
kamalsinha.com lapitec.eu
kamen-e.ru laptoper.net
keyword-suggestions.com ldrtrack.com
keywordspay.com liffa.ru
kinozapas.org likrot.com
klikbonus.com littleberry.ru
krasnodar.ru liupis.com
krynica.info lkbennettoutlet.online
lacloop.info lodki-pvh.dp.ua
lapolis.it lombardfinder.ru
lida-ru.com lostaruban.com
link.ac lyrics.home-task.com
machicon-akihabara.info magnetic-bracelets.ru
makedo.ru mantramusic.ru
manifestation.betteroffers.review manve.info
marktforschung-stuttgart.com mebelcomplekt.ru
martlinker.com micasainvest.com
matb3aa.com mikrobiologies.ru
maximilitary.ru mmgq.ru
meendo-free-traffic.ga moesen-ficken.com
message-warning.net moneyteam24.com
midst.eu most.gov.iq
missclub.info mozello.ru
mobifunapp.weebly.com
mojowhois.com
monclerjacketsoutlet.win
mowser.com
mrsdalloways.com mrsdalloways.com
musicas.baixar-musicas-gratis.com mvpicton.co.uk
mustat.com myiptest.com
muznachas-service.ru netvouz.com
my-bc.ru nikhilbahl.com
mymobilemoneypages.com novodigs.com
na-telefon.biz offer.camp
narkologiya-peterburg.ru ok-ua.info
narkologiya-voronezh.ru onlineslotmaschine.com
naturalpharm.com.ua optom-deshevo.ru
nofreezingmac.click orgasmatrix.com
nomuos.it ownshop.win
obiavo.kz ozas.net
offf.info pagesense.com
office-windows.ru partnerads.men
oklogistic.ru partnersafe.men
omgtnc.com popugaychiki.com
onlineserialy.ru popunder.ru
ooo-olni.ru pornoforadult.com
ourtherapy.ru pr0fit-b0x.com
pawli.eu predmety.in.ua
pay2me.pl privatefx-in.ru
perm-profnastil.ru prodess.ru
photokitchendesign.com profitsport.club
picquery.com prosmibank.ru
pirateday.ru rapidokbrain.com
pk-services.ru razyboard.com
play-movie.pl real-time-analytics.com
playtap.us recipedays.com
porto.abuilder.net reklamuss.ru
portside.cc remorcicomerciale.ro
poshiv-chehol.ru rezeptiblud.ru
privhosting.com robot-forex.biz
pro-tec.kz rossmark.ru
prointer.net.ua royalvegascasino.com
proprostatit.com rozalli.com
rank-checker.online rukino.org
ranksonic.com russintv.fr
rapevideosmovies.com samo-soznanie.ru
rarbg.to samoiedo.it
realizmobi.com schoolfiles.net
reklama-i-rabota.ru searchlock.com
ru-mediaget.ru security60-e.com
ruf777.com seeingmeerkat.com
runstocks.com selectads.men
screentoolkit.com seoboxes.com
scrnet.biz.ua sexpornotales.net
seemoreresultshu-a.akamaihd.net sexyali.com
seo-platform.com shemale-sex.net
sex-pr.net skypasss.com
shopwme.ru slkrm.ru
si-unique.com socialsignals24.com
site-speed-check.site socialtrade.biz
siteimprove.com solmarket.by
smailik.org soochi.co
soaksoak.ru sotkal.lark.ru
soc-proof.su spidtest.space
solinf.co spravka-medosmotr.ru
songoo.wz.cz startufa.ru
spinnerco.ca stauga.altervista.org
spy-sts.com stoki.ru
staff.prairiesouth.ca strv.se
start.myplaycity.com studiokamyk.com.pl
statistici.ro superstarfloraluk.com
strag-invest.ru tarad.com
stretchmate.net tattoo33.ru
stuff-about-money.com technopellet.gr
summerlinhomes411.info thefarmergame.com
tecnoteakviareggio.it thepantonpractice.co.uk
thebluffs.com ticketsys.inetwd.com
thesmartsearch.net titangel-vietnam.com
thin.me.pn tizanidineinfo.blogspot.com
threecolumnblogger.com tizanidinerecreationaluse.blogspot.com
tizanidinecapsules.blogspot.com tizanidinerestlesslegsyndrome.blogspot.com
tizanidinedruginteractions.blogspot.com tk-assortiment.ru
toloka.hurtom.com trainoffend.ml
tongkatmadura.info tri-slona.org
topads.men triumf-realty.ru
topcasinoratings.ru truck-addzilla.life
tracfone.com trucri.me
trafficgenius.xyz ttrraacckkrr.com
trafficmonetize.org twelvevisionspartyofcolorado.com
trafficstars.com ukr-lit.com
tuberkuleznik.ru ukrobstep.com
tubo360.com unblocksit.es
tvnewsclips.info upstore.me
unitygame3d.com uptime-as.net
varbergsvind.se urlopener.com
vasileostrovsky-rayon.ru usdx.us
vesnatehno.com vavilone.com
vezuviy.su venerologiya.com
vkonche.com vtcdns.com
vladimir.zrus.org webalan.ru
vltai.com wechatdownload10.com
vuclip.com wedding-salon.net
vzlomtw.com westum.se
wallpapersdesk.info wholesalejerseysgaa.com
wcb.su wildnatureimages.com
web-betting.ru x-diesel.info
webcamdevochka.com xep.info
webenlace.com.ar xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
webinstantservice.com xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
worldoffiles.ru xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
x-diesel.biz xn--b1agm2d.net
x-lime.net xsfetish.org
x-rates.ru xxxdatinglocal.us
x-true.info xxxmania.top
xerox-douglas.cf yeartwit.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai youporn-ru.com
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai zatjmuzu.info
xn--d1aifoe0a9a.top zeg-distribution.com
xtube.com zocaparj.kz
xxxtubesafari.com zreloeporno.tv
youtube-downloader.savetubevideo.com
youtubologia.it
zastenchivosti.net
zeroredirect5.com
zscalertwo.net

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.53 0; 10.1.1.32 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2907 ### Version: V4.2022.02.2908
### Updated: Wed Feb 9 10:03:33 UTC 2022 ### Updated: Wed Feb 9 22:01:12 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.02.2907 ### Version: V4.2022.02.2908
### Updated: Wed Feb 9 10:03:33 UTC 2022 ### Updated: Wed Feb 9 22:01:12 UTC 2022
### Bad Bot Count: 636 ### Bad Bot Count: 636
################################################### ###################################################
### Version Information ## ### Version Information ##