vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3113 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-18 22:07:18 +00:00
parent 29a51f85de
commit 230f568bcc
47 changed files with 103727 additions and 103717 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3112 #### Version: V4.2022.05.3113
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3112 #### Version: V4.2022.05.3113
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3112 #### Version: V4.2022.05.3113
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12830
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3112 #### Version: V4.2022.05.3113
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12834
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12834
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12834
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12834
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

12834
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

12834
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

12834
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3112 3113

12834
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12834
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.251.118.76 0; 52.165.255.125 0;

12834
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

12834
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

12834
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

12834
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.251.118.76 0; 52.165.255.125 0;

12834
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.251.118.76 0; 52.165.255.125 0;

12834
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 1; 10.1.1.19 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Cloud mapping Asterias
Mail.RU_Bot Chlooe
Mojolicious Copyscape
Netvibes Ebingbong
Sitevigil Image Fetch
Tracemyfile Nessus
Wallpapers Quick-Crawler
Web Enhancer Spanner
crawl.sogou.com Zitebot
seostar seocompany.store

278
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
AIBOT 404enemy
ALittle Client ADmantX
ASPSeek
Aboundexbot Aboundexbot
Alexibot AfD-Verbotsverfahren
AllSubmitter
Alligator Alligator
Anarchy99 Anarchy
Apexoo Apexoo
Aspiegel Atomseobot
Attach Attach
BBBike AwarioRssBot
BDFetch AwarioSmartBot
BackDoorBot BDCbot
BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler Bandit
Bigfoot Barkrowler
BetaBot
Black Hole Black Hole
Blow BlackWidow
BlowFish Blackboard
Bolt Boardreader
BotALot
Buck Buck
BuiltBotTough Buddy
BuiltWith Bullseye
BunnySlippers CSHttp
BuzzSumo
CCBot
CazoodleBot
CensysInspect CensysInspect
CheTeam
CheeseBot
CherryPicker CherryPicker
ChinaClaw
Claritybot Claritybot
Cocolyzebot Cocolyzebot
Collector
Copier
Craftbot Craftbot
Crawling at Home Project
Crescent Crescent
CrunchBot Curious
Custo Custo
CyotekWebCopy DigitalPebble
DIIbot Dirbuster
DatabaseDriverMysqli Discobot
Devil DnBCrawler-Analytics
Dispatch DnyzBot
DomainAppender DomainAppender
DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project Domains Project
Dragonfly EMail Siphon
Ebingbong EasyDL
EirGrabber
Evil Evil
Express WebPictures Exabot
ExtLinksBot ExtLinksBot
Extractor
ExtractorPro
Extreme Picture Finder
EyeNetIE EyeNetIE
FHscan FDM
Fimap
Flunky Flunky
Foobot Foobot
FrontPage FrontPage
GalaxyBot Fyrebot
GermCrawler GetRight
GetWeb
Go!Zilla
GoZilla
Gotit Gotit
GrabNet Grabber
Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot GridBot
HEADMasterSEO HEADMasterSEO
HMView
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Heritrix HaosouSpider
Harvest
IDBot
IRLbot IRLbot
Image Sucker IlseBot
InfoNaviRobot Indy Library
InfoTekies InfoTekies
Intelliseek InternetSeer
Iskanie
JamesBOT
JetCar
JikeSpider
Joomla Joomla
Jorgee Jorgee
Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kinza Kinza
Kozmosbot Kozmosbot
LNSpiderguy LWP::Simple
LeechFTP Larbin
LibWeb LexiBot
Lightspeedsystems Lftp
LinkextractorPro Libwhisker
LinkpadBot Likse
LinkScan
LinkWalker
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
Lipperhey Spider LinqiaScrapeBot
MJ12bot Litemage_walker
MSFrontPage Ltx71
MSIECrawler MFC_Tear_Sample
MIDown tool
Mag-Net Mag-Net
Mail.RU_Bot Majestic-SEO
Majestic12 Majestic12
Mass Downloader Mata Hari
Masscan MauiBot
MeanPath Bot Mediatoolkitbot
Meanpathbot Metauri
MegaIndex.ru
MicroMessenger
Minefield Minefield
Mr.4x3 Mojolicious
Msrabot MolokaiBot
NICErsPRO Morfeus Fucking Scanner
NPbot Mozlila
Net Vampire Musobot
Nikto NearSite
Nessus
NetLyzer
NetZIP
Netvibes
NimbleCrawler
Ninja Ninja
Not Nmap
Nuclei Nuclei
Octopus Octopus
Offline Explorer Offline Explorer
OpenLinkProfiler
OpenVAS OpenVAS
Openfind Openfind
OrangeBot OrangeBot
PHPCrawl OrangeSpider
Pavuk Pandalytics
Picscout PeoplePal
Petalbot
Pi-Monster
Picsearch Picsearch
Psbot Piepmatz
PyCurl PleaseCrawl
Pockey
ProWebWalker
Pu_iN
Pump
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot2 Re-re
RepoMonkey ReGet
Rogerbot RebelMouse
SBIder Recorder
RedesScrapy
Ripper
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats SEOstats
SMTBot ScanAlert
Scanbot Screaming
ScrepyBot ScreenerBot
Searchestate SearchmetricsBot
Seekport Seekport
SemanticJuice SemanticJuice
Semrush
SemrushBot SemrushBot
SeobilityBot Seomoz
Shodan Shodan
Siphon Site Sucker
SiteExplorer
SiteLockSpider SiteLockSpider
Sitebeam Sitebeam
Siteimprove Siteimprove
Sitevigil
SlySearch SlySearch
Snapbot Snapbot
SocialRankIOBot SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider
Sottopop Sottopop
SpaceBison SpaceBison
Spanner Spammen
Spbot
Spinn3r
SputnikBot SputnikBot
Sqlmap
Sqlworm
Steeler
Sucuri
SuperBot SuperBot
SurveyBot SuperHTTP
Suzuran Suzuran
Szukacz Szukacz
T8Abot T0PHackTeam
Telesoft
Telesphorep Telesphorep
The Intraformant
TheNomad TheNomad
Thumbor Typhoeus
Toata URLy.Warning
Twice
URLy Warning
Upflow
VCI VCI
Vacuum
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
VoidEYE VoidEYE
Voil
Voltron Voltron
WASALive-Bot WPScan
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize
Wallpapers/3.0
WeSEE
Web Auto Web Auto
Web Pix Web Fuck
Web Sauger
WebBandit
WebCopier WebCopier
WebEnhancer WebFuck
WebStripper WebImageCollector
WebSucker WebLeacher
WebPix
WebReaper
WebSauger
Webalta Webalta
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Webster Whacker
Whack
Whatweb Whatweb
Widow Widow
WinHTTrack WinHTTrack
WiseGuys Robot WiseGuys Robot
Woobot
Wotbox Wotbox
Xaldon_WebSpider Xaldon_WebSpider
Xenu
Zade
Zauba
Zeus
Zitebot Zitebot
ZmEu ZmEu
adscanner ZoomBot
arquivo-web-crawler
autoemailspider autoemailspider
coccocbot backlink-check
check1.exe
cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j
dataforseo.com dataforseo.com
demandbase-bot demandbase-bot
evc-batch
facebookscraper
gopher gopher
ips-agent
iubenda-radar
linkdexbot linkdexbot
lwp-trivial
magpie-crawler magpie-crawler
meanpathbot
mediawords mediawords
pcBrowser s1z.ru
plumanalytics
polaris version
satoristudio.net satoristudio.net
scalaj-http
seobility seobility
seoscanners seoscanners
sexsearcher
siteripz siteripz
sogouspider
tAkeOut tAkeOut
trendiction.com ubermetrics-technologies.com
voyagerx.com voyagerx.com
webprosbot
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe xpymep1.exe
zgrab

324
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy
ASPSeek ASPSeek
Abonti Abonti
Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren
AhrefsBot
Aipbot
Alexibot Alexibot
AllSubmitter Alligator
AlphaBot
Anarchie Anarchie
Anarchy Anarchy
Anarchy99 Apexoo
Asterias
Atomseobot
AwarioRssBot AwarioRssBot
BackWeb AwarioSmartBot
Backlink-Ceck BLEXBot
BackStreet
BacklinkCrawler BacklinkCrawler
Badass Badass
BetaBot
Bitacle
BlackWidow
Blow Blow
BlowFish BlowFish
BotALot Boardreader
Bolt
Brandwatch Brandwatch
Buddy BuiltBotTough
BunnySlippers Bullseye
BuzzSumo
CCBot CCBot
CSHttp CODE87
Calculon CensysInspect
CazoodleBot
CheTeam
CherryPicker CherryPicker
Chlooe Citoid
Claritybot
Cocolyzebot
Collector Collector
Copier CopyRightCheck
Craftbot Copyscape
Crescent Crescent
CrunchBot CrunchBot
Curious Curious
Custo
CyotekWebCopy CyotekWebCopy
DSearch DBLBot
DatabaseDriverMysqli DatabaseDriverMysqli
Demon
Devil Devil
Digincore Discobot
DigitalPebble
Discoverybot Discoverybot
Dispatch Dispatch
DomCopBot DomainStatsBot
Domains Project Dotbot
EMail Siphon Download Wonder
Dragonfly
EMail Wolf
EasyDL
Ecxi Ecxi
EroCrawler
Evil Evil
Exabot Exabot
ExtLinksBot
Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE Ezooms
FHscan FDM
Fimap FemtosearchBot
Firefox/7.0 Foobot
FyberSpider FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t GalaxyBot
GT::WWW GermCrawler
Getintent Go!Zilla
Gigabot
Go-Ahead-Got-It
Gotit Gotit
GrabNet HTMLparser
Grafula Haansoft
GridBot HaosouSpider
HTTrack
Havij Havij
Heritrix
HybridBot HybridBot
IDBTE4M IDBTE4M
IDBot Id-search
IRLbot
Iblog
Image Fetch
Image Sucker
IndeedBot
InfoTekies InfoTekies
Intelliseek Intelliseek
Internet Ninja Internet Ninja
InternetSeer InternetSeer
Iskanie JamesBOT
JOC Web Spider JennyBot
Jbrofuzz
JetCar
JikeSpider JikeSpider
Joomla JustView
Jyxobot Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine
Kinza Kinza
LWP::Simple Kozmosbot
LNSpiderguy
Lanshanbot
Larbin Larbin
LeechFTP LeechGet
Lftp
LibWeb
Libwhisker
Likse
LinkScan
LinkWalker LinkWalker
Linkbot LinkextractorPro
LinqiaRSSBot LinkpadBot
LinksManager
LinqiaMetadataDownloaderBot
Lipperhey Lipperhey
Lipperhey Spider
Lmspider Lmspider
Ltx71 Ltx71
MJ12bot MFC_Tear_Sample
MQQBrowser MSIECrawler
Majestic-SEO MTRobot
Mag-Net
Majestic SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mass Downloader
Masscan MauiBot
Mata Hari Mb2345Browser
Meanpathbot
Microsoft Data Access
Minefield
Mister PiX
Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Mozlila
Msrabot Mr.4x3
NICErsPRO
NPbot
Navroad Navroad
Nessus NetAnts
NetSpider NetMechanic
NetZIP Nibbler
Ninja Niki-bot
Nuclei Nmap
Offline Explorer Not
OpenVAS Nutch
OpenLinkProfiler
Openvas
OrangeBot OrangeBot
OutfoxBot PECL::HTTP
PHPCrawl PageScorer
Pandalytics Pandalytics
Panscient Panscient
Pavuk Papa Foto
Petalbot
Pi-Monster Pi-Monster
Picscout
PictureFinder
Piepmatz
Pimonster
PleaseCrawl PleaseCrawl
ProWebWalker ProWebWalker
PxBroker Probethenet
PyCurl Pump
QueryN Metasearch RSSingBot
RankFlex RankActive
RankingBot2 Re-re
Rankivabot
RankurBot
RealDownload RealDownload
RocketCrawler Reaper
SBIder RebelMouse
SalesIntelligent Recorder
RepoMonkey
Rogerbot
SEOlyticsCrawler
SISTRIX
Scanbot Scanbot
Screaming ScrepyBot
SemanticJuice Seekport
SentiBot SemrushBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Shodan
Siphon Siphon
SiteLockSpider Site Sucker
SiteSnagger SiteCheckerBotCrawler
Sitebeam Sitebeam
Siteimprove Siteimprove
SlySearch SmartDownload
Snoopy Snapbot
Sociscraper
Sogou web spider Sogou web spider
Sottopop Sosospider
Spammen Spammen
SpankBot SpankBot
Spanner Spbot
Spinn3r Spinn3r
SputnikBot SputnikBot
Stripper Sqlworm
Sqworm
Sucker Sucker
Sucuri SuperHTTP
SuperBot Swiftbot
Suzuran Teleport
T8Abot Telesoft
TeleportPro Telesphoreo
The Intraformant Telesphorep
Thumbor TightTwatBot
Toata Toata
Toweyabot Trendictionbot
Tracemyfile True_Robot
Turnitin TurnitinBot
TwengaBot TwengaBot
URLy Warning
UnisterBot UnisterBot
Upflow V-BOT
VB Project Virusdie
VelenPublicWebCrawler VoidEYE
Voil WASALive-Bot
Voltron WBSearchBot
WISENutbot
WPScan WPScan
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWWOFFLE
Wallpapers/3.0
WallpapersHD WallpapersHD
Web Auto WeSEE
Web Enhancer
Web Pix
Web Sauger
Web Sucker
WebAuto
Web Collage Web Collage
WebCopier Web Enhancer
Web Fetch Web Fetch
WebReaper Web Pix
WebStripper WebBandit
WebEnhancer
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag WebsiteQuester
WebsiteExtractor Whacker
Webster Widow
Who.is Bot Xaldon WebSpider
WinHTTrack YoudaoBot
WiseGuys Robot
Woobot
Zade
Zauba
Zermelo
Zeus
ZmEu ZmEu
ZoominfoBot ZoominfoBot
ZumBot ZyBorg
archive.org_bot adscanner
arquivo.pt arquivo-web-crawler
autoemailspider coccocbot
backlink-check crawler4j
cognitiveseo dataforseo.com
com.plumanalytics
domainsproject.org domainsproject.org
heritrix eCatch
ips-agent evc-batch
iubenda-radar
lwp-request
lwp-trivial lwp-trivial
magpie-crawler
mediawords mediawords
netEstate NE Crawler netEstate NE Crawler
oBot oBot
page scorer
pcBrowser
plumanalytics plumanalytics
probe-image-size
s1z.ru s1z.ru
scalaj-http
scan.lol scan.lol
seobility spyfu
sexsearcher trendiction.com
siteripz
trendiction.de
voyagerx.com voyagerx.com
webprosbot webgains-bot
webmeup-crawler
webpros.com
x09Mozilla
x22Mozilla
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
amazingpic.net autochoixspinelli.com
buqyxa.rincian.info bupropion-sr-150-mg.us
customergrowthsystems.com cbox.ws
farm26.ru exoclick.com
oneclickfiles.com i-hobot.ru
ric.info kladrus.ru
sprttrack.com searchwebknow-a.akamaihd.net
stockquotes.wooeb.com sexyteens.hol.es
virus-respirators.com slowmacfaster.trade
yoursearch.me xn----ctbigni3aj4h.xn--p1ai

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03p.info 000free.us
0daymusic.org 123any.com
123kuma.com 123kuma.com
123locker.com
18ps.ru 18ps.ru
1webmaster.ml 1hwy.com
2daytrendingnews.com 3dracergames.com
2girls1cup-free.com 4webmasters.com
2pxg8bcf.top 5u.com
4pp13.com aa625d84f1587749c1ab011d6f269f7d64.com
72-news.com aa81bf391151884adfa3dd677e41f94be1.com
98oi.ru
a.frcls.fr
aa9d046aab36af4ff182f097f840430d51.com aa9d046aab36af4ff182f097f840430d51.com
absolute-s.ru accessoires-mode.in
adktrailmap.com adelly.bg
adsfresh.men adrenalin-stk.ru
adviceforum.info ads-seo.men
akita.kz adtech.de
adultmeetonline.info
adverster.com
advokat-grodno.by
adzerg.com
akama.com
alekseevec.ru
alert-fdm.xyz alert-fdm.xyz
alert.scansafe.net alf-img.com
alfa-medosmotr.ru alfa-medosmotr.ru
algerianembassy.co.in ameblo.top
alkoravto.ru animaltoplist.com
amateurlivechat.org anonymizeme.pro
angigreene.com aosheng-tech.com
apartmentbay.ru apiappenableinfo-a.akamaihd.net
api.stathat.com apibatbrowsecom-a.akamaihd.net
apimegabrowsebiz-a.akamaihd.net apimossnetinfo-a.akamaihd.net
apimountainbikei-a.akamaihd.net apparel-offer.com
apioasisspacenet-a.akamaihd.net
appleid-verification.com appleid-verification.com
approved.su aquarium-pro.ru
arabseyes.com aruplighting.com
arate.ru asociatia-tipografilor-transilvania.ro
arcteryxstore.online atleticpharm.org
arendas.net auto-moto-elektronika.cz
arkartex.ru autobrennero.it
as5000.com aviav.org
asdfz.pro avitocars.ru
asscenihotosidea.blogspot.co.za azlex.uz
autoblogger24.info babieca.com
automatic-seo.com badavit.com.ua
aviapanda.ru baltgem.com
avtovolop.ru bannerspace.com
ayodhya.co bbtec.net
backiee.com beachfix.co
bag77.ru begalka.xyz
banan.tv belsetirehafi.tk
bbsoldes.fr bestdooz.com
best-seo-software.xyz bestimagecoollibrary.com
besttorrentknifta.weebly.com bgviagrachrx.com
bestwebsiteawards.com bif-ru.info
bestwrinklecreamnow.com
betonka.pro
bioinnovate.co
bird1.ru bird1.ru
bitcoins-live.ru bitporno.sx
biznesluxe.ru biznesschat.net
blackbabesporn.com blog.koorg.ru
blackle.com board.f00d.de
blavia.00author.com bolegapakistan.com
bloglag.com
bongacams.com
bongiornos.info bongiornos.info
bookmark4you.com boom-celebs.com
bosefux.esy.es brendbutik.ru
btc4u.ru buron.pw
buckspar.gq buy-cheap-online.info
businescoop.men buynorxx.com
businesn.men buzzurl.jp
buy-forum.ru
buypanicdisorderpill.com
buyviagraa.blogspot.com
bvlgariring.xyz bvlgariring.xyz
carrauterie.be cakemediahq.com.au
cartierreplica.win callawaygolfoutlet.online
cctva.tv carfax.com.ua
cezartabac.ro cartierlovestore.com
cfjump.com caveavins.fr
chelnytruck.ru cerev.info
chiblackhawks-jerseys.com cfacarrosserie74.com
chastnoeporno.com
chatroulette.life
chatroulette.si
cheap-pills-norx.com
chinajerseyswholesalecoupons.com
chlooe.com
choice-credit.ru choice-credit.ru
choosecuisine.com cigarpass.com
christianlouboutinreplica.win cinemaenergy-hd.ru
christianlouboutinsaleoutletonline.us clevernt.com
ci.ua
cialis-samples.com
clarithromycin500mg.com
clickhype.com
clicksor.net clicksor.net
clothesforcash.com cmd.kz
comeondog.info compliance-brian.xyz
compliance-ivan.top compliance-viktor.top
compliance-julia.top comsysnet.com
compliance-margo.top coolgramgoods.com
compliance-olga.top copenergo.ru
cookie-law-enforcement-aa.xyz copro.pw
cookie-law-enforcement-ee.xyz
coolbar.pro
cottageofgrace.com cottageofgrace.com
covadhosting.biz counter.yadro.ru
crackguru.tk counterbot.com
cranly.net craftburg.ru
customboxes4less.com crazyboost.pro
cutalltheshit.com crazyprotocol.com
cv.wallhade.co creditwell.ru
cvety24.by cubs-jerseys.us
decorationspcs.com culturevie.info
decorazilla.com cupday.com
delayreferat.ru cutt.us
demenageur.com dafatiri.com
detskie-zabavi.ru daneshetabiat.com
div.as deda-moroza-zakaz.ru
dnsrsearch.com demian.kz
domain-submit.info dentalpearls.com.au
domaincrawler.com dfwu1019.info
domik-derevne.ru diamond-necklace.info
dotmass.top diegolopezcastan.com
downloadkakaotalk.com djonwatch.ru
dlya-android.org
dms-sw.ru
dnepropetrovsk.mistr-x.org
documentsite.net
donna7753191.ru
dood.live
dosugrostov.site
downloads-whatsapp.com
draniki.org
droidlook.net
drugs-no-rx.info drugs-no-rx.info
drugstoreforyou.com dvd-famille.com
e705.net eager-nash.188-93-233-196.plesk.page
eets.net ec-file.info
ekosmetyki.net.pl edge.sharethis.com
electro-prom.com ekspresihati.info
elektroniksigaraankara.info ekto.ee
elektrozigarette2.wordpressy.pl elektrischezi.canalblog.com
empis.magix.net elektrischezigarette1.onsugar.com
encodable.com elektronischezi.livejournal.com
eshop.md elektrozigarettekaufen1.devhub.com
europeanwatches.ru elektrozigaretten1.tumblr.com
event-tracking.com elstal.com.pl
elvel.com.ua
eredijovon.com
evaashop.ru
exchangeit.gq exchangeit.gq
exoneration-project.us expdom.com
expertnaya-ocenka.ru extremal-blog.com
extremepornos.net ezrvrentals.com
extremez.net f1nder.org
extstat.com
f5mtrack.com f5mtrack.com
face.hostingx.eu familyphysician.ru
facialporntube.com fandlr.com
feorina.ru fdzone.org
fertilitetsradgivningen.se feel-planet.com
fickluder69.com ferieboligkbh.dk
filmgo.ru ff30236ddef1465f88547e760973d70a.com
financetip.eu fickenprivat.info
finansov.info fifa-coins.online
findanysex.com figuringmoneyout.com
findsexguide.com financeloan.us
find1friend.com
firstdrugmall.ru firstdrugmall.ru
fishingwholesale.us
fm-upgrade.ru fm-upgrade.ru
fodelsedagspresenter.nu forpostlock.ru
folowsite.com foxydeal.com
forex-indextop20.ru frankofficial.ru
fotoxxxru.com frcls.fr
freenom.link free-share-buttons.com
freeseedsonline.com freshwallpapers.info
freeuploader.ml fun-mobi.pl
froggytube.com gael-s.ru
fugarif.ga gallery.rennlist.com
fungamelands.com gambarkataku.co
fxund.us gameplexcity.com
gamblingnerd.com gaming-journal.com
gamebackyard.com garciniaxt.us
gay-site.store
gaz-voshod.ru
gearcraft.us gearcraft.us
gearsadspromo.club
generalporn.org
gentamicineyedrops.blogspot.com gentamicineyedrops.blogspot.com
get-your-social-buttons.info gilbertbanda.net
getdot.ru globetrotting-culture.ru
gifspics.com google-liar.ru
gorabagrata.ru gorabagrata.ru
gosarhivrt.ru gowreckdiving.com
grand-chlen.ru graphics8.info
gratuitbaise.com gratuitbaise.com
groupmoney.ru growshop.es
gstatey.net gungamesz.com
guardlink.org hackers-crackers.tk
guge.io handicapvansarea.com
gwebtools.com hao123.com
hdimagelib.com hatdc.org
hentaiheroes.com health-medical-portal.info
hermesreplica.pw heroz.fr
hit-kino.ru hetmanship.xyz
hoholikik.club hkladys.com
horoshieokna.com hothot.ru
host-tracker.com
hoste.octopis.com
hostnow.men
hotblognetwork.com
houtings.xyz
hoverboard360.at
hoverboard360.es hoverboard360.es
ideashome.id iamsport.org
ido3.com ico.re
ilo134uloh.com ilikevitaly.com
imagez.co ilte.info
imagine-ex.co imediadesk.com
immigrational.info inboundlinks.win
infodocsportal.com inclk.com
investingclub.ru infogame.name
invitefashion.com intellektmedia.at
io9.com iradiology.ru
itsdp3.com irkutsk.zrus.org
janavibekken.no ixora.pro
japfm.com iyasimasennka.com
jav-way.site jaxcube.info
javidol.site jerseysbizwholesalecheap.com
jmat.cn jerseyswholesalechinalimited.com
jerseywholesalechinabiz.com
jewelryandfiligree.com
jobmarket.com.ua jobmarket.com.ua
joessmogtestonly.com jumptap.com
johannesburgsingles.co.za justdating.online
junketjuice.blogspot.com kamalsinha.com
jus0wil.pp.ua kartiniresto.com
jyvopys.com kashubadesign.ru
kam-dom.ru
kandidos.com
karadene.com
karpun-iris.ru
karusel-market.ru
katadhin.co
kaz.kz kaz.kz
kazinogames.lv kerwinandcariza.com
keywords-monitoring-success.com ketoanhanoi.info
kiprinform.com keywordsuggest.org
kliksaya.com kfon.eu
konoplisemena.com kino-fun.ru
kotaku.com kino-key.info
krasnodar.ru kinogonew.ru
kretpicf.pw kinostorm.net
ladov.ru klin3952.ru
lafriore.ru knigonosha.net
lakomka.com.ua koopilka.com
landoftracking.com koptims.tiu.ru
lankarns.com kovrenok.ru
lapitec.eu kozhniebolezni.com
lapolis.it kupiproday.com.ua
law-check-two.xyz kupit-adenu.ru
lemon-ade.site lacave.ntic.fr
letto.by lacloop.info
lida-ru.com ladsup.com
limtu.ifmo.ru lanasshop.ru
linkwithin.com languagecode.com
lionking-1994.blogspot.com larutti.ru
liran-locks.com lazy-z.com
lizace.com legalrc.biz
lonfon.xyz lego4x4.ru
lookingglassemb.com lennatin.info
lowephotos.info lignofix.ua
mail.allnews24.in lineavabit.it
malwareremovalcenter.com lion.conboy.us
manifestyourmillion.com lirunet.ru
markjaybeefractal.com liveinternet.ru
massage-info.nl ljusihus.se
mcar.in.ua lockerz.com
mcnamaratech.com lombia.co
medicine-4u.org lotto6888.com
medispainstitute.com.au lyrster.com
meendo-free-traffic.ga magazin-pics.ru
megahdporno.net marketingvici.com
mensandals.xyz martlinker.com
migente.com mathgym.com.au
missclub.info mearns-tractors.co.uk
mlf.hordo.win meble-bogart.info
mlvc4zzw.space medanestesia.ru
mmgq.ru medi-fitt.hu
mojowhois.com meetingrainstorm.bid
monclerboots.xyz mega-bony-2017.pl
money-every-day.com megagrabber.ru
montredemarque.nl megavolt.net.ua
morefastermac.trade meltwater.com
most-kerch.org metaxalonevstizanidine.blogspot.com
moviezbonkerssk.cf mhi-systems.ru
mowser.com microsoftportal.net
mp3ringtone.info minecraft-rus.org
mirobuvi.com.ua
mixed-wrestling.ru
mjchamonix.org
mmofreegames.online
mobplayer.net
moneymaster.ru
moz.com
mrpornogratis.xxx
mrvideospornogratis.xxx mrvideospornogratis.xxx
musicpro.monster mundoaberrante.com
my-floor.in.ua muzaporn.com
mybinaryoptionsrobot.com myliveblog.ru
mycaf.it myprintscreen.com
myindospace.com najaden.no
na-telefon.biz narkologiya-penza.ru
natali-forex.com
naturalbreakthroughsresearch.com
nbsproject.ru nbsproject.ru
negociosdasha.com next-dentists.tk
nelc.edu.eg nosecret.com.ua
newtechspb.ru notasprensa.info
nfljerseys.online november-lax.com
nibbler.silktide.com novgorod.xrus.org
nikhilbahl.com
nina.az
njkmznnb.ru
njpalletremoval.com
nodup.ru
notfastfood.ru
nrv.co.za
obiavo.su obiavo.su
obsessionphrases.com oddamzadarmo.eu
odywpjtw.bloger.index.hr oklogistic.ru
offers.bycontext.com one-gear.com
office2web.com online-sbank.ru
old-rock.com
one.net.in
online-zaymy.ru online-zaymy.ru
onload.pw onlineslotmaschine.com
onlyforemont.ru onstrapon.purplesphere.in
optom-deshevo.ru optom-deshevo.ru
orion-v.com owohho.com
ororodnik.goodbb.ru pandroid.co
paramountmarble.co.uk pdn-4.com
partner-pop.men pdns.cz
perfectplanned.com pensplan.com
perm.xrus.org picture-group.com
photosaga.info picturesboss.com
phuketscreen.com pijoto.net
picsfair.com pintattoos.com
pio.polytopesexempt.com piter.xrus.org
pixelrz.com pizdopletka.club
pladform.ru pl-vouchers.com
plastgranar.nu pl.aasoldes.fr
playmsn.com plastjulgranar.se
pogosh.com platinumdeals.gr
pokemongooo.ml pokemon-go-play.online
pony-business.com popup-jdh.xyz
popcash.net porndroids.com
pornogratisdiario.com pornmania.pl
pornosok.ru porno-transsexuals.ru
portside.xyz pornoelita.info
pozvonim.com pornosemki.info
pr-ten.de portadd.men
predominant-invent.tk pr0fit-b0x.com
princeadvantagesales.com priora-2.com
prizefestival.mobi privatefx-in.ru
prizesbook.online prizeestates.cricket
professionalwritingservices15.blogspot.ru pro-poly.ru
prokotov.com producm.ru
promover.org professionaldieselcare.com
prostitutki-belgoroda.org profit-opportunity.com
prostitutki-tyumeni.org prohoster.info
prpops.com promodj.com
pukaporn.com promoheads.com
prospekt-st.ru
prostitutki-tolyatti.org
pst2017.onlinewebshop.net
pulse33.ru
pussyfleet.com pussyfleet.com
putevka24.ru qexyfu.bugs3.com
qualitymarketzone.com raschtextil.com.ua
rabot.host.sk ratemodels.net
rankings-analytics.com razleton.com
rcb101.ru razvratnoe.org
realizmobi.com registrationdomainsite.com
reklama-i-rabota.ru registry-cleaner.net
remont-fridge-tv.ru reimageplus.com
restorator-msk.ru reklamuss.ru
rewardit.com relatodelpresente.com.ar
rosbalt.com.ua remontbiz.ru
rostov.xrus.org renhacklids.tk
rubbed.us rentalcarnavi.info
rusvideos.su rentehno.ru
sarf3omlat.com rfd-split.hr
scanner-alex.top rightenergysolutions.com.au
scanner-julianna.top rumamba.com
scanner-marwin.top runofilms.ru
scanner-olga.top russia-today-video.ru
schlampen-treffen.com s.lollypopgaming.com
scoopquest.com sadaholding.com
scrnet.biz.ua sarafangel.ru
sdrescher.net sardinie.us
search.1and1.com savetubevideo.com
scanner-andrew.top
scanner-barak.top
scanner-elena.top
scanner-nelson.top
scanspyware.net
schalke04fc.info
scrapy.org
seadragonherbery.com
search-error.com
searchkut.com
seasaltwithfood.com seasaltwithfood.com
seeresultshub-a.akamaihd.net seminarygeorgia59.ga
selectads.men seorankinglinks.com
seoboxes.com seorankinglinks.us
seocdvig.ru sex-dating.co
seorankinglinks.xyz sex-tracker.de
sethrollins.net sex.hotblog.top
sex-tracker.com sexkrasivo.net
sexobzor.info sexreliz.com
sexphoto.site sexs-foto.top
shell-pmr.ru sexyali.com
shemale-sex.net sharebutton.to
shitting.pro shisha-swag.de
shopperifymac.com shoesonlinebuy.xyz
sigmund-freud.co.uk sirpornogratis.xxx
sildenafilcitratemed.com siteaero.com
silverage.ru siteonomy.com
simple-image.com.ua sobecjvuwa.com.ru
simul.co sofit-dmd.ru
sindragosa.comxa.com sonata-arctica.wz.cz
sitebeam.net sosdepotdebilan.com
sitechecker.pro space4updating.win
sivs.ru spbchampionat.ru
skuteczna-dieta.co.pl sprttrack.com
skytraf.xyz ssconstruction.co
smadihome.com
small-game.com
socialbutton.xyz
sockshares.tv
soft1.ru
soietvousmaime.fr
sotkal.lark.ru
spinazdrav.ru
sport7777.net
spylog.com
spywarenuker.com
srgwebmail.nl
staff.prairiesouth.ca
star61.de star61.de
starwars.wikia.com statoutlook.info
stathat.com
stevemonsen.com
strana-krasoty.ru strana-krasoty.ru
strongholdsb.ru stretchmate.net
studiokamyk.com.pl strfls.com
surflinksmedical.com stroiminsk.com
surintech.ac.th stromerrealty.com
svetlotorg.ru strongsignal-a.akamaihd.net
svnuppsalaorebro.se sudexpert66.ru
swinger-mobil.net syvertsen-da.no
tanieaukcje.com.pl tandvardshuset.net
tastyfoodideas.com taximytishi.ru
tattooreligion.ru
tds-advert005.info tds-advert005.info
technopellet.gr telefonsexi.com
teencastingporn.com theallgirlarcade.com
teesdaleflyballclub.co.uk thedownloadfreeonlinegames.blogspot.com
terrafootwear.us theendivechronicles.com
thecoolimages.net themestotal.com
theguardlan.com thfox.com
thepokertimer.com timeallnews.ru
thewebsitetemplate.info tizanidinehcl2mgtablet.blogspot.com
thewomenlife.com
titelhelden.eu
tizanidine4mg.blogspot.com
tizanidineformigraines.blogspot.com
tizanidinehclsideeffects.blogspot.com tizanidinehclsideeffects.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com tizanidineoral.blogspot.com
tizanidinestreetprice.blogspot.com tizanidineorflexeril.blogspot.com
tizanidinestreetvalue.blogspot.com todohr.com
tksn.ru topbestgames.com
toloka.hurtom.com topclickguru.com
totu.info totu.info
toyota.7zap.com totu.us
tradgardspartner.se trichizobswiv.agddns.net
trafficgenius.xyz twittruth.com
trafficmonetize.org uchebavchehii.ru
traffixer.com udayavani.com
trustaffs.com ufa.xrus.org
tubeoffline.com ukrtvir.com.ua
tuckpointingmasonrysystems.com ukrtvory.in.ua
tula.howotorg.ru unrealcommander.com
tula.mdverey.ru vacuumscleaner.com
twodollarshows.com vclicks.net
uac.net.au vegascosmetics.ru
ugogo.info vekzdorov.ru
umnovocaminho.com
unitygame3d.com
urlopener.blogspot.com.au
uroki.net
vadimkravtcov.ru
valsalud.com
valuado.com
varikozdok.ru
velen.io
velpanex.ru velpanex.ru
videotuber.ru videochat.world
virus-schutzmasken.de vipromoffers.com
vrotike.ru virus-respirators.com
vzlom-na-zakaz.com vizag.kharkov.ua
web-analytics.date vkak.ru
webextract.profound.net vkmusics.ru
webinstantservice.com vmnmvzsmn.over-blog.com
website-audit.com.ua vpnmouse.com
wemedinc.com vseigry.fun
whengirlsgowild.com vzlomtw.com
white-truck.life wallpaperaccess.com
wildcattube.com webscouter.net
wleuaprpxuvr.ga webshoppermac.com
writersgroup580.web.fc2.com website-speed-check.site
websiteaccountant.de
welcomeauto.ru
wemarketing.se
wetgames.ru
wfb.hatedriveapart.com
wholesalenfljerseys.us.com
wildworld.site
windowssearch-exp.com
wingsofrefuge.net
wjgony.com
wma-x.com
womens-journal.net
wosik-dach.service-for-web.de
wttavern.com wttavern.com
www.event-tracking.com x-diesel.biz
xmnb.net x-rates.ru
xaylapdiendanang.com
xep.info
xerox-douglas.cf
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn----7sbifcamovvfggw9d.xn--p1ai xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn--80aafb2a.xn--p1ai xn----dtbndd4ae7eub.top
xn--h1aakne2ba.xn--p1ai xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--oogle-wmc.com xn--80ajbshivpvn2i.xn--p1ai
xsfetish.org xn--90acenikpebbdd4f6d.xn--p1ai
xxxmania.top xtrafficplus.com
yebocasino.co.za xxxhdvideo.site
yellowstonevisitortours.com yellowads.men
yopoint.in yougotanewdomain.com
youjizz.vc youresponsive.com
yourads.website
youradulthosting.com
yourporn.com
yousense.info
yugk.net yugk.net
z-master.ru
zahvat.ru
zajm-zalog-krasnodar.ru
zakon-ob-obrazovanii.ru
zakonobosago.ru
zarepta.com
zdesoboi.com zdesoboi.com
zeroredirect12.com zebradudka.com
zetgie.com.pl zeleznobeton.ru
zhongwenlink.com
zionstar.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1j7740kd.website 007angels.com
2daytrendingnews.com 00author.com
2x2fan.ru 1008.su
5000-cotydzien.com 214jaluwobapef.cf
883zy.com 2pxg8bcf.top
absolute-s.ru 5-steps-to-start-business.com
add-add.men DomainStatsBot
alborzan.com aab94f698f36684c5a852a2ef272e031bb.com
allknow.info abtasty.com
alphacarolinas.com acads.net
amigobulls.com acheterviagrafr24.com
angel1777.kz addblueoff.com.ua
anti-virus-removal.info adhome.biz
app5.letmacworkfaster.world adktrailmap.com
arabgirls.adultgalls.com aimayangzhi.com
atley.eu.pn air-edem.ru
availit.weebly.com aktivator-windows10.blogspot.com
aviva-limoux.com alert-jdh.xyz
avkzarabotok.info alpharma.net
bang-hotties.com am15.net
bankcrediti.ru angkortours.vn
battleforupdating.site animali.deagostinipassion.it
belsetirehafi.tk asrvvv-a.akamaihd.net
benchmarkcommunications.co.uk auto4style.ru
berrymall.ru autobudpostach.club
best-businessman.ru aversis.net
bestempresas.es awaybird.ru
bestmouthwash.club axbocz.net
bestofpicture.com azartmix.com
bget.ru backiee.com
big-cash.net biznesschat.net
bitcoinpile.com
bitporno.sx
blacktwhite.com blacktwhite.com
blogs.rediff.com bluejays-jerseys.us
bmw-ark.ru carabela.com.do
bonvillan.com carloans.com
braintobe.win cartierreplica.top
bristolhotel.com.ua cartierreplica.xyz
bumskontakte.org casinorewards.info
business-made-fun.com centre-indigo.org.ua
buyessay3.blogspot.ru changde.58.com
buyhoverboard.com charmstroy.info
c-gainsbourg.com
cartierreplica.pw
cdnnetwok.xyz
chatroulette.video
cheatcode-lita12.rhcloud.com
checkm8.com
checkpagerank.net checkpagerank.net
chocolateslim-original.com cleanpcnow.com
cinemaflix.website clipartnew.com
cjmc.info clothesforcash.com
clickbank.net colehaanoutlet.store
compliance-checker.info com-onlinesupport.site
compliance-jess.top coolgramgoods.com
compliance-walt.top copyrightclaims.org
conboy.us cutalltheshit.com
cpabegins.ru dalavia.ru
cranly.net db.speedup.gdn
creatives.adbetclickin.pink delayreferat.ru
d-black.bz dfwu1013.info
dating-app.ru doxyporno.com
deadlinkchecker.com
design-lands.ru
detmebel.su
devochki.top
dickssportinggoods.com
divatraffic.com
djekxa.ru
domainroam.win
dorratex.tn
drillsaw.ru
drunkenstepfather.com drunkenstepfather.com
electronicadirect.com elektir.ru
engines-usa.com elektrozigarette2.webs.com
englishgamer.com eloxal.ru
excitacion.info erotik0049.com
executehosting.com erotische-geschichten-xxl.com
exoclick.com european-torches.ru
ezigarettenkaufen2.dreamwidth.org europeanwatches.ru
ezigaretteshop2.mywapblog.com extlinks.com
ezofest.sk eyes-on-you.ga
f-online.de factorynightclub.com
falopicm.pw faithe.top
filmi-onlain.info fakehandbags.xyz
formularz-konkurs.tk fashion.stellaconstance.co
fba-mexico.com
filesclub.net
forex-procto.ru
forum.doctissimo.fr
franklinfire.co
free-stock-illustration.com free-stock-illustration.com
freejabs.com freeseedsonline.com
gallery.rennlist.com freshberry.com.ua
gay-file.com fuckmill.com
gazoblok.net.ua fullfileaccess.com
generousdeal-a.akamaihd.net gabeshop.ru
gettpromos.com galaxyflowers.ru
gogps.me gobongo.info
gowreckdiving.com goldandcard.ru
gtopstats.com gosmeb.ru
gymi.name guiadeserraazul.com
happy.new.yeartwit.com guod.me
hawaiielectriclight.com hatedriveapart.com
hawaiisurf.com hdapp1008-a.akamaihd.net
hifidesign.ru heygidday.biz
himgaws.pw histock.info
home-task.com hollywoodactress.info
home.myplaycity.com homedecorpicture.us
honyaku.yahoofs.jp hop.clickbank.net
hosting-tracker.com hotsocialz.com
hotloans.ru htmlcorner.com
houdom.net hully.altervista.org
humbmosquina.tk ideoworld.org
hunterboots.online iklysha.ml
imagine-ex.co imgata.com
imitex-plus.ru iminent.com
imk.com.ua in-tandem.co
inome.com.ua infostatsvc.com
iplusbit.blogspot.co.za innoslicon.com
irkutsk.zrus.org insider.pro
itservicesthatworkforyou.com intellekt21.ru
jerseyswholesalechinalimited.com investsuccess.org
johannesburgsingles.co.za jagg.info
kamalsinha.com janerikholst.se
ketoanhanoi.info je7.us
kinofak.net jovencitas.gratis
kolotiloff.ru kakablog.net
konoplisemena.com keyhantercume.com
kovrenok.ru kino-filmi.com
kurbappeal.info kinoplen.ru
letto.by laptoper.net
lexixxx.com letolove.ru
lifebyleese.com lifeinsurancekb.com
linkrr.com mackeeper-land-672695126.us-east-1.elb.amazonaws.com
lisque.batcave.net makedo.ru
lonely-mature.com maximpartnerspr.com
longgreen.info medizinreisen.de
lovelycraftyhome.com menstennisforums.com
maladot.com mex-annushka.ru
mansparskats.com missvietnam.org
medtherapy.ru mncrftpcs.com
megavolt.net.ua mnogabukaff.net
metalonly.info monclerjacketsoutlet.pw
meteocast.net moncleroutletonline.win
minecraft-rus.org mossmesi.com
mini-modus.ru mrinsidesales.com
misslike.ru mulberryoutletonlineeu.com
morlat.altervista.org mustat.com
mrwhite.biz my-aladin.com
msk-diplomat.com
mtmtv.info
my-bc.ru
mycaf.it mycaf.it
mycouponizemac.com mydirtyhobby.com
myhydros.org mygameplus.com
myperiod.club myghillie.info
myphotopipe.com myhitmp3.club
naturalshair.site mylesosibirsk.ru
news-readers.ru mymobilemoneypages.com
nikhilbahl.com mypornfree.ru
njkmznnb.ru myseoconsultant.com
nudepatch.net narutonaruto.ru
o-o-6-o-o.ru navalwiki.info
o-o-8-o-o.ru nfvsz.com
obiavo.ru no-rx.info
oralsexfilme.net normalegal.ru
ourtherapy.ru nucia.biz.ly
paleohub.info obsessionphrases.com
panamaforbeginners.com offer.camp
pcimforum.com okmusic.jp
pdfprof.com online-zaymy.ru
ooo-olni.ru
paperwritingservice17.blogspot.ru
paradontozanet.ru
park.above.com
paydayloanslocal.com
pearlisland.ru pearlisland.ru
photo-clip.ru pelfind.me
picanalyzer.data-ox.com perm-profnastil.ru
phpdevops.com
picturesify.com
pinkduck.ga
pivka.xyz
placid-rounded-coast.glitch.me
pomoc-drogowa.cba.pl pomoc-drogowa.cba.pl
pornosexrolik.com pornmania.pl
privatefx-in.ru pornoblood.com
psvita.ru pornoted.com
puzo2arbuza.ru pro-tec.kz
qcstrtvt.bloger.index.hr pxhdwsm.com
qwertty.net py100.ru
registry-clean-up.net ranking2017.ga
remont-fridge-tv.ru rating-bestcasino.com
retailwith.com razyboard.com
rukino.org realtytimes.com
sack.net rebelmouse.com
scottbywater.com repeatlogo.co.uk
search.alot.com rogervivierforsale.com
semxiu.com rozalli.com
shoppingjequiti.com.br russian-postindex.ru
sitevalued.com sabaapress.com
smartadserver.com saddiechoua.com
socas.pluto.ro sbtdesign.co.uk
space4update.pw scanner-jack.top
spammen.de seoexperimenty.ru
specialfinanceoffers.com service.adtech.fr
staff.prairiesouth.ca sexuria.net
strangeduckfilms.com sfd-chess.ru
stretchingabuckblog.com shopperifymac.com
stroimajor.ru sigmund-freud.co.uk
suggest-keywords.com simplepooltips.com
sukirgenk.dvrlists.com skladvaz.ru
surfbuyermac.com sluganarodu.ru
svbur.ru slutloadlive.com
svolze.com snow.nvr163.com
swimpool.ca socialbuttons.xyz
t-bygg.com soda.media
taraz.xkaz.org spaceshipad.com
telegraf.by statustroll.com
theendivechronicles.com sygraem.com
thexart.club tarad.com
titelhelden.eu tds-advert002.info
tizanidine4mgtablets.blogspot.com telefonsex-ohne0900.net
tizanidinemusclerelaxant.blogspot.com templates.radiodigital.co
terrafootwear.us
tfxiq.com
thegalerie.eu
thenews-today.info
thetoiletpaper.com
tizanidinefibromyalgia.blogspot.com
tizanidineonline.blogspot.com
tksn.ru
tn811.us
tomck.com tomck.com
torrent-newgames.com top10-way.com
track-rankings.online topflownews.com
trainoffend.ml torrents-tracker.com
tupper-posuda.ru tour-line.net
uac.net.au trackzapper.com
ukrtvir.com.ua truemfilelj.gq
undergroundcityphoto.com tuberkulezanet.ru
uptime-alpha.net twiclub.in
vasileostrovsky-rayon.ru twincitiescarservice.com
vellings.info unrealcommander.org
vertaform.com utiblog.fr
vesnatehno.com vegascosmetics.ru
videooko.weebly.com vektorpress.ru
vip-file.com vmnmvzsmn.over-blog.com
vkmusics.ru vrotike.ru
vysigy.su vzlomtw.com
vzlomfb.com wapsite.me
vzubah.com watchmygf.net
wallpaperstock.net webmonetizer.net
warning.or.kr wineitudes.wordpress.com
watch-movies.ru worst-sites.online
waterpurifier.club ww2awards.info
webix.biz x-lime.com
weboptimizes.com xaijo.com
whos.amung.us xfluro.com
womensterritory.ru xn----ctbigni3aj4h.xn--p1ai
www.kabbalah-red-bracelets.com xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
x-lime.net xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai xnxx699.com
xn--80aaagvmjabrs1aoc9luc.xn--p1ai xvideospornoru.com
xn--80atua3d.xn--p1ai xxlargepop.com
xn--b1addnj3cah.xn--p1ai zaim-pod-zalog-krasnodar.ru
xportvusbdriver8i.snack.ws zambini.ru
yoursearch.me
za-fun-offer.com
zeleznobeton.ru zeleznobeton.ru
zeroredirect.com zootoplist.com
zigarettenonlinekaufen2.drupalgardens.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.232 0; 10.1.1.19 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3112 ### Version: V4.2022.05.3113
### Updated: Wed May 18 10:03:38 UTC 2022 ### Updated: Wed May 18 22:01:23 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3112 ### Version: V4.2022.05.3113
### Updated: Wed May 18 10:03:38 UTC 2022 ### Updated: Wed May 18 22:01:23 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##