vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.08.2540 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-08-21 10:09:47 +00:00
parent cafce28f33
commit 463b186677
47 changed files with 71114 additions and 71104 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2539 #### Version: V4.2021.08.2540
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2539 #### Version: V4.2021.08.2540
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2539 #### Version: V4.2021.08.2540
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8754
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2539 #### Version: V4.2021.08.2540
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8758
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8758
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8758
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8758
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

8758
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

8758
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

8758
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2539 2540

8758
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8758
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.65.113.159 0; 52.188.123.253 0;

8758
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

8758
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

8758
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

8758
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.65.113.159 0; 52.188.123.253 0;

8758
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.65.113.159 0; 52.188.123.253 0;

8758
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Extractor AhrefsBot
Fimap Lanshanbot
Foobot Linkdexbot
Jyxobot Sitebeam
Nibbler WISENutbot
Semrush WWW::Mechanize
SlySearch
SuperHTTP
Web Auto Web Auto
Xaldon WebSpider evc-batch
siteripz
trendiction.com

314
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
80legs 01h4x.com
ADmantX 360Spider
ASPSeek 404checker
Abonti AIBOT
Aboundex AlphaBot
Aboundexbot
AfD-Verbotsverfahren
AiHitBot
Aipbot
AllSubmitter
Anthill Anthill
Apexoo
Attach Attach
BLEXBot AwarioSmartBot
BBBike
BDFetch
BackDoorBot BackDoorBot
BackStreet BackStreet
Badass BackWeb
Battleztar Bazinga Bitacle
BetaBot Boardreader
Blackboard Buck
Bolt
BotALot
Buddy Buddy
BunnySlippers BuzzSumo
CODE87 CODE87
CazoodleBot CheTeam
Cogentbot CheeseBot
Chlooe
Cocolyzebot
Collector
Copier Copier
CopyRightCheck
Copyscape
Cosmos Cosmos
Crawling at Home Project
Crescent
CrunchBot CrunchBot
DTS Agent CyotekWebCopy
Demon DBLBot
DigitalPebble DSearch
Dirbuster Deusu
DittoSpyder Disco
DomCopBot Dispatch
DnyzBot
DomainSigmaCrawler DomainSigmaCrawler
Download Wonder Download Wonder
ECCP/1.0 Drip
EasyDL EMail Wolf
Ebingbong EirGrabber
Ecxi Evil
Exabot Exabot
Express WebPictures Extreme Picture Finder
ExtractorPro EyeNetIE
FDM Ezooms
FHscan FHscan
FemtosearchBot FemtosearchBot
Fimap
FlashGet FlashGet
Flunky Foobot
Freeuploader Freeuploader
Fuzz FrontPage
FyberSpider
GalaxyBot
Genieo Genieo
GetWeb GetWeb
Getintent
Gigablast Gigablast
Gigabot GridBot
Go!Zilla HMView
Go-Ahead-Got-It HTMLparser
Grabber HTTP::Lite
GrapeFX HTTrack
HEADMasterSEO
Haansoft Haansoft
Havij HaosouSpider
Humanlinks Heritrix
Hloader
IDBTE4M
IDBot IDBot
IRLbot
Iblog Iblog
Id-search Image Sucker
IndeedBot
Indy Library
InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
Internet Ninja Internet Ninja
Iria IstellaBot
Iskanie
JamesBOT
JennyBot JennyBot
JetCar
JikeSpider JikeSpider
JustView JustView
Keyword Density Keyword Density
LNSpiderguy
LWP::Simple LWP::Simple
Libwhisker Lanshanbot
Likse Leap
Linkdexbot LibWeb
LinkextractorPro Lightspeedsystems
Lmspider LinqiaMetadataDownloaderBot
Ltx71 LinqiaScrapeBot
MFC_Tear_Sample Lipperhey
MIIxpc MIIxpc
MJ12bot
MQQBrowser MQQBrowser
MSIECrawler
Mail.RU_Bot
Majestic-SEO Majestic-SEO
MauiBot MegaIndex.ru
Meanpathbot
Mediatoolkitbot
Metauri Metauri
MicroMessenger Microsoft Data Access
Mr.4x3 Microsoft URL Control
Musobot Mojeek
Navroad Mojolicious
Morfeus Fucking Scanner
Mozlila
NICErsPRO
NPbot
Nameprotect
NearSite
Needle Needle
Nessus Nessus
Net Vampire NetAnts
NetMechanic
Nettrack
NextGenSearchBot NextGenSearchBot
Niki-bot Nibbler
Nikto
NimbleCrawler
Nimbostratus
Nmap
Nuclei
Octopus
Offline Navigator Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler OpenLinkProfiler
OrangeSpider OrangeBot
PHPCrawl PECL::HTTP
Page Analyzer POE-Component-Client-HTTP
PageScorer
PageThing.com
Pandalytics Pandalytics
Petalbot Pavuk
PictureFinder
Piepmatz Piepmatz
Pixray Pixray
PleaseCrawl
ProPowerBot
ProWebWalker
Psbot Psbot
Pu_iN
Pump
PxBroker PxBroker
RankActiveLinkBot QueryN Metasearch
RankFlex RankActive
RankingBot2 RankingBot
Rankivabot Re-re
RankurBot RealDownload
ReGet
RebelMouse RebelMouse
SBIder RedesScrapy
SEOkicks Ripper
SEOprofiler Rogerbot
SEOstats
SISTRIX SISTRIX
ScanAlert
Scanbot Scanbot
ScoutJet
Scrapy Scrapy
Screaming
ScreenerBot
SearchmetricsBot SearchmetricsBot
SentiBot SemrushBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot SeobilityBot
Shodan Seomoz
Site Sucker SiteExplorer
SiteLockSpider SiteLockSpider
SiteSucker SiteSucker
Siteimprove
SlySearch
SmartDownload
Snake
Snapbot Snapbot
Sociscraper Snoopy
Sosospider Sottopop
SpaceBison SpaceBison
Spammen Spammen
Spbot Stripper
Spinn3r Swiftbot
SputnikBot Szukacz
Sqworm
Sucker
SuperHTTP
Suzuran
T8Abot
Teleport
Telesoft Telesoft
The Intraformant Telesphorep
Thumbor Thumbor
TightTwatBot
Toata
Toweyabot Toweyabot
Tracemyfile
Trendiction
Turnitin Turnitin
TurnitinBot Typhoeus
TwengaBot URLy.Warning
Twice UnisterBot
VCI V-BOT
Vagabondo
VeriCiteCrawler
Virusdie Virusdie
Voltron Voil
WASALive-Bot WASALive-Bot
WEBDAV WEBDAV
WISENutbot
WPScan WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
WallpapersHD
WeSEE WeSEE
Web Auto
Web Collage Web Collage
Web Fuck
Web Sauger
Web Sucker
WebAuto
Web Enhancer Web Enhancer
WebImageCollector Web Fetch
WebLeacher Web Pix
WebReaper
Web Sauger Web Sauger
WebStripper WebAuto
WebSucker WebBandit
WebWhacker WebCollage
WebCopier
WebFuck
WebLeacher
Webalta Webalta
WebmasterWorldForumBot
Website Quester Website Quester
WebsiteExtractor
WebsiteQuester WebsiteQuester
Who.is Bot Webster
Widow Whack
Whacker
WiseGuys Robot
Wonderbot Wonderbot
Wotbox Wotbox
Wprecon Xaldon WebSpider
Xenu Zauba
YoudaoBot
Zade
Zermelo Zermelo
Zeus
Zitebot
ZoomBot
ZumBot ZumBot
ZyBorg ZyBorg
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
backlink-check backlink-check
check1.exe cah.io.community
clark-crawler clark-crawler
coccocbot-web cognitiveseo
eCatch com.plumanalytics
evc-batch crawler.feedback
domainsproject.org
heritrix heritrix
instabid instabid
isitwp.com lwp-trivial
magpie-crawler muhstik-scan
meanpathbot netEstate NE Crawler
mediawords oBot
page scorer
plumanalytics plumanalytics
polaris version polaris version
probe-image-size
s1z.ru s1z.ru
satoristudio.net
scalaj-http scalaj-http
scan.lol scan.lol
seoscanners seoscanners
seostar sexsearcher
serpstatbot
siteripz siteripz
sogouspider
sp_auditbot sp_auditbot
spyfu
sysscan sysscan
tAkeOut trendiction.com
trendiction.de trendiction.de
voyagerx.com ubermetrics-technologies.com
webmeup-crawler webpros.com
x09Mozilla webprosbot
x22Mozilla
zauba.io

286
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -2,249 +2,249 @@
80legs 80legs
ADmantX ADmantX
ALittle Client ALittle Client
ASPSeek
Abonti
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot AllSubmitter
Aipbot AlphaBot
Alexibot
Alligator
Anarchy Anarchy
Anthill Anarchy99
Aspiegel Aspiegel
Attach Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot
BBBike BBBike
BDCbot BLEXBot
BackStreet BackStreet
BackWeb BacklinkCrawler
Badass
Barkrowler Barkrowler
BatchFTP Battleztar Bazinga
BetaBot BetaBot
Black Hole
Blow
BlowFish BlowFish
Bolt
Buddy
Bullseye Bullseye
BuzzSumo BuzzSumo
CCBot
CODE87 CODE87
CSHttp
Calculon Calculon
CensysInspect CazoodleBot
CheeseBot ChinaClaw
Cliqzbot Cocolyzebot
Custo Cogentbot
DIIbot Craftbot
DSearch Crescent
CyotekWebCopy
DBLBot
DataCha0s DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Deusu Deusu
DigitalPebble
Dirbuster
Disco Disco
Discobot Dispatch
Discoverybot
DittoSpyder DittoSpyder
DomainCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot
Download Wonder Download Wonder
Dragonfly Dragonfly
EMail Siphon Drip
EasyDL
Ebingbong Ebingbong
Ecxi EirGrabber
EroCrawler
Evil
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FHscan FHscan
FemtosearchBot
Firefox/7.0 Firefox/7.0
FrontPage Freeuploader
FyberSpider FyberSpider
G-i-g-a-b-o-t Fyrebot
Getintent GT::WWW
Gigablast GalaxyBot
Go!Zilla GetWeb
Go-Ahead-Got-It Gotit
Grabber Haansoft
HEADMasterSEO HaosouSpider
HMView Havij
Humanlinks Humanlinks
InfoNaviRobot HybridBot
IDBTE4M
IDBot
Iblog
Image Fetch
Image Sucker
InfoTekies InfoTekies
InternetSeer Intelliseek
Iskanie Iskanie
JOC Web Spider
Jbrofuzz
JetCar
Jetty Jetty
JikeSpider
Jorgee Jorgee
Keyword Density JustView
Kinza LWP::Simple
Lanshanbot Larbin
Leap LeechFTP
LeechGet
Lftp
LibWeb LibWeb
Libwhisker
LieBaoFast LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkScan LinkWalker
Linkbot
Linkdexbot
LinkextractorPro
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lmspider
Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MSFrontPage
MIDown tool Mag-Net
MSIECrawler Magnet
Majestic SEO
Majestic12
MarkMonitor MarkMonitor
MarkWatch MarkWatch
Masscan Mass Downloader
Mata Hari MauiBot
MeanPath Bot Mediatoolkitbot
Meanpathbot Metauri
MegaIndex.ru Microsoft Data Access
MicroMessenger Moblie Safari
Microsoft URL Control
Mojeek Mojeek
Mojolicious Mojolicious
Mozlila
NICErsPRO
NPbot NPbot
NetSpider Name Intelligence
NearSite
Needle
Nessus
Net Vampire
NetLyzer
NetZIP NetZIP
Netcraft Netcraft
NextGenSearchBot Nettrack
Nibbler Nibbler
Niki-bot Nuclei
Nikto
NimbleCrawler
Nimbostratus
Ninja
Nmap
Nutch Nutch
Offline Navigator Offline Navigator
Openfind OpenLinkProfiler
Openvas OpenVAS
OrangeBot PageAnalyzer
PHPCrawl
PageScorer PageScorer
PageThing.com
Pandalytics Pandalytics
Panscient Panscient
Papa Foto
Pavuk
Petalbot Petalbot
PictureFinder Picscout
Piepmatz
Pixray Pixray
ProPowerBot ProWebWalker
Psbot Pump
PyCurl QueryN Metasearch
RankFlex RankFlex
RankingBot2 RankingBot
RankurBot RankurBot
Re-re ReGet
RealDownload RealDownload
Reaper
RebelMouse RebelMouse
Recorder
RedesScrapy
RepoMonkey
Ripper
RocketCrawler RocketCrawler
Rogerbot
SEOkicks SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOstats
SISTRIX SalesIntelligent
SMTBot ScoutJet
Scanbot ScrepyBot
SearchmetricsBot
Semrush
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Seomoz Seomoz
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer SiteSnagger
SiteLockSpider SmartDownload
SiteSucker Snapbot
Sitebeam
Sitevigil
Snake
Snoopy
SocialRankIOBot
Sociscraper Sociscraper
Sosospider Sogou web spider
Spammen SpaceBison
SpankBot SpankBot
Spbot Spanner
SputnikBot Stripper
Sqlmap
Sucker
Sucuri Sucuri
SuperBot
Surfbot
SurveyBot SurveyBot
T0PHackTeam
T8Abot T8Abot
Telesoft Telesphorep
Telesphoreo Thumbor
TheNomad TightTwatBot
Tracemyfile Titan
Toata
Trendiction Trendiction
Trendictionbot
Turingos Turingos
Turnitin Turnitin
TurnitinBot TwengaBot
UnisterBot URLy Warning
Upflow
VB Project VB Project
VCI Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VeriCiteCrawler
Voil Voil
Voltron
WASALive-Bot WASALive-Bot
WBSearchBot WEBDAV
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE WWWOFFLE
WeSEE Wallpapers/3.0
Web Sucker Web Enhancer
Web Fuck
WebAuto
WebBandit
WebCopier
WebEnhancer WebEnhancer
WebFuck WebFuck
WebGo IS WebGo IS
WebSauger WebPix
WebStripper WebStripper
WebZIP WebZIP
WebsiteExtractor
Webster Webster
Whack
Whatweb
Who.is Bot Who.is Bot
WiseGuys Robot Widow
Wonderbot Wonderbot
Wotbox Xaldon WebSpider
Wprecon Xaldon_WebSpider
YoudaoBot Xenu
Zitebot
ZmEu ZmEu
ZoomBot ZoomBot
ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler backlink-check
autoemailspider
cah.io.community cah.io.community
check1.exe check1.exe
cognitiveseo cognitiveseo
com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler.feedback
crawler4j
demandbase-bot
eCatch eCatch
evc-batch internetVista monitor
ips-agent magpie-crawler
lwp-request page scorer
lwp-trivial
mediawords
polaris version polaris version
probe-image-size
ripz
satoristudio.net satoristudio.net
scalaj-http scan.lol
seobility seoscanners
sexsearcher
sitechecker.pro
siteripz
sp_auditbot sp_auditbot
spyfu spyfu
sysscan
tAkeOut tAkeOut
voyagerx.com
webmeup-crawler
webpros.com
x09Mozilla x09Mozilla
xpymep1.exe
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
adloads.com 1hwy.com
brimstonehillfortress.org beztuberkuleza.ru
dogbreedspicture.net carsdined.org
ecommerce-seo.com findpics.pw
fashion-stickers.ru hamacapty.com
meetlocalchicks.com meetingrainstorm.bid
razyboard.com mielec.pl
talant-factory.ru obiavo.in
txxx.com omgtnc.com
zigzog.ru seocdvig.ru

920
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1008.su 178evakuator178.ru
100dollars-seo.com 1webmaster.ml
123cha.com 20pascals.nl
1xbet4you.com 214jaluwobapef.cf
28n2gl3wfyb0.ru 2ads.co.uk
5elementov.ru 7yue.org
5forex.ru a14download.com
5u.com addbin.men
8gold.com adhome.biz
8kisses.eu adloads.net
9icmzvn6.website ads-cool.pro
aad1f4acb0a373420d9b0c4202d38d94fa.com adultnet.in
acgs.tk advertisingtag.net
adtech.fr afftrack001.com
adult3dgames.com afslankpillen2017nl.eu
adultmeetonline.info alborzan.com
adventureparkcostarica.com alert-jdh.xyz
agardomains.com
ageofclones.com
alevinefesleri.com alevinefesleri.com
alfa-medosmotr.ru algarveglobal.com
all4invest.info allwidewallpapers.com
allbizne.co.ua animebox.com.ua
alyeskaresort.com aosheng-tech.com
amehdaily.com apimossnetinfo-a.akamaihd.net
android-style.com apioutoboxnet-a.akamaihd.net
angel1777.kz appfixing.space
animal36.com aquarium-pro.ru
apartamentwroclaw.eu arate.ru
apccargo.com arkkivoltti.net
apiitsthirteende-a.akamaihd.net artefakct.com
apiluckyleapnet-a.akamaihd.net asia-forum.ru
apimegabrowsebiz-a.akamaihd.net atdedinotuho.tk
apispringsmartne-a.akamaihd.net atovh.local-finders.com
appiq.mobi auto.rusvile.lt
appleid-verification.com autonew.biz
arquivo.pt
artdeko.info
artpicso.com
arvut.org
asdfg.pro
asrvrep-a.akamaihd.net
asscenihotosidea.blogspot.co.za
ausergrubhof.info
autoblogger24.info
autowebmarket.com.ua
aviapanda.ru aviapanda.ru
aviav.co
aviav.ru
aviva-limoux.com aviva-limoux.com
ayakino.net avramstroy.ru
ayanaspa.com avtoarenda.by
babyguns.ru avtointeres.ru
badavit.com.ua azazaporn.com
bankofthewext.com bala.getenjoyment.net
bbtec.net balla-boo.se
beatifullandscape.co banki76.ru
belinka.com.ua bayareaaeroclub.org
bestchoice.cf bestmobilityscooterstoday.com
bestkfiledxd.cf bezsporno.ru
bestofpicture.com bimatoprost-careprost.com
bestwebsiteawards.com bingo8888.com
bitnote.co biographiya.com
bjorkbacken.nu bitcoinremote.com
blackcurranthumidifiers.site bloggers.nl
blacktwhite.com bmw.afora.ru
blagovest-med.ru bmwhighperformers.com
blockety.co boleznikogi.com
bonzybuddy.com bombla.org
bookmark4you.com.biz boo-arts.com
braindaily.xyz bookhome.info
brainxs.us bouda.kvalitne.cz
break-the-chains.com bronzeaid-a.akamaihd.net
burger-imperia.com browsepulse-a.akamaihd.net
business-made-fun.com builtwith.com
buy-loft.ru buketeg.xyz
buyfriend.ru burn-fat.ga
buzz-porno.info c-gainsbourg.com
buzzurl.jp c1.onedmp.com
candelluxsklep.pl calstaterealty.us
candlehandmade.com canrioloadm.gq
capture-room.com carsdined.org
cartierbracelet.xyz cartierjusteunclou.xyz
cartierbraceletsreplica.pw cartierloveringreplica.xyz
cartierlove.xyz cbcseward.com
cartierlovebraceletreplica.xyz cdnanalytics.xyz
cartierreplica.top centrumcoachingu.com
catalogs-parts.com
cheapjerseysa.com
cheapjerseysap.com cheapjerseysap.com
check-this-out-now.online chiblackhawks-jerseys.com
chlooe.com choosecuisine.com
chorus.terakeet.com christianlouboutinsaleoutletonline.us
christianlouboutinreplica.pw cialis-samples.com
cialisndbrx.com cialisovercounteratwalmartusa.com
cicaki.net cigarpass.com
ciproandtizanidine.blogspot.com cinemaenergy-hd.ru
cityadspix.com clean-virus-mac.com
cleanersoft.com cleanmypc.com
clicksor.net clevernt.com
cmd.kz clickbank.net
compiko.info clickprotects.com
compliance-don.top collegeessay19.blogspot.ru
compliance-irvin.top compliance-alexa.xyz
compliance-viktor.top compliance-george.top
concealthyself.com compliance-jessica.top
converse.ddsoldes.fr compliance-julia.top
cookie-law-enforcement-hh.xyz compliance-nelson.top
compliance-willy.top
construmac.com.mx
contextualyield.com
cool-wedding.net cool-wedding.net
cosmetic.donna7753191.ru coolbar.pro
countbertwistdisp26.soup.io coolwallpapers-hd.com
craftinsta.ru cougfan.info
creditnation.ru craftburg.ru
daisye.top crd.clan.su
customsua.com.ua
cypernhuset.se
czeshop.info
dailyfinancefix.com
dailystrength.org
damedingel.ya.ru damedingel.ya.ru
dandiyabeats.in
daneshetabiat.com
daooda.com
dating-app.ru
dating-time-now.com dating-time-now.com
ddlmega.net dear-diary.ru
dealitright.click delayreferat.ru
delitime.info detsky-nabytek.info
deluxedumps.com dichvuvesinhhanoi.com
dev-seo.blog diesel-parts28.ru
devochki.top dikx.gdn
diarioaconcagua.com dneprsvet.com.ua
dieta-personalna.pl documentsite.net
dildofotzen.net domainroam.win
diminishedvalueoforegon.com domainsphoto.com
dinosaurus.site domination.ml
divci-hry.info dwomlink.info
dlya-android.org earncash.com.ua
doeco.ru earthmagic.info
dogclothing.store easyukraine.com
dolohen.com editmedios.com
domaincdn.xyz edwinkonijn.com.au
domik-derevne.ru egovaleo.it
dpihatinh.gov.vn ekaterinburg.xrus.org
droidlook.net ekspertmed.com
drunkenstepfather.com el-nation.com
dvd-famille.com electronicadirect.com
dyshagi.ru elektrozigarette2.webs.com
education-cz.ru elektrozigarettekaufen2.blogse.nl
elektrischezigarette1.onsugar.com elitepcgames.com
elektrozigarette2.wordpressy.pl eloconcream.blogspot.com
elektrozigaretten1.postbit.com englishdictionaryfree.com
elidelcream.weebly.com enhand.se
emediate.eu
en.altezza.travel
erotik-kostenlos.net
erotikchat-24.com erotikchat-24.com
expresstoplivo.ru eupornstar.info
ezigaretteshop.webs.com extremepornos.net
ezofest.sk eyessurgery.ru
f012.de ezigarettenshop2.postbit.com
farmingworm.com farmingworm.com
fashion-stickers.ru
fast-wordpress-start.com
fatfasts-4tmz.com fatfasts-4tmz.com
fearcrow.com favoritemoney.ru
feargames.ru feel-planet.com
fetishinside.com felizporno.com
filesclub.net fenoyl.batcave.net
filmci.pro fertilitetsradgivningen.se
finteks.ru fickkontakte.org
firma-legion.ru film-one.ru
fiuxy.com filmfanatic.com
flashbannernow.com finstroy.net
floridamobilebillboards.com fivedwld.cf
flprog.com forensicpsychiatry.ru
foto-sisek.porngalleries.top forex21.ru
frameimage.org formularz-konkurs.tk
frankofficial.ru forpackningsutveckling.se
free-laptop-reward.com forzeronly.com
freewlan.info foxinsocks.ru
freshsuperbloop.com fr.netlog.com
free-share-buttons.com
free-share-buttons.top
free-stock-illustration.com
freesitetest.com
freeuploader.ml
freewhatsappload.com
fres-news.com
freza-sverlo.ru freza-sverlo.ru
fullgirl.ru friendflnder.com
galleryawesome.com fuckingawesome.com
gamebackyard.com fusoradio.info
gay-file.com fyxabomiw.ru
gaytube.com garciniaxt.us
gcup.ru gazobeton-p.com.ua
geoads.com gdcentre.ru
gepezz.info gigapeta.com
get-seo-domain.com gilsonchiro.xyz
getpopunder.com gk-atlant.info
getresponse.com glcomputers.ru
gigablast.com globalscam.ga
glasof.es
goodhousekeeping.com goodhousekeeping.com
goodly.pro googlemare.com
grtyi.com gosmeb.ru
gta-top.ru gribokstop.com
gwagka.com guildebzh.info
hawaiisurf.com gungamesz.com
hdpixent.com gwebtools.com
healbio.ru hahashka.ru
healmytrauma.info handicapvantoday.com
helicalpile.us hdpixion.com
hermesreplica.win hermesbirkinhandbagoutlets.com
hobbyhuren24.net heygidday.biz
homeart.space hidemyass.com
hosting-tracker.com himazin.info
hotgirlhdwallpaper.com histock.info
hothor.se hit-kino.ru
hotxnights.info hmywwogw.bloger.index.hr
howlongdoestizanidinestayinyoursystem.blogspot.com homegardenlova.com
huhn.altervista.org howopen.ru
hvd-store.com howtoclean.club
humanelydrew.com
hydropump.su hydropump.su
idegenvezeto.eu ibb.com.ua
ideibiznesa2015.ru ibmdatamanagement.co
images.gyffu.com ikearugs.xyz
imagine-ex.co imgata.com
imicrovision.com imk.com.ua
imitex-plus.ru
import-sales.com import-sales.com
impotentik.com
incest-ru.com
infektsii.com
inome.com.ua inome.com.ua
intelhdgraphicsgtdrive6w.metroblog.com int.search.mywebsearch.com
investmac.com intermesh.net
io9.com intimshop-fantasy.ru
ipchicken.com invivo.hu
iplogger.org ipostroika.ru
iqoption.pro iqoption-bin.com
islandminingsupply.wordpress.com istock-mebel.ru
itch.io ivearchenceinflu.cf
iwantmyfreecash.com jacago.com
jagg.info javspace.net
janavibekken.no je7.us
junglenet-a.akamaihd.net jerseyschinabizwholesale.us
kamen-e.ru jjbabskoe.ru
kareliatobacco.ru jo24news.com
joingames.org
justkillingti.me
kaliningrad.zrus.org
kartiniresto.com kartiniresto.com
keyhantercume.com kchaxton.com
keyword-suggestions.com ketrzyn.pl
keywordsking.com keywordsuggest.org
keywordspay.com
keywordteam.net
kidskunst.info
kihi.gdn kihi.gdn
kiinomaniak.pl kinocccp.net
kino-rating.ru kinoflux.net
kinoplen.ru kinomaniatv.pl
kinozapas.org kleine-titten.biz
klejonka.info klin3952.ru
kliksaya.com konkursov.net
kmd-pto.ru krasnodar.ru
kndxbkdx.bloger.index.hr kreativperlen.ch
knigonosha.net krynica.info
konica.kz ks1234.com
kosmetyki.tm.pl kskjco.club
krasnodar-avtolombards.ru ktotut.net
kredit-pod-zalog-krasnodar.ru
kuzinsp.ru kuzinsp.ru
labplus.ru laapp.com
ladov.ru laudit.ru
ladsblue.com law-check-eight.xyz
lancheck.net lazymanyoga.com
laparfumotec.com
law-check-seven.xyz
le-clos-des-alouettes.com
leadwayau.com
leboard.ru leboard.ru
likrot.com lion.conboy.us
lincolntheatre.com littleberry.ru
littlecity.ch littlecity.ch
livejasmin.com littlesunraiser.com
lodki-pvh.dp.ua live-cam6.info
lol-smurfs.com liverpool.gsofootball.com
lonely-mature.com lolnex.us
lonfon.xyz longadventure.com
lovelycraftyhome.com lostfilm-online.ru
low-format.ru louboutinbooties.xyz
lowephotos.info mabdoola.blogspot.com
lyrster.com mafa-free.com
m.b00kmarks.com malwareremovalcenter.com
magicalfind-a.akamaihd.net
manifestyourmillion.com manifestyourmillion.com
manipulyator-peterburg.ru mathgym.com.au
manualterap.roleforum.ru
mebel-vstroika.ru
med-zdorovie.com.ua med-zdorovie.com.ua
medmajor.ru medmajor.ru
mega-bony-2017.pl methodsmarketing.com
mesbuta.info minyetki.ru
message-warning.net mir-limuzinov.ru
metascephe.com mirobuvi.com.ua
metaxalonevstizanidine.blogspot.com mmostrike.ru
microstatic.pl mojaocena.com
mister-shop.com
moi-glazki.ru
monclercoats.xyz
monitorwebsitespeed.com monitorwebsitespeed.com
moroccosurfadventures.com mosdverka.ru
moskva.nodup.ru moskva.nodup.ru
mrwhite.biz motherboard.vice.com
muabancantho.info moviezbonkerssk.cf
mydearest.co mrvideospornogratis.xxx
mylesosibirsk.ru musezone.su
myplaycity.com mydownloadengine.com
myra.top myftpupload.com
na-telefon.biz myhealthcare.com
nailsimg.com mymobilemoneypages.com
naj-filmy24.pl mywallpaper.top
na15.ru
naphukete.ru naphukete.ru
newstaffadsshop.club nash-krym.info
navalwiki.info
new7ob.com
newstraveller.ru newstraveller.ru
nikkiewart.ru nina.az
njpalletremoval.com nomuos.it
no-fuel.org novosibirsk.xrus.org
o-o-8-o-o.com nullrefer.com
obuv-kupit.ru oakridgemo.com
ochistka-stokov.ru offtime.ru
odywpjtw.bloger.index.hr ok-ua.info
oecnhs.info okayimage.com
oivcvx.website onemantrip.com
okel.co online-podarki.com
online-sbank.ru onlinebay.ru
onlineslotmaschine.com osagonline.ru
optom-deshevo.ru osb.se11.ru
outclicks.net ostroike.org
ownshop.cf
oynat.info
oyster-green.com oyster-green.com
panamaforbeginners.com parlament.biz
perfectpracticeweb.com partners-ship.pro
peugeot-club.org pawli.eu
pflexads.com pepperstyle.ru
photorepair.ru perfectplanned.com
phrcialiled.com
picscout.com
pictures-and-images.com pictures-and-images.com
pihl.se picturesify.com
pinup-casino1.ru pinwallpaper.xyz
pixelrz.com pivka.xyz
pl.id-forex.com pix-hd.com
playfortuna-play.ru pizza-tycoon.com
plohaya-kreditnaya-istoriya.ru pl-vouchers.com
pokemongooo.ml plastgran.com
popander.mobi pojdelo.weebly.com
popugauka.ru polcin.de
popugaychiki.com popugaychiki.com
porngalleries.top popunder.ru
porno-dojki.net pornhub-ru.com
porno-home365.com pornoelita.info
porno.simple-image.com.ua pornosemki.info
pornoforadult.com pornozhara.com
pornoinn.com pornstartits.xblog.in
potoideas.us porodasobak.net
powc.r.ca.d.sendibm2.com printie.com
preconnubial.usuby.site privatefx-in.ru
preparevideosafesystem4unow.space privatov-zapisi.ru
pretty-mart.com prize44.com
prizefestival.mobi promoforum.ru
pro-okis.ru prostitutki-novgoroda.org
profit-opportunity.com proxyelite.biz
pronorm.fr pts163.ru
proprostatit.com pulse33.ru
psoriasis-file.trade pussyfleet.com
pst2017.onlinewebshop.net quality-traffic.com
puserving.com qualitymarketzone.com
puzzleweb.ru quebec-bin.com
qor360.com quelle.ru
quickcashlimited.com rankscanner.com
qwarckoine.com ranksonic.org
rankingchart.de rating-bestcasino.com
razvratnoe.org realizmobi.com
referencemoi.com realresultslist.com
refererx.com realting-moscow.ru
registry-clean-up.net relatodelpresente.com.ar
remont-comp-pomosh.ru rentehno.ru
rff-cfal.info resant.ru
rimedia.org reward-survey.net
riralmolamsaca.tk rezeptiblud.ru
robertefuller.com rixpix.ru
rospromtest.ru ruex.org.ua
ruf777.com runstocks.com
rus-teh.narod.ru sa-rewards.co.za
saddiechoua.com sady-urala.ru
sanatorrii.ru safe-app.net
saneyes.com sbf441.com
scalerite.co.za sbwealthsolutions.ca
scanner-george.top scanmarine.info
scanner-mark.top scanner-barak.top
scenicmissouri.us scanner-willy.top
scrapinghub.com scat.porn
se-welding.ru scmor.ilxc.cc
scrnet.biz.ua
se.bnt-team.com
searchtooknow-a.akamaihd.net
security60-e.com
seksotur.ru seksotur.ru
seonetwizard.com seofirmreviewsus.info
seozoom.it sexpartygirls.net
serptehnika.ru sexsaoy.com
servicecenter.co.ua shijian.ac.cn
serw.clicksor.com
sexad.net
shisha-swag.de
shopcheermakeup.info
signoredom.com signoredom.com
silverage.ru simplepooltips.com
simple-share-buttons.com sinestesia.host.sk
simply.net site-analyzer.com
site-speed-checker.site slavkokacunko.de
slkrm.ru slimcdn.com
smart-scripts.com smadihome.com
snip.to
snow.nvr163.com
socialbookmarksubmission.org
socialtrade.biz socialtrade.biz
sodexo.com sockshare.net
songplanet.ru sotkal.lark.ru
spywareit.com speechfoodie.com
stair.registrydomainservices.com spitfiremusic.com
stats-public.grammarly.io sportbetfair.com
stpolice.com spy-app.info
squidoo.com
stariy-baku.com
starwars.wikia.com
strangeduckfilms.com
stroicol.net stroicol.net
stroy-portal22.ru success-seo.com
superoboi.com.ua supervesti.ru
superstats.com svetlotorg.ru
support.nopeas.sk swimpool.ca
surffoundation.nl syndicate.fun
susanholtphotography.com takeflyte.com
svtrd.com td-33.ru
sweepstakes.rewardit.com tdsing.ru
tacbelarus.ru telefonsex-ohne0900.net
td-l-market.ru thegameriders.com
terraclicks.com thepokertimer.com
thefarmergame.com thetardistimes.ovh
thegalerie.eu tilido.com
thenews-today.info titan-ads.life
titangel-vietnam.com tizanidine4mgstreetprice.blogspot.com
tizanidine4mgstreetvalue.blogspot.com tizanidinedosageforsleep.blogspot.com
tizanidinehydrochloride2mg.blogspot.com tizanidineduringpregnancy.blogspot.com
tizanidinevscyclobenzaprine.blogspot.com tizanidinehcl2mg.blogspot.com
tizanidinevssoma.blogspot.com tizanidineinfo.blogspot.com
toolsky.com tizanidinenarcotic.blogspot.com
top-deal.com.pl tonerbox.kz
topsiteminecraft.com tooplay.com
torrentdownloadhub.com totu.us
tourcroatia.co.uk toys.erolove.in
toyota.7zap.com tpu.ru
track-rankings.online track112.site
track2.shop trafficmonetize.org
tradgardspartner.se tramadolandtizanidine.blogspot.com
trafficstars.com
tubeline.biz tubeline.biz
ugguk.online twelvevisionspartyofcolorado.com
unblocksit.es udsgame.online
ufa.xrus.org
ukrtvory.in.ua
unitygame3d.com unitygame3d.com
unlimitdocs.net uptime-delta.net
unpredictable.ga uptimebot.net
url2image.com usbggettwku.ga
utrolive.ru usdx.us
uzungil.com vellings.info
vabasa.inwtrade.com velobikestock.com
vancleefreplica.pw video-hollywood.ru
vashsvet.com video-production.com
vbikse.com videochat.mx
via-energy-order.com videochat.world
video-chat.in villacoloniale.com
videosbox.ru vip-file.com
videospornogratisx.net vod.com.ua
visa-pasport.ru voloo.ru
vkontarkte.com voloomoney.com
vladimir.zrus.org vzlomtw.com
vse-pesni.com wallpaperstock.net
vsexkontakte.net webinstantservice.com
vuclip.com webradiology.ru
vzlomfb.com weekes.biz.tc
wahicbefa31.soup.io
warningwar.ru
wdfdocando.com
weblibrary.win
websiteexplorer.info
weightatraining.com weightatraining.com
welck.octopis.com wesharepics.site
wemedinc.com westum.se
whatistizanidinehclusedfor.blogspot.com wfb.hatedriveapart.com
wholesalejerseyscheapjerseys.us.com
whos.amung.us
whosonmyserver.com whosonmyserver.com
wildcattube.com williamrobsonproperty.com
wnoz.de worldis.me
wordpress-crew.net writersgroup580.web.fc2.com
world-mmo.com www.kabbalah-red-bracelets.com
wosik-dach.service-for-web.de
wstroika.ru
www.event-tracking.com
www.get-free-traffic-now.com
wygraj-skiny.win
wyniki-lista.pl wyniki-lista.pl
x-stars.ru xaijo.com
x5market.ru xblog.in
xfire.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xn----ctbigni3aj4h.xn--p1ai
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--b1adccaf1bzj.xn--p1ai xn--80aikhbrhr.net
xpresscare.ru xn--80aodinpgi.xn--p1ai
xtubeporno.net xn--90acenikpebbdd4f6d.xn--p1ai
xvideosporn.biz xnxx699.com
xvideospornoru.com xnxxandxvideos.com
xxlargepop.com xolodremont.ru
y8games-free.com xrus.org
yebocasino.co.za xuki.us
youjizz.vc xvideosbay.com
yourtemplatefinder.com yadro.ru
youtoner.it yellowads.men
yuweng.info yogamatsexpert.com
zeikopay.com yougetsignal.com
zerocash.msk.ru youtuhe.com
zeroredirect12.com za-fun-offer.com
zetmaster.ru zaim-pod-zalog-krasnodar.ru
zhongwenlink.com
zigarettenonlinekaufen2.bloog.pl
zryydi.com
zverokruh-shop.cz

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
108shot.com 0c47f8422d3f.com
11pikachu.ru 273-fz.ru
1688.com 3w24.com
2drittel.de 4kepics.com
4istoshop.com 51.la
66cpwgln.space 57883.net
72-news.com 6128786.com
actionnooz.com a.frcls.fr
adtology.com a.pr-cy.ru
afslankpillen2017nl.eu acmebtn.ml
alldezservice.kz acunetix-referrer.com
allknow.info adsfresh.men
amanda-porn.ga adtech.fr
adultfriendfinder.com
aghanyna.com
airmaxshoes-2016.com
allwidewallpapers.com
amazon-adsystem.com amazon-adsystem.com
arabgirls.adultgalls.com apartment.ru
artpicso.com appearance-cool.com
atovh.local-finders.com approvedlocal.co.za
auto-moto-elektronika.cz artavenuegardenstudios.com
autobrennero.it asrv-a.akamoihd.net
b-styles.xyz aviav.eu
bannerads.de avon-severozapad.ru
bannerbank.ru backgroundpictures.net
bedandbreakfast.com bestprofits-there.com
belinvestdom.by betune.onlinewebshop.net
bestbuy.ca
bestofupload.info
bestwebsitesawards.com
bif-ru.info bif-ru.info
bingo8888.com bioca.org
blogos.kz blobar.org
boleznikogi.com bnt-team.com
bonky.biz boanonihaca.tk
bonzibuddi.com book-bets.com
brewdom.ru brainsandeggs.blogspot.com
brianhenry.co bum.com.ru
burger-imperia.com cah.io.community
buzz-porno.info callejondelpozo.es
bvlgaribracelet.xyz calvet.altervista.org
candelluxsklep.pl cannibalcheerleader.com
capsderfudd.tk cartierbraceletsreplica.pw
carapics.com censys.io
cartierlove.xyz centrumcoachingu.com
cartierlovebracelet.xyz cheapestjerseys-wholesale.com
cb.iphantom.com cheapmoncler.win
cdncash.org cheapsergiorossi.online
celejihad.info christianlouboutinoutlet.win
cercacamion.it com-onlinesupport.host
cicaki.net communisave.co.za
cienum.fr compucelunlock.net
citysecurity.nu concealthyself.com
cl.s7.exct.net contentsexpress.com
clarithromycin500mg.com coolwallpapers4k.info
cmd.kz
cocolyze.com
compliance-irvin.top
cosmetic.donna7753191.ru cosmetic.donna7753191.ru
cowlmash.com cougfan.info
culmimg.pw countbertwistdisp26.soup.io
customsua.com.ua counterbot.com
dojki365.online cpajunkies.com
dokfilms.net dame-ns.kz
dpihatinh.gov.vn de.zapmeta.com
dprtb.com design-lands.ru
editors.choice6912650.hulfingtonpost.com dispo.de
elektrischezigarette2.onsugar.com djstools.com
elitedollars.com dkvorota.ru
elvel.com.ua domination.ml
etenininrade.ga download-walpaperhd.blogspot.com
etur.ru ednorxmed.com
factorynightclub.com emergencyneeds.org
familyholiday.ml englate.com
fermersovet.ru ez8motelseaworldsandiego.com
fillmewithhappiness.com familienzahnaerzte.com
flashahead.info fanoboi.com
flytourisme.org fanrto.com
forum.doctissimo.fr fastfixing.tech
forum.tvmir.org fdzone.org
free-social-buttons.com filesclub.net
freesitetest.com forum69.info
freshdz.com free-stock-illustration.com
friendflnder.com freefoto.ca
genericlowlatencyasiodriverhq.aircus.com freemaintenancesysforpcandmac.top
goodnightjournal.com frequiry.com
graphics8.info g33.org
greatdealshop.com gay.adultgalls.com
gtopstats.com getmiro.com
handicapvantoday.com girlspicsa.com
hdfreeporno.net gkvector.ru
glavprofit.ru
gogalleryawesome.com
gosarhivrt.ru
gta-club.ru
gyffu.com
himgaws.pw himgaws.pw
hotgirlhdwallpaper.com hitmuzik.ru
housekuba.org homeandhealth.ru
houseofrose.com hoporno.com
hoverboard360.se hostnow.men
hoztorg-opt.ru
hptwaakw.blog.fc2.com hptwaakw.blog.fc2.com
ico.re hunterboots.online
id-forex.com icaseclub.ru
ilikevitaly.com iideaidekonkatu.info
ilovevitaly.com images-graphics-pics.com
imadedinner.net impisr.edunsk.ru
iminent.com inclk.com
inbabes.sexushost.com inome.com.ua
incomekey.net intervsem.ru
investmac.com isotoner.com
investzalog.ru istock-mebel.ru
ivearchenceinflu.cf jagg.info
javrip.net javlibrary.cc
justdating.online jerseychinabizwholesale.com
keki.info jolic2.com
kino-ecran.ru jrcigars.com
konturkrasoty.ru katadhin.co
krasnodar-avtolombards.ru kinobest.pl
kustanay.kz kinogonew.ru
lifehacĸer.com kiwi237au.tk
livingroomdecor.info kmgamex.cf
lonfon.xyz lalalove.ru
m3gadownload.pl languagecode.com
magazin-pics.ru lapitec.eu
magento-crew.net likesuccess.com
maglid.ru limads.men
linkbux.ru
linkredirect.biz
littleberry.ru
littlesexdolls.com
livingcanarias.com
lunamedia.co
lyrster.com
m4ever.net
macdamaged.space
macfix.life
mackeeper-land-672695126.us-east-1.elb.amazonaws.com
mail.allnews24.in mail.allnews24.in
mailemedicinals.com maju.bluesalt.co
mainlinehobby.net max-eclat.men
manyresultshub-a.akamaihd.net metaxalonevstizanidine.blogspot.com
mcadamssupplyco.com mikozstop.com
medicines-choice.com mikrobiologies.ru
medkritika.ru mitrasound.ru
mex-annushka.ru mjchamonix.org
mil-stak.com moonci.ru
mister-shop.com most-kerch.org
mmoguider.ru mtmtv.info
mnimmigrantrights.net musflashtv.com
mobile-appster.ru myftpupload.com
mobilierland.com naturalshair.site
moje-recenze.cz naughtyconnect.com
moz.com needtosellmyhousefast.com
mozello.ru networkad.net
mydownlodablefiles.com nevansk.ru
myhydros.org newtechspb.ru
narkologiya-voronezh.ru nudejapan.net
narosty.com offergroup.info
newfilmsonline.ru ok.ru
nikhilbahl.com okayimage.com
nootrino.com okonich.com.ua
o-o-11-o-o.com
obiavo.kz
okmusic.jp
oktube.ru oktube.ru
onclkads.com omsk.xrus.org
open-odyssey.org online-x.ru
orion-v.com pacificair.com
ossmalta.com panamaforbeginners.com
p-business.ru panicatack.com
park.above.com partnerads.men
pateaswing.com partnerworkroom.men
petrovka-online.com pflexads.com
picmoonco.pw phobia.us
pillscheap24h.com pizza-tycoon.com
pk-pomosch.ru pornhubforum.tk
pl-top.pl pornosexrolik.com
poem-paying.gq pornowarp.info
popugauka.ru pr-ten.de
powitania.pl prizestohandle.club
pozvonim.com
prlog.ru prlog.ru
proftests.net prodvigator.ua
prokotov.com
propranolol40mg.blogspot.com
prostitutki-kiev.org prostitutki-kiev.org
prostitutki-odessa.org
pshare.biz pshare.biz
rada.ru quickbuck.com
rankingchart.de quizzitch.net
ranksignals.com randalljhoward.com
rating-casino2021.ru razamicroelectronics.com
razvratnoe.org razvratnoe.org
restorator-msk.ru redirectingat.com
roll123.com rent2spb.ru
rome2rio.com replicaclub.ru
sape.top resellerclub.com
securityallianceservices.com resultshub-a.akamaihd.net
seobility.net russintv.fr
seocdvig.ru rustic-quiver.win
sex.hotblog.top sabizonline.com
sexblog.pw sashagreyblog.ga
sharebutton.net sayyoethe.blogspot.co.za
sideeffectsoftizanidine.blogspot.com scanner-nelson.top
similardeals.net seccioncontrabajo.com
sitevaluation.org seeresultshub-a.akamaihd.net
sixcooler.de semp.net
skutecznetabletkinaporostwlosow.pl seo18.su
sladkoevideo.com seolab.top
seryeznie-znakomstva.ru
sexvporno.ru
shell-pmr.ru
shitting.pro
shtaketniki.ru
sispe.com.br
slowmac.tech
smartbalanceworld.com
snjatie-geroinovoy-lomki.ru snjatie-geroinovoy-lomki.ru
sobecjvuwa.com.ru softonicads.com
softlinesolutions.me speeddream.xyz
solinf.co spin2016.cf
spaceshipad.com staff.prairiesouth.ca
spinnerco.ca tecnoteakviareggio.it
spy-app.info teenbbw.yopoint.in
stariy-baku.com teesdaleflyballclub.co.uk
sticken.co telefonsexsofort.tk
strana-solnca.ru tfxiq.com
styro.ru thelottosecrets.com
subj.ukr-lit.com thenews-today.info
supermama.top tizanidineinfo.blogspot.com
supermodni.com.ua toolsky.com
svtrd.com torrent-newgames.com
swinon.site track.deriv.com
taqplayer.info trafficstars.com
tds-advert002.info traffixer.com
teastory.co trahvid.com
telefonsexi.com turizmus.us
tgtclick.com uchil.net
thepantonpractice.co.uk unrealcommander.org
thin.me.pn volgograd.xrus.org
timecrimea.ru vtcdns.com
tizanidinewithdrawal.blogspot.com waterefficiency.co
tonivedu.it webads.co.nz
topmira.com woman-orgasm.ru
trafaret74.ru wovis.site
trkur.com wrona.it
tuckermktg.com wrz0iuebwhp5fg.freeddns.com
tupper-shop.ru ws.ampower.me
tvory.predmety.in.ua www.ehscloud.cn
uamtrk.com xingzi-vision.com
upproar.com xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
urccvfmc.bloger.index.hr xn----ctbigni3aj4h.xn--p1ai
usdx.us xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
vedomstvo.net xxlargepop.com
videochat.bz yourporngay.com
vikistars.com yurgorod.ru
vkgaleria.com zeroredirect10.com
vpnmouse.com zirondelli.it
vzlomtw.com
walkme.com
waysbetter.cn
webinstantservice.com
weboptimizes.com
wholesalejerseys-cheapest.com
wildworld.site
wnhjavlhezp.gq
woman-h.ru
works.if.ua
x-true.info
xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn--80abgj3a5acid6ghs.top
xn--80aodinpgi.xn--p1ai
xpresscare.ru
za-music.mymobiplanet.com
zambini.ru
zeroredirect6.com
zetmaster.ru
zvuker.net

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.117 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2539 ### Version: V4.2021.08.2540
### Updated: Fri Aug 20 22:01:34 UTC 2021 ### Updated: Sat Aug 21 10:03:58 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2539 ### Version: V4.2021.08.2540
### Updated: Fri Aug 20 22:01:34 UTC 2021 ### Updated: Sat Aug 21 10:03:58 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##