vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2628 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-03 10:11:29 +00:00
parent df0d9c4ce0
commit 2ee364c653
47 changed files with 85699 additions and 85709 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2627 #### Version: V4.2021.10.2628
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2627 #### Version: V4.2021.10.2628
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2627 #### Version: V4.2021.10.2628
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10578
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2627 #### Version: V4.2021.10.2628
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10582
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10582
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10582
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10582
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 1; 10.1.0.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 0; 10.1.0.84 0;
127.0.0.1 0; 127.0.0.1 0;

10582
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 1; 10.1.0.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 0; 10.1.0.84 0;
127.0.0.1 0; 127.0.0.1 0;

10582
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 1; 10.1.0.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 0; 10.1.0.84 0;
127.0.0.1 0; 127.0.0.1 0;

10582
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2627 2628

10582
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10582
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.175.254.190 0; 51.141.163.164 0;

10582
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 1; 10.1.0.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 0; 10.1.0.84 0;
127.0.0.1 0; 127.0.0.1 0;

10582
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 1; 10.1.0.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 0; 10.1.0.84 0;
127.0.0.1 0; 127.0.0.1 0;

10582
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 1; 10.1.0.84 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 0; 10.1.0.84 0;
127.0.0.1 0; 127.0.0.1 0;

10582
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.175.254.190 0; 51.141.163.164 0;

10582
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.175.254.190 0; 51.141.163.164 0;

10582
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 1; 10.1.0.84 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
CopyRightCheck BunnySlippers
Cosmos Express WebPictures
Fyrebot EyeNetIE
G-i-g-a-b-o-t Fimap
MQQBrowser Joomla
Metauri JustView
ProPowerBot Magnet
RankingBot Titan
ScrepyBot Voil
Web Collage Zermelo

284
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker ALittle Client
404enemy Aboundex
Aboundexbot
Acunetix Acunetix
AhrefsBot AfD-Verbotsverfahren
AllSubmitter
Alligator Alligator
Anarchie Anarchie
Anarchy99 Ankit
Anthill Anthill
Apexoo AwarioRssBot
Aspiegel AwarioSmartBot
Asterias
BDCbot BDCbot
BLEXBot BDFetch
BackWeb Backlink-Ceck
BacklinkCrawler Bigfoot
Badass
Bandit
Barkrowler
BatchFTP
Battleztar Bazinga
Bitacle Bitacle
Black Hole
Blow Blow
Boardreader BlowFish
Brandprotect Brandwatch
Buck Buck
BuiltWith
Bullseye Bullseye
BunnySlippers BuzzSumo
CATExplorador
CCBot CCBot
CazoodleBot Cegbfeieh
CensysInspect CensysInspect
CherryPicker CheeseBot
Claritybot Cloud mapping
Cogentbot Cocolyzebot
CopyRightCheck Collector
Copyscape
Cosmos Cosmos
Craftbot
Crawling at Home Project
CrazyWebCrawler
CrunchBot CrunchBot
Custo Custo
DIIbot
DSearch DSearch
Demon Deusu
Digincore Digincore
DigitalPebble
Dirbuster Dirbuster
Disco Discobot
DnyzBot DnyzBot
DomCopBot
DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Domains Project
Dotbot
Dragonfly
Drip Drip
EMail Siphon
EMail Wolf EMail Wolf
Express WebPictures Evil
Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE EyeNetIE
Ezooms Ezooms
FDM Foobot
Fimap FrontPage
G-i-g-a-b-o-t Fuzz
GermCrawler Go-Ahead-Got-It
GetWeb Gotit
Getintent
Gigabot
GoZilla
GrabNet GrabNet
GridBot Grafula
GrapeFX
GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HTTrack HTMLparser
HTTP::Lite
Haansoft
HaosouSpider HaosouSpider
Harvest
Havij Havij
Humanlinks Hloader
IRLbot IDBTE4M
Image Sucker IlseBot
IndeedBot IndeedBot
InfoNaviRobot InfoNaviRobot
InfoTekies Internet Ninja
JOC Web Spider Iria
Joomla Iskanie
IstellaBot
JennyBot
JetCar
JikeSpider
Jorgee
Jyxobot
Keyword Density
Kinza Kinza
LNSpiderguy
Leap
LeechGet
LibWeb LibWeb
Likse Libwhisker
LieBaoFast
LinkScan LinkScan
Linkbot Lipperhey Spider
LinksManager
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey
Litemage_walker
MIDown tool
MIIxpc MIIxpc
MSIECrawler MSIECrawler
MTRobot
Mag-Net Mag-Net
Mail.RU_Bot Magnet
Majestic SEO
Majestic-SEO
MarkWatch MarkWatch
Masscan Masscan
MauiBot MauiBot
Mb2345Browser Mb2345Browser
Microsoft Data Access MeanPath Bot
Meanpathbot
Microsoft URL Control Microsoft URL Control
Moblie Safari Mister PiX
Mojeek
Mojolicious Mojolicious
Mozlila
Msrabot Msrabot
Navroad Name Intelligence
Needle
Net Vampire Net Vampire
NetZIP NetLyzer
Netcraft Netcraft
Nettrack
Netvibes Netvibes
NextGenSearchBot NextGenSearchBot
Niki-bot Niki-bot
NimbleCrawler Nimbostratus
Nmap
Nuclei Nuclei
Octopus Octopus
Offline Explorer Offline Explorer
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openfind OrangeSpider
PECL::HTTP OutfoxBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer PageAnalyzer
PageGrabber
Pandalytics
Papa Foto Papa Foto
PeoplePal Pavuk
Pimonster Pi-Monster
Pixray Picscout
PleaseCrawl PleaseCrawl
Pockey Pockey
ProWebWalker Probethenet
RSSingBot Pu_iN
Pump
QueryN Metasearch
RankActive RankActive
RankFlex RankingBot2
Rankivabot
RankurBot
Re-re Re-re
ReGet Ripper
RealDownload
Reaper
Recorder
SBIder SBIder
SEOkicks SEOkicks
SEOkicks-Robot
SISTRIX SISTRIX
SalesIntelligent SalesIntelligent
ScanAlert ScanAlert
ScreenerBot Scanbot
ScoutJet
Screaming
ScrepyBot ScrepyBot
SemrushBot SearchmetricsBot
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Shodan
Siphon
Site Sucker Site Sucker
SiteLockSpider SiteSnagger
SiteSucker
Sitebeam Sitebeam
Siteimprove Sitevigil
SlySearch SlySearch
Snake Snake
Snoopy Snoopy
SocialRankIOBot SpankBot
Sogou web spider Spanner
Sottopop
SputnikBot SputnikBot
Sqlmap
Sqlworm Sqlworm
Sqworm Sqworm
Steeler Steeler
Stripper
Sucuri Sucuri
SurveyBot
Suzuran
Swiftbot Swiftbot
Szukacz Szukacz
TeleportPro T0PHackTeam
Teleport
Telesoft Telesoft
TheNomad Telesphorep
TightTwatBot
Titan
Toweyabot
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot
Turnitin Turnitin
Typhoeus TurnitinBot
URLy Warning Twice
UnisterBot URLy.Warning
Upflow Upflow
V-BOT
VCI VCI
Vagabondo Vagabondo
VeriCiteCrawler VelenPublicWebCrawler
Virusdie Virusdie
VoidEYE VoidEYE
WBSearchBot WASALive-Bot
WEBDAV
WISENutbot WISENutbot
WWW::Mechanize WPScan
WWW-Collector-E
WWW-Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Web Collage
Web Auto Web Enhancer
Web Pix
Web Sauger Web Sauger
Web Sucker Web Sucker
WebAuto WebCollage
WebLeacher WebCopier
WebStripper WebFetch
WebWhacker WebPix
WebZIP WebSauger
WebSucker
Webalta Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag Website Quester
WebsiteExtractor WebsiteExtractor
Webster Webster
Whack Whack
Whacker
Who.is Bot
Widow Widow
WinHTTrack
Wotbox Wotbox
Xaldon WebSpider Wprecon
Xaldon_WebSpider
Zauba
Zermelo
Zeus Zeus
ZmEu ZyBorg
autoemailspider arquivo.pt
cah.io.community cah.io.community
clark-crawler clark-crawler
coccocbot-web gopher
dataforseo.com muhstik-scan
demandbase-bot oBot
evc-batch pcBrowser
facebookscraper polaris version
ips-agent s1z.ru
meanpathbot scan.lol
netEstate NE Crawler seoscanners
page scorer
plumanalytics
ripz
seobility
seostar seostar
serpstatbot sitechecker.pro
siteripz
sogouspider sogouspider
trendiction.de sp_auditbot
ubermetrics-technologies.com trendiction.com
voyagerx.com voyagerx.com
webpros.com webpros.com
xpymep1.exe zauba.io

288
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker 404enemy
ADmantX ALittle Client
AIBOT ASPSeek
Aboundex
Acunetix Acunetix
AllSubmitter AfD-Verbotsverfahren
Alligator AiHitBot
AlphaBot Alexibot
Anarchy99 Anarchie
Ankit Ankit
Anthill Apexoo
Aspiegel
AwarioSmartBot AwarioSmartBot
BDFetch
BLEXBot BLEXBot
BackDoorBot Bigfoot
BackStreet BlackWidow
Badass
Bandit
BetaBot
Bitacle
Blackboard Blackboard
Bolt BuzzSumo
Buck
BuiltBotTough
Bullseye
BunnySlippers
CODE87
CSHttp CSHttp
Calculon
CazoodleBot
CensysInspect
CheTeam CheTeam
CheeseBot CherryPicker
Chlooe Chlooe
Cocolyzebot Claritybot
Cliqzbot
Cloud mapping
Copier
CopyRightCheck CopyRightCheck
Copyscape Crescent
Craftbot Curious
CrazyWebCrawler
DSearch
DTS Agent
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon
Devil Devil
Dirbuster DittoSpyder
Disco DnyzBot
DomCopBot DomainAppender
DomainSigmaCrawler
Download Wonder
Drip Drip
ECCP/1.0
EMail Siphon EMail Siphon
EasyDL EMail Wolf
Ebingbong EirGrabber
Ecxi
Evil Evil
Exabot
Extractor
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
Fuzz Fimap
Flunky
Fyrebot Fyrebot
GalaxyBot G-i-g-a-b-o-t
Genieo Genieo
GermCrawler
GetRight GetRight
GetWeb GetWeb
GrabNet Getintent
GridBot Gigabot
Go!Zilla
GrapeshotCrawler
HMView HMView
HTTP::Lite
HTTrack HTTrack
HaosouSpider Haansoft
Hloader Harvest
Humanlinks
HybridBot HybridBot
IDBTE4M
IDBot
IRLbot IRLbot
Id-search
Image Fetch
Image Sucker
InfoNaviRobot InfoNaviRobot
InfoTekies InfoTekies
Intelliseek Intelliseek
InterGET InternetSeer
Internet Ninja Iria
IstellaBot IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT Jbrofuzz
JetCar JikeSpider
Jorgee
JustView JustView
Kenjin Spider
Keyword Density Keyword Density
Kinza
LWP::Simple LWP::Simple
Lanshanbot LexiBot
LeechFTP
LibWeb LibWeb
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkWalker LinkScan
LinkpadBot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot Lipperhey
Lipperhey Spider
Lmspider Lmspider
Ltx71 MIDown tool
MFC_Tear_Sample MJ12bot
MarkWatch MQQBrowser
MTRobot
Mail.RU_Bot
Majestic-SEO
Majestic12
Mass Downloader
Mata Hari Mata Hari
Metauri Meanpathbot
MicroMessenger MicroMessenger
Microsoft URL Control
Mister PiX
Moblie Safari Moblie Safari
Mojolicious Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Navroad
NICErsPRO NearSite
Name Intelligence
Needle Needle
NetAnts Nessus
NetLyzer
NetMechanic NetMechanic
NetSpider NetSpider
NetZIP
Nettrack Nettrack
Nibbler
Niki-bot
Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nuclei
Ninja
Nmap
Offline Explorer
Offline Navigator Offline Navigator
OnCrawl Openfind
OpenLinkProfiler OrangeBot
OutfoxBot OrangeSpider
OutclicksBot
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer PageScorer
PageAnalyzer
Pandalytics
Papa Foto Papa Foto
Pi-Monster
Picscout Picscout
Picsearch
Piepmatz
Pimonster Pimonster
ProPowerBot Probethenet
Psbot Psbot
Pump Pu_iN
Quick-Crawler PyCurl
RankActive RSSingBot
RankingBot RankFlex
Rankivabot RankurBot
Re-re
Reaper
RebelMouse
Recorder Recorder
RepoMonkey RedesScrapy
Ripper
RocketCrawler RocketCrawler
SEOkicks Rogerbot
SEOkicks-Robot SBIder
ScanAlert ScanAlert
Scanbot
ScoutJet ScoutJet
Screaming Screaming
ScreenerBot
ScrepyBot ScrepyBot
Searchestate Searchestate
Shodan SearchmetricsBot
SiteLockSpider Semrush
SiteSucker Siphon
Sitebeam SiteCheckerBotCrawler
Sitevigil Sitevigil
SlySearch Snoopy
Snapbot
Sociscraper Sociscraper
Spammen Sosospider
SpankBot Sottopop
SpaceBison
Spanner Spanner
Sqworm Spinn3r
Sucker Stripper
SuperBot Sucuri
SuperHTTP
Surfbot Surfbot
Suzuran Suzuran
Szukacz
T8Abot T8Abot
TeleportPro
Telesphorep Telesphorep
TheNomad TheNomad
TightTwatBot
Tracemyfile
Trendiction
Turingos Turingos
Turnitin
TurnitinBot TurnitinBot
Twice URLy Warning
URLy.Warning
UnisterBot UnisterBot
Upflow V-BOT
VB Project VCI
Vacuum
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie Virusdie
VoidEYE VoidEYE
WASALive-Bot WASALive-Bot
WBSearchBot
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
Wallpapers Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE
Web Auto Web Auto
Web Enhancer Web Collage
Web Fetch Web Pix
Web Fuck WebAuto
Web Sucker
WebBandit WebBandit
WebCollage WebCollage
WebCopier WebCopier
WebEnhancer
WebFetch WebFetch
WebFuck WebFuck
WebGo IS WebImageCollector
WebLeacher WebPix
WebReaper WebReaper
WebSauger WebSucker
WebWhacker
WebZIP WebZIP
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag WebsiteExtractor
Website Quester
WebsiteQuester WebsiteQuester
Webster Webster
Who.is Bot Who.is Bot
Wonderbot Wotbox
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider
Xenu
YoudaoBot YoudaoBot
Zauba
Zermelo
Zeus
Zitebot Zitebot
ZumBot ZumBot
ZyBorg ZyBorg
archive.org_bot archive.org_bot
arquivo.pt
autoemailspider autoemailspider
backlink-check coccocbot-web
check1.exe
clark-crawler
cognitiveseo
com.plumanalytics com.plumanalytics
evc-batch crawler.feedback
demandbase-bot
facebookscraper facebookscraper
gopher
instabid instabid
ips-agent isitwp.com
magpie-crawler lwp-request
lwp-trivial
muhstik-scan muhstik-scan
oBot netEstate NE Crawler
page scorer ripz
pcBrowser
probe-image-size
s1z.ru
scalaj-http
seoscanners seoscanners
sexsearcher seostar
sitechecker.pro siteripz
sogouspider
sp_auditbot
sysscan sysscan
voyagerx.com trendiction.de
webpros.com webprosbot
zauba.io xpymep1.exe
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
40cg.com atelielembrancaqueencanta.com.br
courtshipgift.com country-chic.ru
dodlive.mil downloadkakaotalk.com
gopixdatabase.com earl-brown.info
hermesbirkinhandbagoutlets.com hidemyass.com
iwantmyfreecash.com kamalsinha.com
jwcialislrt.com mouselink.co
seminarygeorgia59.ga razvratnoe.org
veerotech.com supermesta.ru
ww2awards.info xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai

948
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
01apple.com 0c47f8422d3f.com
2020iscoming.info 1001desert.com
20pascals.nl 1008.su
404.mn 1xbet4you.com
4kpics.rocks 21h2o.com
4ur.click 4senses.co
5u.com 7fon.ru
72-news.com abusora.com
7a2rnuey1tw9ar.ru acgs.tk
7ooo.ru actionnooz.com
aa625d84f1587749c1ab011d6f269f7d64.com adimmix.com
aa81bf391151884adfa3dd677e41f94be1.com adloads.net
absolute-s.ru adrenalin-stk.ru
abwa.tk ads-cool.pro
add-add.men adserver-e7.com
adhome.biz adultmeetonline.info
admeasures.com advancedmassagebysara.com
admitad.com air-edem.ru
adtech.us akvamaster.dp.ua
aimayangzhi.com alessandraleone.com
alarmobninsk.ru alphaforum.ru
algerianembassy.co.in amazon-adsystem.com
all4invest.ru android-systems.ru
allknow.info apiitsthirteende-a.akamaihd.net
andpolice.com apiportalmorecom-a.akamaihd.net
android-style.com apispringsmartne-a.akamaihd.net
animal-drawings.com appleid-verification.com
anjalika.co.in arendovalka.xyz
anonymousfox.co artel116.ru
anticrawler.org asmxsatadriverin.aircus.com
apisecretsaucebi-a.akamaihd.net atofilms.com
apple.com-webbrowsing-security.science autotrends.today
appsaurus.com avtocredit-legko.ru
aramaicmedia.org azlex.uz
asrv-a.akamaihd.net babs.com.ua
asseenontvonline.ru babyfactory.fr
backlinkwatch.com backlink4u.net
badavit.com.ua
bagsonsale.online bagsonsale.online
baladur.ru
balitouroffice.com balitouroffice.com
balook.com bangla-chat-uk-paralud.ga
banan.tv bankmib.ru
bang-hotties.com bbtec.net
belsetirehafi.tk bestcalovebracelet.cn
bidbuy.co.kr bestcurs.org
big-cash.net bestdraws.com
bird1.ru besthoro.ru
bjetjt.com bestmouthwash.club
bestofferswalkmydogouteveryday.gq
bestssaker.com
beyan.host.sk
big-boards.info
bioinnovate.co
biznesluxe.ru
bjgugu.net.ua bjgugu.net.ua
blagovest-med.ru blackcurranthumidifiers.site
blog.axant.it blogporn.in
bloggen.be bodybuilding-shop.biz
bluesman.nu bombla.org
bmw-ark.ru bongacams.com
bonux.nextview.ru brainboosting.club
bonvillan.com brandedleadgeneration.com
bookmark4you.com.biz
brainsandeggs.blogspot.com
brimstonehillfortress.org
brokergid.ru brokergid.ru
bugshoot.cn bronzeaid-a.akamaihd.net
business-made-fun.com buckspar.gq
buyclomidonlaine.com budpost.com.ua
bvlgariwallet.xyz buehne-fuer-menschenrechte.de
callawaygolfoutlet.online bumskontakte.org
camdenmemorials.com buntube.net
camdolls.com buy-cheap-online.info
canoncdriverq3.pen.io buybest1.biz
canrioloadm.gq buyscabiescream.com
canrole.com canacopegdl.com
carfax.com.ua candytiner.com
carmuffler.net capsderfudd.tk
carrauterie.be cartierlovestore.xyz
cartierring.xyz ceotrk.com
casablancamanor.co.za cfjump.com
cashonads.com chastnoeporno.com
cayado.snn.gr
celebsopics.com
cenokos.ru
cercacamion.it
cfcl.co.uk
cheap-trusted-backlinks.com cheap-trusted-backlinks.com
cheapbarbour.online cheapestjerseys-wholesale.com
cheapmarmot.online
cheapmoncler.win cheapmoncler.win
chimiver.info checkm8.com
chiptuninger.com chocolateslim-en-espana.com
chocolateslim-u-srbiji.com chocolateslim-en-france.com
cialischsrx.com chuckguilford.com
cleanallvirus.com cialis-samples.com
clevernt.com cityadspix.com
clickhype.com cl.s7.exct.net
clipartnew.com cleansearch.net
clksite.com clippingphotoindia.com
cmd.kz club-samodelkin.ru
cognitiveseo.com coderstate.com
compliance-barak.top colehaanoutlet.store
consorzioilmosaico.org comeondog.info
cookie-law-enforcement-aa.xyz compiko.info
cookie-law-enforcement-ee.xyz compliance-andrew.top
copro.pw compliance-donald.xyz
copyrightclaims.org compliance-viktor.top
corta.co constantaservice.net
counter.bloke.com coslab.club
cranly.net covadhosting.biz
creatives.adbetclickin.pink cracksplanet.com
crest3d.ru creditmoney.com.ua
cyse.tk customsua.com.ua
dalavia.ru cutalltheshit.com
damedingel.ya.ru daretodonate.co
day-news.info darodar.com
ddlmega.net ddpills.com
dekoration.us dealighted.com
demian.kz destinationrealestate.com
detoxmed24.ru dipstar.org
dev78.dioniqlabb.se directrev.com
dgfitness.co dispo.de
diamond-necklace.info doeco.ru
dodlive.mil dogoimage.com
domain-tracker.com dogsrun.net
domini.cat dollartree.info
dominterior.org domainroam.win
drpornogratisx.xxx doreenblog.online
dverimegapolis.ru dosugrostov.site
dwomlink.info dprtb.com
e90post.com drev.biz
ec-file.info drivotracker.com
educontest.net drugs-no-rx.info
edudocs.net drupa.com
efnor-ac.com dvr.biz.ua
ege-russian.ru e-avon.ru
el-nation.com e-poker-2005.com
electronicadirect.com eachdayisagift.review
elektrischezigarette1.blog.pl editors.choice6912650.hulfingtonpost.com
elektrischezigaretten2.beeplog.com ednorxmed.com
encodable.com elektronischezigarettekaufen1.myblog.de
enews.tech elektrozigarette2.wordpressy.pl
enthuse.computernetworksonline.com emergencyneeds.org
envaseslotusama.com enge-fotzen.info
etm-consult.de enginebay.ru
evening-dating-club.info enternet.ee
extremepornos.net escortplius.com
ezigarettenshop2.postbit.com estibot.com
facecup.top ez8motelseaworldsandiego.com
family1st.ca f1nder.org
fapgon.com familienzahnaerzte.com
fast-torrent.ru
felizporno.com
femdom.twiclub.in
feorina.ru feorina.ru
fetroshok.ru ff30236ddef1465f88547e760973d70a.com
fighrofacciufreesig.ga fickluder69.com
financeloan.us fidelityfunding.com
flooringinstallation-edmonton.com filesclub.net
floridamobilebillboards.com fillmewithhappiness.com
filmania-x.ru
flagcounter.me
focalink.com focalink.com
forex.osobye.ru foto-telok.net
forum.doctissimo.fr frcls.fr
franklinfire.co free-gluten.ru
fuck-paid-share-buttons.xyz free-video-tool.com
fun2cell.net fsalas.com
futbolkisales.ru funponsel.com
galeria-zdjec.com gallery.rennlist.com
gambarkataku.co gameplexcity.com
gamebackyard.com getprismatic.com
ge0ip.com golden-praga.ru
getlaid-xxxhookupdirect.com goodly.pro
getpopunder.com goodwriterssales.com
gheus.altervista.org graphid.com
girlgamerdaily.com
girlspicsa.com
glavprofit.ru
glcomputers.ru
glogow.pl
go2album.com
goldadpremium.com
goldandcard.ru
good-mummy.ru
goodnightjournal.com
gosmeb.ru
grand-chlen.ru
gratuitbaise.com
greatdealshop.com greatdealshop.com
gsmlab.pl greatfind-a.akamaihd.net
gta-club.ru greenzaim.ru
happy.new.yeartwit.com guildebzh.info
harmonyglen.us hackers-crackers.tk
hauleddes.com hamacapty.com
hausfrauensex18.com hard-porn.mobi
hdimagegallery.net hdfreeporno.net
hem.passagen.se hdpixion.com
hermesbirkinhandbagoutlets.com hdseriale.pl
hesteel.pl hdwallpapers-free.com
himazin.info hermesreplica.pw
hitcpm.com heroz.fr
hosting-tracker.com homelygarden.com
hostingclub.lk hoporno.com
hotloans.ru horoshieokna.com
hotshoppymac.com hostsshop.ru
https-legalrc.biz hotelcrocenzi.sm
hut1.ru hotsocialz.com
hyipmanager.in howlongdoestizanidinestayinyoursystem.blogspot.com
id-forex.com howopen.ru
iinstalll-fii1leis.jus0wil.pp.ua humanelydrew.com
iklysha.ml husky-shop.cz
impresagaia.it iboss.com
int.search.mywebsearch.com ideibiznesa2015.ru
intelhdgraphicsgtdrive6w.metroblog.com iflycapetown.co.za
interfucks.net ilbe.club
investpamm.ru ilovevitaly.xyz
investzalog.ru imadedinner.net
irkutsk.online-podarki.com imagerydatabase.com
irunfar.com imgarit.pw
islamtoday.co.za inet-traffic.com
ispaniya-costa-blanca.ru infospot.pt
itronics.ca internet-apteka.ru
iwanttodeliver.com intervsem.ru
iweblist.info investsuccess.org
javidol.site ipornox.xxx
iqoption.pro
istizanidineanarcoticdrug.blogspot.com
ivanovo.zrus.org
jimmychoosale.online jimmychoosale.online
jjbabskoe.ru johannesburgsingles.co.za
jo24news.com journalhome.com
kakablog.net kakadu-interior.com.ua
karpun-iris.ru kandidos.com
keywordhouse.com karaganda.xkaz.org
keywordhut.com kartiniresto.com
kino-rf.ru kazan.xrus.org
kinoflux.net kerei.ru
kinostorm.net kiinomaniak.pl
kinoduh.ru
kit-opt.ru
klin3952.ru
klitimg.pw klitimg.pw
kometars.xyz konkursowo-24.pl
komputernaya-pomosh-moscow.ru konpax.com
konkursov.net kopihijau.info
kreativperlen.ch koptims.tiu.ru
kredit-pod-zalog-krasnodar.ru kotaku.com
kurwa.win kovrenok.ru
lakomka.com.ua krasnodar.ru
lanasshop.ru krasnodar.zrus.org
lapolis.it labafydjxa.su
laulini.soclog.se laptop-4-less.com
lawyersinfo.org law-check-nine.xyz
lenvred.org ldrtrack.com
light.ifmo.ru lernur.net
likesuccess.com lescinq.com
likrot.com letslowbefast.today
limads.men lida-ru.com
llastbuy.ru linkdebrideur.xyz
lmrauction.com linksharingt.com
localmatchbook.com littlesunraiser.com
lostfilm-online.ru liveinternet.ru
locimge.pw
lubetube.com lubetube.com
lyngdalhudterapi.no magicalfind-a.akamaihd.net
machicon-ueno.info mansiondelrio.co
mailemedicinals.com mantramusic.ru
manyresultshub-a.akamaihd.net marcowebonyodziezowe.pl
map028.com markjaybeefractal.com
marcogrup.com marsgatan.com
masserect.com mastershef.club
mbiologi.ru max-p.men
mearns-tractors.co.uk mebel-ekb.com
medizinreisen.de mecash.ru
meduza-consult.ru medispainstitute.com.au
mega-polis.biz.ua medtherapy.ru
member-quiz.com meget.co.za
mesbuta.info menetie.ru
metalonly.info mhtr.be
metascephe.com michellblog.online
mini-modus.ru mirmedinfo.ru
mlf.hordo.win moivestiy.biz
mmoguider.ru monetizationking.net
mnogabukaff.net money-every-day.com
moneyteam24.com moneyviking-a.akamaihd.net
monsterdivx.com morefastermac.trade
msk.afora.ru mowser.com
musflashtv.com mp3downloadhq.com
myliveblog.ru my-floor.in.ua
myprintscreen.com myblogregistercm.tk
navalwiki.info mycaf.it
netfacet.net mylida.org
newhairstylesformen2014.com mymobilemoneypages.com
news-readers.ru myseoconsultant.com
natali-forex.com
nevansk.ru
nfljerseys.online
nfljerseysforsalewholesaler.com
nhl17coins.exblog.jp
nightvision746.weebly.com nightvision746.weebly.com
nlfjjunb5.ru nisuturnetdgu.tk
nordvpn.com nomuos.it
noumeda.com nosecret.com.ua
novatech.vn nottyu.xyz
ntic.fr nuup.info
offer.camp
onblastblog.online
onescreen.cc
oneshotdate.com oneshotdate.com
onstrapon.purplesphere.in onlainbesplatno.ru
online-sbank.ru
online-x.ru
onlinemeetingnow.com
onlythegames.com
onthemarch.co onthemarch.co
openfrost.net ooo-olni.ru
openmediasoft.com oops-cinema.ru
optikremont.ru osoznanie-narkotikam.net
optitrade24.com ostrovtaxi.ru
orgasmatrix.com ourtherapy.ru
orgasmus-virtual.com ownshop.win
orion-v.com palvira.com.ua
palocco.it pawli.eu
pay2me.pl
pcimforum.com pcimforum.com
pdamods.ru petsblogroll.com
peekyou.com photo.houseofgaga.ru
perm-profnastil.ru photokitchendesign.com
picsforkeywordsuggestion.com pic-re.blogspot.com
pinup-casino1.ru piccdata.com
pinwallpaper.top pills24h.com
pirateday.ru pinkduck.ga
pixell.club piratecams.com
pizda.lol platinumdeals.gr
platezhka.net pokemon-go-play.online
playboyfiles.xblog.in pokemongooo.ml
playtap.us polytopesexempt.com
pogruztehnik.ru porndairy.in
poiskzakona.ru pornmania.pl
popugaychiki.com pornogig.com
pornohub.me pornotubexxx.name
pornolook.net pourvous.info
pornophoto.xyz preconnubial.usuby.site
postclass.com pricheskaonline.ru
presleycollectibles.com printie.com
printingpeach.com prize44.com
privacylocationforloc.com prizesbook.online
progress-upakovka.ru prod2016.com
promover.org proekt-mos.ru
prostitutki-belgoroda.org prohoster.info
proxyelite.biz prokotov.com
pskcijdc.bloger.index.hr proprostatit.com
pulse33.ru prostitutki-tyumeni.org
purchasepillsnorx.com pshare.biz
purplesphere.in psoriasis-file.trade
ragecash.com pts163.ru
ranksignals.com q-moto.ru
rapidokbrain.com qor360.com
ratemodels.net queerspace.com
razyboard.com rada.ru
realtytimes.com ranking2017.ga
rebelmouse.com rentehno.ru
recipedays.ru ric.info
redbottomheels.xyz rightenergysolutions.com.au
redhotfreebies.co.uk robertefuller.com
rednise.com robotixix.com
registratciya-v-moskve.ru rossmark.ru
registrationdomainsite.com
reklama1.ru
rent2spb.ru
restorator-msk.ru
reward-survey.net
rixpix.ru
royalvegascasino.com royalvegascasino.com
rus-pornuha.com roznica.com.ua
russia-tao.ru ruclicks.com
s-forum.biz running-line.ru
s8-nowy-wygraj.comli.com rutor.vip
sammyweaver.com ryetaw.com
samoiedo.it s1z.ru
sbricur.com sakhboard.ru
sbt-aqua.ru saletool.ru
scanner-julia.top salutmontreal.com
scanner-walter.top scanner-mark.top
scanspyware.net scanner-willy.top
scottbywater.com searchencrypt.com
sdi-pme.com searchimage.co
search.1and1.com secretscook.ru
searchengineranker.email segol.tv
see-your-website-here.com seo-prof1.xyz
seozoom.it seo-smm.kz
seo-tools-optimizing.com
seorankinglinks.com
service.adtech.fr
servicecenter.co.ua
servingnotice.com
sex-sex-sex5.com sex-sex-sex5.com
sexkrasivo.net sexiporno.net
sexs-foto.com sexyali.com
sexsaoy.com shareyards.com
sexsearch.com sherlock.se
sextracker.be shopcheermakeup.info
sexyebonyteen.com shoppingmiracles.co.uk
sexytrend.ru shtora66.ru
sfd-chess.ru siha.de
shama-rc.net simpoed.ufop.br
similardeals.net sladkoevideo.com
simple-share-buttons.com slavkokacunko.de
slomm.ru small-games.biz
soaksoak.ru
soc-proof.su
social-s-hhh.xyz
social-vestnik.ru social-vestnik.ru
socialbookmarksubmission.org socialbuttons.xyz
sodexo.com softonicads.com
soft-program.com solinf.co
solitaire-game.ru songplanet.ru
spasswelt.xyz souvenir.cc
special-porn.com space4update.pw
speedup-my.site spinnerco.ca
spywarenuker.com spyfu.com
stanthonyscatholicchurch.org starwars.wikia.com
start.myplaycity.com statistici.ro
sticken.co steamoff.net
streha-metalko.si stickers-market.ru
strongholdsb.ru stpicks.com
stuff-about-money.com streamin.to
sukirgenk.dvrlists.com strfls.com
support.nopeas.sk studentguide.ru
suralink.com sudexpert66.ru
swinginwithme.ru suggest-keywords.com
tastyfoodideas.com susanholtphotography.com
tattoo33.ru swiped.su
tattooreligion.ru syndicate.fun
tanieaukcje.com.pl
tdsing.ru tdsing.ru
tecspb.ru terrafootwear.us
testingads.pro thebestweightlosspills.ovh
the-torrent-tracker.blogspot.com thebluenoodle.com
thebitcoincode.com thedownloadfreeonlinegames.blogspot.com
theendivechronicles.com thefds.net
themeforest.net thefotosgratis.eu
thepokertimer.com thegameriders.com
thesmartsearch.net thegolfclub.info
timecrimea.ru titan-ads.life
tizanidineingredients.blogspot.com titangel-vietnam.com
tizanidine4mguses.blogspot.com
tizanidine6mg.blogspot.com
tizanidineandgabapentin.blogspot.com
tizanidinehcl2mg.blogspot.com
tizanidinehcl2mgsideeffects.blogspot.com
tizanidinehcl4mgtab.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com tizanidinesideeffectsweightloss.blogspot.com
tizanidinezanaflex.blogspot.com tizanidinezanaflex.blogspot.com
tootoo.to tjkckpytpnje.com
top-l2.com tksn.ru
top10registrycleaners.com tmtrck.com
topappspro.com tnctrx.com
topmira.com tourismvictoria.com
toyota.7zap.com toursmaps.com
track112.site trafficgenius.xyz
traderzplanet.in truemfilelj.gq
traffic100.com tsan.net
traffixer.com tuberkulezanet.ru
traffpartners.com turbodsp.com
tramadolandtizanidine.blogspot.com twiclub.in
triumf-realty.ru typer.one
trucri.me tzritel.tk
ttsq.fr ua-company.ru
tula.howotorg.ru uchebavchehii.ru
turbabitload.weebly.com ukr-lit.com
twincitiescarservice.com umityangin.net
twittrading.com usacasino.com
u17795.netangels.ru utiblog.fr
uk-zheu20.ru vandrie-ict.nl
ukkala.xyz vapomnoncri.tk
ukrtextbook.com vegan-foods.us
umekana.ru vellings.info
unmaroll.ya.ru viktoria-center.ru
unrealcommander.com vilingstore.net
upproar.com vipsiterip.org
upstore.me visitcambridge.org
uptime-gamma.net
urdoot.win
uroffer.link
valmetrundan.se
valsalud.com
velpanex.ru
vepad.com
viandpet.com
videokrik.net
vids18.site
vip.51.la
viveresaniesnelli.it
volgograd.xrus.org volgograd.xrus.org
vsexkontakte.net voprosotvet24.ru
vtc.pw walkme.com
vvpg.ru wareseeker.com
vysigy.su warezaccess.com
waterefficiency.co webenlace.com.ar
we-are-gamers.com
web-betting.ru
webads.co.nz
weblibrary.win
webnode.me
webradiology.ru webradiology.ru
website-analytics.online website-stealer.nufaq.com
weclipart.com weburok.com
wfb.hatedriveapart.com wesharepics.com
whengirlsgowild.com wesharepics.info
withstandingheartwarming.com westen-v.life
wma-x.com whatistizanidine2mg.blogspot.com
woman-tampon.ru whereverdesperate.gq
wonderfulflowers.biz whosonmyserver.com
wwwadultcheck.com wieseversa.no
x-porno.video wladimirpayen.com
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai wnhjavlhezp.gq
xn--80aafb2a.xn--p1ai woodyguthrie.se
xn--80ahdheogk5l.xn--p1ai workle.website
xn--80aikhbrhr.net wpthemedetector.co.uk
www.jbetting.com
x-diesel.biz
xaijo.com
xmlinde.com
xmronta.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
xn----ctbigni3aj4h.xn--p1ai
xn--80aanaardaperhcem4a6i.com
xn--80aeahghtf8ac5i.xn--p1ai
xn--b1agm2d.net xn--b1agm2d.net
xpresscare.ru xn--d1abj0abs9d.in.ua
xrus.org xnxx-n.com
xwatt.ru xtraffic.plus
xxxhdvideo.site xvideospornoru.com
youradexchange.com xxxasianporn.net
yourserverisdown.com xxxtubesafari.com
youtubologia.it youradulthosting.com
za-music.mymobiplanet.com zaim-pod-zalog-krasnodar.ru
zahvat.ru zaimhelp.ru
zaloadi.ru zakonobosago.ru
zarenica.net zeroredirect6.com
zhacker.net zeroredirect9.com
zhongwenlink.com zhongwenlink.com
zocaparj.kz znakomstva-moskva77.ru
zolotoy-lis.ru zoogdinsney.com
zryydi.com zrizvtrnpale.tk
zvuker.net zygophyceous.womanstars.site
zzlgxh.com

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
13tabs.com 007angels.com
202ch.com 0912701309f8ce.com
24videos.tv 1adult.com
4ureyesonly.com
5-steps-to-start-business.com
5ws.dating-app.ru
7fon.ru 7fon.ru
a.pr-cy.ru 7wind.ru
above.com 8xv8.com
adimmix.com advokateg.xyz
advancedsoftwaresupport.com agapovdg.ru
air-edem.ru alekseevec.ru
all4wap.ru algerianembassy.co.in
aloofly.com allblogroll.com
ameblo.jp alldezservice.kz
amung.us allpdfmags.net
anal-acrobats.hol.es anjalika.co.in
app5.letmacworkfaster.world apessay.com
apper.de apper.de
asociatia-tipografilor-transilvania.ro arate.ru
atdedinotuho.tk arcadepages.com
auto.rusvile.lt arenanews.com.ua
avtointeres.ru arendatora.ru
avtovolop.ru arius.tech
bag77.ru autobudpostach.club
belinka.com.ua autorn.ru
best-businessman.ru awency.com
best-seo-software.xyz azartniy-bonus.com
bestdraws.com bankmib.ru
bestprofits-there.com bestpornuha.com
bizlist.com.de bluejays-jerseys.us
blobar.org bobinoz.com
bookmark4you.com bocoarchives.org
bosman.pluto.ro boltalko.xyz
buckspar.gq brandedleadgeneration.com
bugshoot.cn bristolhotel.com.ua
bus-offer.com buynorxx.com
businescoop.men cactussoft.biz
buyessaynow.biz calc-for-credit.ru
buypillsorderonline.com cartierwatch.xyz
cardul.ru chocolateslim-original.com
carsplate.com christianlouboutinoutlet.win
cartierloveringreplica.xyz cienum.fr
cartierreplica.top co.lumb.co
caveavins.fr
chatroulette.world
cleanallspyware.com
clips.ua.ac.be
clothesforcash.com
coinsspb.com coinsspb.com
compliance-fred.xyz confib.ifmo.ru
compliance-jack.top counterbot.com
computernetworksonline.com crirussian.ru
construmac.com.mx customergrowthsystems.com
cordstrap.cc daneshetabiat.com
craftinsta.ru detalizaciya-tut.biz
crystalslot.com detoxmed24.ru
cutt.us digilander.libero.it
damedingel.ya.ru dildofotzen.net
db.speedup.gdn dinosaurus.site
derevesendeco.com directrev.com
devochki.top domainsatcost.com
divisioncore.com domination.ml
divku.ru dostavimvdom.ru
droidlook.net easytuningshop.ru
drugs-no-rx.info ebonyporn.site
drugstoreforyou.com ege-essay.ru
drunkenstepfather.com everflora.ru
eets.net expertnaya-ocenka.ru
election.interferencer.ru faithe.top
eropornosex.ru fickkontakte.org
escortslet.net film-one.ru
essayassist.com filmi-v.online
essaypro.com
european-torches.ru
expdom.com
exportshop.us
ez8motelseaworldsandiego.com
ezigarettenkaufen2.dreamwidth.org
ezigarettenshop2.postbit.com
fast-wordpress-start.com
fetlifeblog.com
filesvine.com
fingerprintjs.com fingerprintjs.com
flowersbazar.com flipper.top
fok.nl floridamobilebillboards.com
foxjuegos.com flprog.com
fr-bearings.ru free-share-buttons.blogspot.com
friendflnder.com free-video-tool.com
fuel-gas.com frighteningremain.cf
gateway.zscalertwo.net froggytube.com
gentamicineyedrops.blogspot.com fusoradio.info
gktt.ru fx-brokers-review.com
glopages.ru fz139.ttk.ru
goldandcard.ru gazoblok.net.ua
goodwriterssales.com ghostvisitor.com
gotwebsite1.com gilsonchiro.xyz
gpirate.com global-ics.co.za
gratuitbaise.com globatur.ru
gungamesz.com gorabagrata.ru
gymi.name greenzaim.ru
hannasolution.ru grtyi.com
hauleddes.com gunsvicceadadebt.tk
hccoder.info highland-homes.com
healbio.ru iideaidekonkatu.info
healing-dysplasia.ru ilikevitaly.com
heartofbeijing.blogspot.com incest-ru.com
hermesbracelets.xyz invest-pamm.ru
hoholikik.club izoll.ru
holidaypics.org janavibekken.no
hotblog.top jasonpartington.com
hotdl.in jus0wil.pp.ua
housemilan.ru justprofit.xyz
hptwaakw.blog.fc2.com kaac.ru
hubbble.com kakablog.net
hunthillfarmtrust.org kanimage.com
igtools.club keywordspay.com
itag.pw kino-fun.ru
ivearchenceinflu.cf kinobest.pl
iwantmyfreecash.com klosetkitten.com
izhevsk.xrus.org
karpun-iris.ru
keywordhut.com
kichenaid.ru
kidskunst.info
kino-key.info
kino-rf.ru
kinofak.net
klumba55.ru klumba55.ru
ks1234.com konica.kz
kunstaktien.info landoftracking.com
landmania.ru lecbter-relationships.ga
law-check-nine.xyz libertybilisim.com
lego4x4.ru likrot.com
lflash.ru limads.men
lifeinsurancekb.com liveu.infoteka.hu
link.ac ljusihus.se
live-sexcam.tk lmrauction.com
llastbuy.ru lostaruban.com
lyrics.home-task.com lovelycraftyhome.com
mahnwachen-helfen.info maladot.com
managerpak204.weebly.com mantramusic.ru
maslenka.kz martlinker.com
masturbate.co.uk
mcar.in.ua mcar.in.ua
meduza-consult.ru medanestesia.ru
mega-bony2017.pl medicovi.com
microsearch.ru meetlocalchicks.com
mjchamonix.org meltwater.com
mobile.ok.ru menetie.ru
mojaocena.com merryhouse.co.uk
monetizer.com-01.site metarip.ru
mp3ringtone.info micasainvest.com
mvpicton.co.uk minet.club
netanalytics.xyz mirobuvi.com.ua
mnogabukaff.net
monclercheap.xyz
moncleroutletonline.xyz
monthlywinners.com
mowser.com
mriyadh.com
mustat.com
mypornfree.ru
myxdate.info
newyorkhotelsmotels.info
next-dentists.tk next-dentists.tk
nextbackgroundcheck.gq nextconseil.com
nonameread45.live nextlnk12.com
nuit-artisanale.com nhl09.ru
ochistka-stokov.ru niroo.info
ohmyrings.com njkmznnb.ru
onescreen.cc nordstar.pro
online-podarki.com obiavo.by
ownshop.cf obiavo.kz
omoikiri-japan.ru
online247.ml
onlythegames.com
open-odyssey.org
openfrost.com
ororodnik.goodbb.ru
osnova3.ru
p-business.ru p-business.ru
paparazzistudios.com.au palvira.com.ua
partner-high.men parajumpersoutlet.online
pk-services.ru partner-cdn.men
pl-vouchers.com pb-dv.ru
plaff-go.ru pc4download.co
podshipniki-ntn.ru pelfind.me
picsearch.com
pixshark.com
pizdeishn.com
pliks.pl
polska-poezja.com polska-poezja.com
pomoc-drogowa.cba.pl pornophoto.xyz
popup-hgd.xyz pornoreino.com
porno.simple-image.com.ua portadd.men
pornotubexxx.name powitania.pl
prlog.ru privhosting.com
purplestats.com promalp-universal.ru
py100.ru purplesphere.in
quit-smoking.ga puteshestvennik.com
rankexperience.com r-e-f-e-r-e-r.com
realizmobi.com rcpmda.ikan1080.xyz
regionshop.biz
rentaremotecomputer.com rentaremotecomputer.com
replica-watch.ru riders.ro
resultshub-a.akamaihd.net ringporno.com
revistaindustria.com rus-teh.narod.ru
rome2rio.com saitevpatorii.com
rozalli.com saltspray.ru
ru-dety.ru salut-camp.ru
saecsa.co sanidumps.com
salutmontreal.com semrush.com
savingsslider-a.akamaihd.net sexstream.pl
schalke04fc.info
sdrescher.net
sethrollins.net
sexvideo-sex.com
sexyebonyteen.com sexyebonyteen.com
shikiso.info shoesonlinebuy.xyz
shoppytoolmac.com si-unique.com
silvermature.net simply.net
siteripz.net softonicads.com
slomm.ru softwaretrend.net
social-buttons.com sonnikforme.ru
soviet-portal.do.am space4updating.win
standardchartered-forex.com spaceshipad.com
starwars.wikia.com sps-shop.com
students-cheapskate.ml spy-sts.com
sunflowerdrawingpaintings.blogspot.com stauga.altervista.org
superstarfloraluk.com stockquotes.wooeb.com
taqplayer.info streha-metalko.si
tedy.su stroicol.net
teenbbw.yopoint.in sundrugstore.com
thebitcoincode.com superiends.org
thebluenoodle.com superlist.biz
thruport.com supermodni.com.ua
tobeyouday.win swimpool.ca
toondinsey.com t-bygg.com
topcasinoratings.ru takethatad.com
topshef.ru tedxrj.com
texbaza.by
thedownloadfromwarez.blogspot.com
thefotosgratis.eu
tizanidinehydrochloride2mg.blogspot.com
tizanidinemusclerelaxant.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com
torrents.life torrents.life
tovaroboom.vast.ru tovaroboom.vast.ru
traffic2cash.org tozup.com
tsan.net trackmedia101.com
tsstcorpcddvdwshbbdriverfb.aircus.com trafficgenius.xyz
u555u.info turizm.bz
udayavani.com u-cheats.ru
unacittaconte.org unlimitdocs.net
valueclick.com unpredictable.ga
vashsvet.com v24s.net
veles.shop vezuviy.su
vertaform.com videospornogratisx.net
viagroid.ru vip-parfumeria.ru
villacoloniale.com voloomoney.com
voluumtracker1.com vostoktrade.info
vsesubwaysurfers.com wcb.su
wait3sec.org wdfdocando.com
walkme.com webcamtalk.net
warezkeeper.com www.timer4web.com
waycash.net x-lime.com
webix.me xep.info
webstatsdomain.org xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
www.pinnacle-bets.com xn--80atua3d.xn--p1ai
xsfetish.org xn--h1ahbi.com.ua
zaimhelp.ru xn--oogle-wmc.com
zakazfutbolki.com xxxdatinglocal.us
zapiszto.pl yougotanewdomain.com
zone-kev717.info yourads.website
zoomovies.org yoursearch.me
zreloeporno.tv youtuhe.com
zaapplesales.blogspot.com
zaidia.xhost.ro
zatjmuzu.info
zvooq.eu
zx6.ru

12
dev-tools/test_units/ratelimittest-master.txt
View file

@ -1,21 +1,11 @@
<!DOCTYPE html>
<html> <html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html> ml><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
<center><h1>503 Service Temporarily Unavailable</h1></center> <center><h1>503 Service Temporarily Unavailable</h1></center>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.220 0; 10.1.0.84 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2627 ### Version: V4.2021.10.2628
### Updated: Sat Oct 2 22:01:31 UTC 2021 ### Updated: Sun Oct 3 10:05:36 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2627 ### Version: V4.2021.10.2628
### Updated: Sat Oct 2 22:01:31 UTC 2021 ### Updated: Sun Oct 3 10:05:36 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##