vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.09.3344 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-09-11 22:07:04 +00:00
parent fc7f06dec8
commit fac964caa4
47 changed files with 95517 additions and 95527 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.09.3343 #### Version: V4.2022.09.3344
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.09.3343 #### Version: V4.2022.09.3344
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.09.3343 #### Version: V4.2022.09.3344
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11804
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.09.3343 #### Version: V4.2022.09.3344
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11808
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11808
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11808
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11808
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 1; 10.1.0.56 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 0; 10.1.0.56 0;
127.0.0.1 0; 127.0.0.1 0;

11808
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 1; 10.1.0.56 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 0; 10.1.0.56 0;
127.0.0.1 0; 127.0.0.1 0;

11808
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 1; 10.1.0.56 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 0; 10.1.0.56 0;
127.0.0.1 0; 127.0.0.1 0;

11808
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3343 3344

11808
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11808
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.97.112.143 0; 23.102.125.174 0;

11808
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 1; 10.1.0.56 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 0; 10.1.0.56 0;
127.0.0.1 0; 127.0.0.1 0;

11808
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 1; 10.1.0.56 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 0; 10.1.0.56 0;
127.0.0.1 0; 127.0.0.1 0;

11808
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 1; 10.1.0.56 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 0; 10.1.0.56 0;
127.0.0.1 0; 127.0.0.1 0;

11808
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.97.112.143 0; 23.102.125.174 0;

11808
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.97.112.143 0; 23.102.125.174 0;

11808
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 1; 10.1.0.56 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anarchy99 GermCrawler
Brandprotect IDBTE4M
FlashGet Net Vampire
HonoluluBot Nutch
JetCar QueryN Metasearch
SalesIntelligent SocialRankIOBot
Web Sucker WebAuto
WebSucker Webalta
heritrix Xenu
page scorer ips-agent

310
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 80legs
404checker Aboundex
ADmantX Aboundexbot
ALittle Client AiHitBot
ASPSeek Anthill
Alexibot Aspiegel
Anarchy
Anarchy99
Ankit
Atomseobot Atomseobot
BDFetch Attach
BBBike
BDCbot
BLEXBot
BackDoorBot
BackStreet BackStreet
BackWeb Backlink-Ceck
BacklinkCrawler
Badass
Barkrowler Barkrowler
BatchFTP Battleztar Bazinga
Bigfoot Bitacle
Black Hole Black Hole
BlackWidow Blackboard
Boardreader Blow
BotALot BuiltBotTough
Brandprotect
Buck
Buddy
BuiltWith BuiltWith
BuzzSumo BuzzSumo
Calculon CODE87
CheeseBot Chlooe
Citoid Claritybot
Cliqzbot Cloud mapping
Copyscape
Cosmos Cosmos
Crawling at Home Project Craftbot
CrazyWebCrawler CrunchBot
Crescent Curious
Custo Custo
DBLBot CyotekWebCopy
DIIbot DIIbot
DataCha0s DSearch
Demon Demon
Dirbuster Dirbuster
Disco Disco
Discoverybot
Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot DomainStatsBot
DomainAppender
DomainSigmaCrawler
Dotbot Dotbot
Drip Dragonfly
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL
Ecxi
EroCrawler EroCrawler
Exabot Express WebPictures
EyeNetIE Extractor
Fimap Extreme Picture Finder
FlashGet FemtosearchBot
Flunky Firefox/7.0
Foobot Foobot
Freeuploader Freeuploader
FrontPage
FyberSpider FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW GalaxyBot
GermCrawler
GetRight GetRight
GetWeb GetWeb
Getintent
GoZilla GoZilla
GrapeshotCrawler GrabNet
HEADMasterSEO Grafula
HTTP::Lite HMView
HTTrack Haansoft
Havij HaosouSpider
HonoluluBot HonoluluBot
HybridBot HybridBot
IDBTE4M InternetSeer
IDBot Iskanie
IRLbot
Iblog
Id-search
IlseBot
IndeedBot
Indy Library
InfoNaviRobot
InfoTekies
Intelliseek
Internet Ninja
IstellaBot IstellaBot
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz
JennyBot
Jorgee Jorgee
Kenjin Spider Jyxobot
Keybot Translation-Search-Machine
Keyword Density
LWP::Simple LWP::Simple
Larbin Lftp
LeechFTP
LeechGet
LexiBot
Libwhisker
Lightspeedsystems Lightspeedsystems
Likse Likse
LinkScan LinkScan
LinkWalker
Linkbot Linkbot
LinkextractorPro Lipperhey
Lipperhey Spider MIDown tool
MQQBrowser
MSFrontPage MSFrontPage
Mail.RU_Bot MTRobot
Majestic SEO Magnet
Majestic-SEO Majestic-SEO
Majestic12 MarkMonitor
Mass Downloader MarkWatch
Masscan
MauiBot
Mb2345Browser
Mediatoolkitbot
MicroMessenger MicroMessenger
Microsoft URL Control Microsoft Data Access
Moblie Safari
Mojeek Mojeek
Mojolicious Mojolicious
Msrabot Morfeus Fucking Scanner
NPbot Mozlila
Name Intelligence Mr.4x3
Navroad
Needle
Nessus Nessus
Net Vampire
NetAnts
NetLyzer
NetMechanic
Nettrack
Netvibes Netvibes
Nibbler
Niki-bot Niki-bot
Nikto
Nimbostratus Nimbostratus
Ninja Ninja
Nmap
Not Not
Nuclei Nuclei
Nutch
Octopus Octopus
Offline Navigator OnCrawl
OpenLinkProfiler
Openfind Openfind
OrangeSpider
OutclicksBot
OutfoxBot OutfoxBot
PHPCrawl
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer
Panscient Panscient
Petalbot Pavuk
Pi-Monster
Picscout Picscout
Picsearch PictureFinder
Piepmatz PleaseCrawl
Pu_iN
Pump Pump
PyCurl PyCurl
QueryN Metasearch
RSSingBot
RankActive RankActive
Rankivabot RankurBot
Re-re Re-re
RealDownload RealDownload
RebelMouse RebelMouse
SEOkicks-Robot Recorder
SEOprofiler RedesScrapy
SISTRIX RepoMonkey
SMTBot ScanAlert
Scanbot
ScoutJet
Scrapy Scrapy
Searchestate Screaming
Semrush Semrush
SemrushBot
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot SeobilityBot
SiteCheckerBotCrawler Site Sucker
SlySearch SiteExplorer
SmartDownload Snake
Snapbot
SocialRankIOBot SocialRankIOBot
Sociscraper Sogou web spider
Sottopop Sosospider
Spammen SpaceBison
SpankBot SpankBot
Spanner Spinn3r
SputnikBot
Sqlmap Sqlmap
Sqlworm Sqworm
Sucuri Steeler
SurveyBot SuperHTTP
Swiftbot Swiftbot
Teleport T8Abot
The Intraformant The Intraformant
TheNomad TheNomad
Thumbor
Titan Titan
Toweyabot Toata
Tracemyfile Trendictionbot
Trendiction Turingos
TurnitinBot
TwengaBot TwengaBot
Twice URLy Warning
URLy.Warning
Upflow Upflow
V-BOT V-BOT
VCI
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
VidibleScraper VidibleScraper
Voil Virusdie
Voltron
WASALive-Bot
WBSearchBot
WISENutbot
WPScan WPScan
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
Web Auto WallpapersHD
Web Fetch WeSEE
Web Fuck Web Enhancer
Web Pix
Web Sauger
Web Sucker
WebBandit
WebCollage
WebCopier
WebEnhancer
WebFetch WebFetch
WebGo IS WebFuck
WebImageCollector WebLeacher
WebReaper WebReaper
WebStripper WebStripper
WebSucker WebSucker
WebWhacker WebWhacker
Website Quester Webalta
Webster Webshag
WebsiteExtractor
Whack
Whacker Whacker
WinHTTrack Wotbox
WiseGuys Robot Wprecon
Wonderbot YoudaoBot
Xaldon WebSpider Zade
Xenu
Zauba Zauba
Zermelo
Zeus Zeus
ZmEu ZoominfoBot
ZoomBot ZumBot
arquivo.pt ZyBorg
adscanner
autoemailspider autoemailspider
cah.io.community
check1.exe check1.exe
clark-crawler clark-crawler
coccocbot cognitiveseo
demandbase-bot crawl.sogou.com
eCatch dataforseo.com
evc-batch gopher
heritrix instabid
lwp-trivial ips-agent
magpie-crawler isitwp.com
meanpathbot linkdexbot
lwp-request
mediawords mediawords
netEstate NE Crawler
oBot oBot
page scorer seocompany.store
pcBrowser
plumanalytics
probe-image-size
seobility
seostar seostar
serpstatbot
sexsearcher sexsearcher
sitechecker.pro sitechecker.pro
siteripz sogouspider
spyfu spyfu
trendiction.de trendiction.com
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler webmeup-crawler
webpros.com
x09Mozilla x09Mozilla
zgrab

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider
404enemy 404enemy
80legs ADmantX
ASPSeek ASPSeek
Aboundex Abonti
Aboundexbot Aboundexbot
AfD-Verbotsverfahren Acunetix
AhrefsBot
Aipbot
AllSubmitter
Alligator
AlphaBot AlphaBot
Anarchie Anarchie
Apexoo Ankit
Aspiegel Anthill
Atomseobot Atomseobot
AwarioRssBot
BBBike
BackWeb
BacklinkCrawler BacklinkCrawler
Badass
Bandit
Barkrowler Barkrowler
Battleztar Bazinga
BetaBot BetaBot
Blow Bigfoot
BlowFish Blackboard
Boardreader Boardreader
Bolt
BotALot BotALot
Bullseye BuiltBotTough
BuzzSumo
CCBot CCBot
CSHttp Calculon
Cegbfeieh Cegbfeieh
CheTeam CheTeam
CherryPicker CheeseBot
ChinaClaw ChinaClaw
Chlooe Citoid
Cliqzbot
Cogentbot
Collector
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Cosmos Craftbot
CrazyWebCrawler Crescent
CrunchBot
Custo
DBLBot DBLBot
DIIbot
DSearch
DTS Agent DTS Agent
DataCha0s
Demon Demon
Deusu
Devil
Digincore
DnyzBot DnyzBot
DomainAppender DomainStatsBot
DomainSigmaCrawler
Dotbot
Download Wonder Download Wonder
ECCP/1.0 Dragonfly
Ecxi EMail Wolf
Evil Ebingbong
Extractor ExtLinksBot
ExtractorPro Ezooms
EyeNetIE FHscan
Flunky
Foobot
FrontPage FrontPage
FyberSpider
GalaxyBot
Genieo
GetWeb GetWeb
Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
Gotit Gotit
Grafula GrabNet
Grabber
GrapeFX GrapeFX
Haansoft HMView
Harvest HTTP::Lite
Hloader HTTrack
Heritrix
HonoluluBot
IDBTE4M
IDBot IDBot
IRLbot
Id-search Id-search
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker IndeedBot
InfoNaviRobot
InfoTekies InfoTekies
Internet Ninja Internet Ninja
Iria
IstellaBot IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JetCar
Jetty Jetty
Joomla
Jorgee
JustView JustView
Kenjin Spider
Keybot Translation-Search-Machine
Keyword Density Keyword Density
Kinza Kinza
Lanshanbot Kozmosbot
LNSpiderguy
Leap Leap
LeechFTP LeechGet
LexiBot
Lftp Lftp
LibWeb LibWeb
LieBaoFast Likse
LinkScan
LinkWalker
Linkbot Linkbot
LinksManager LinkextractorPro
LinqiaMetadataDownloaderBot Lipperhey Spider
Ltx71 Litemage_walker
MIDown tool MFC_Tear_Sample
MJ12bot MIIxpc
MQQBrowser MTRobot
MSFrontPage Majestic SEO
MSIECrawler Majestic-SEO
Mail.RU_Bot Majestic12
Masscan MarkMonitor
Mata Hari Mass Downloader
Meanpathbot Mb2345Browser
MeanPath Bot
Mediatoolkitbot
MegaIndex.ru
Metauri
MicroMessenger MicroMessenger
Mister PiX Microsoft Data Access
Mojolicious Mojolicious
Morfeus Fucking Scanner MolokaiBot
Mozlila Msrabot
Mr.4x3 Name Intelligence
Musobot Nessus
NPbot Net Vampire
Nameprotect NetLyzer
NearSite
Needle
NetMechanic
NetSpider
NetZIP NetZIP
NimbleCrawler NextGenSearchBot
Nibbler
Niki-bot
Nimbostratus Nimbostratus
Ninja
Not Not
Nutch
Octopus Octopus
Offline Explorer
OnCrawl OnCrawl
OpenVAS OpenLinkProfiler
Openvas OrangeBot
OrangeSpider OrangeSpider
OutclicksBot
OutfoxBot OutfoxBot
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer
PageAnalyzer PageAnalyzer
PageScorer Panscient
PageThing.com
Papa Foto Papa Foto
Pavuk
PeoplePal PeoplePal
Petalbot
Pi-Monster Pi-Monster
Pockey PictureFinder
Pixray
PleaseCrawl
Psbot Psbot
PxBroker PxBroker
PyCurl QueryN Metasearch
Quick-Crawler
RSSingBot RSSingBot
Rankivabot RankActive
RankActiveLinkBot
Re-re Re-re
RealDownload
RebelMouse
Recorder Recorder
RedesScrapy
RepoMonkey RepoMonkey
RocketCrawler Ripper
Scrapy Rogerbot
ScreenerBot SBIder
Searchestate SEOprofiler
SemrushBot SalesIntelligent
ScanAlert
ScrepyBot
SearchmetricsBot
Semrush
SeoSiteCheckup SeoSiteCheckup
Seomoz Shodan
Siphon
Site Sucker Site Sucker
SiteExplorer SiteExplorer
Siteimprove Sitebeam
Sociscraper
Sogou web spider Sogou web spider
Sosospider SpaceBison
Spammen SpankBot
Spanner SputnikBot
Sqlmap Sqlmap
Sqworm Stripper
Steeler Sucker
SuperBot
Surfbot
SurveyBot
Suzuran
Swiftbot
Szukacz Szukacz
T0PHackTeam
T8Abot
TeleportPro
Telesoft
The Intraformant The Intraformant
TheNomad
Titan Titan
Toweyabot Toata
Tracemyfile
Trendiction Trendiction
Trendictionbot
True_Robot
Turingos Turingos
TwengaBot Turnitin
UnisterBot UnisterBot
Upflow V-BOT
VCI
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler Virusdie
VeriCiteCrawler Voltron
VidibleScraper
VoidEYE
Voil
WASALive-Bot WASALive-Bot
WEBDAV WEBDAV
WISENutbot
WPScan WPScan
WWW-Collector-E WWWOFFLE
WWW-Mechanize
Wallpapers
WallpapersHD WallpapersHD
WeSEE Web Collage
Web Auto Web Pix
Web Sucker WebBandit
WebAuto
WebCollage WebCollage
WebCopier WebFuck
WebGo IS WebImageCollector
WebReaper WebReaper
WebStripper
WebWhacker
WebZIP WebZIP
Webalta
Website Quester Website Quester
WebsiteExtractor Whacker
Whack
Who.is Bot
Widow Widow
Wonderbot Wonderbot
Woobot
Wotbox Wotbox
Xenu Xenu
YoudaoBot Zermelo
Zitebot ZmEu
ZoomBot
ZoominfoBot
ZyBorg ZyBorg
adscanner archive.org_bot
autoemailspider autoemailspider
backlink-check
check1.exe
clark-crawler
coccocbot
cognitiveseo
com.plumanalytics com.plumanalytics
crawler4j crawler.feedback
demandbase-bot
eCatch eCatch
instabid gopher
ips-agent page scorer
iubenda-radar ripz
magpie-crawler s1z.ru
netEstate NE Crawler scalaj-http
oBot
probe-image-size
satoristudio.net
scan.lol scan.lol
seocompany.store sexsearcher
seoscanners siteripz
serpstatbot
sogouspider sogouspider
sp_auditbot
spyfu spyfu
webgains-bot
webmeup-crawler webmeup-crawler
webprosbot
zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
allnews24.in apper.de
anmysite.com gaypornmovie.net
arendatora.ru humanorightswatch.org
courtshipgift.com makis.nu
femdom.twiclub.in pl-top.pl
itbc.kiev.ua privetsochi.ru
kidd.reunionwatch.com soietvousmaime.fr
securesmrt-dt.com topanasex.com
thebestweightlosspills.ovh wovis.site
whos.amung.us xn--80aaagvmjabrs1aoc9luc.xn--p1ai

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0c47f8422d3f.com 1000n1.ru
1001watch.com.ua 100dollars-seo.com
10bet.com 101flag.ru
123any.com 108shot.com
123movies.love 123kuma.com
18ps.ru 12bet.com
20pascals.nl 2daytrendingnews.com
34.gs 70casino.online
3weekdiet.com aa625d84f1587749c1ab011d6f269f7d64.com
55wmz.ru aa8b68101d388c446389283820863176e7.com
66cpwgln.space
76brighton.co.uk
98oi.ru
9icmzvn6.website
a14download.com
aa9bd78f328a6a41279d0fad0a88df1901.com
aa9d046aab36af4ff182f097f840430d51.com
aaa38852e886ac4af1a3cff9b47cab6272.com aaa38852e886ac4af1a3cff9b47cab6272.com
abovetherivernc.com aac500b7a15b2646968f6bd8c6305869d7.com
addray.pro aaronabel.com
adsland.men acmebtn.ml
adtech.fr adhome.biz
advancedmassagebysara.com adloads.com
agroeconom.kz adsloads.com
alevinefesleri.com agardomains.com
allbizne.co.ua akita.kz
akvamaster.dp.ua
all4invest.info
alldubai.biz
allknow.info allknow.info
allsilver925.co.il amateurgalls.com
anmysite.com amateurmatch.com
ap.senai.br ample-awards-today.us
apiitsthirteende-a.akamaihd.net amung.us
apispringsmartne-a.akamaihd.net angkortours.vn
arcadeplayhouse.com animalrank.com
arkartex.ru anlimebel.kiev.ua
asiengirls.net anti-virus-removal.info
asrv-a.akamoihd.net antons-transporte.de
apiallgeniusinfo-a.akamaihd.net
apisecretsaucebi-a.akamaihd.net
apiwebwebgetcom-a.akamaihd.net
applicationg29.com
apps-analytics.net
arkkivoltti.net
ascat.porn
asseenontvonline.ru asseenontvonline.ru
aussie-prizes.com australia-opening-times.com
autonew.biz autochoixspinelli.com
aversis.net autodo.info
aviav.co autoua.top
avtovolop.ru awency.com
azbukafree.com azartniy-bonus.com
balois.worldbreak.com backgroundpictures.net
bang-hotties.com backlink4u.net
bankcrediti.ru bannerbank.ru
beauty-things.com bard-real.com.ua
beachdriveblog.com
becuo.com becuo.com
belinka.com.ua bestadbid.com
best-games.today bestcalovebracelet.cn
bestbuy.ca bestsexyblog.com
bestimagecoollibrary.com betonka.pro
bestplacetobuyeriacta.jw.lt bget.ru
beyan.host.sk bidbuy.co.kr
bimatoprost-careprost.com.ua bio.trade-jp.net
blockety.co bitcoin-ua.top
blogqpot.com biznesrealnost.ru
bmwhighperformers.com bizru.info
boeuklad.com boltalko.xyz
bolegapakistan.com bonux.nextview.ru
boom-celebs.com bonvillan.com
bpro1.top brainxs.us
brakehawk.com brus-vsem.ru
brandedleadgeneration.com
btnativenav.com
btvn.ru btvn.ru
burger-imperia.com buypuppies.ca
buttons-for-website.com buytizanidineonlinenoprescription.blogspot.com
buyantiviralwp.com buzzurl.jp
byme.se
cakemediahq.com.au cakemediahq.com.au
callmd5map.com cakesplus.com.au
candycrushshop.com camdolls.com
canrioloadm.gq canacopegdl.com
capture-room.com
carabela.com.do
cardiosport.com.ua cardiosport.com.ua
cartierlove2u.xyz cars-modification.net
cartierloveringreplica.xyz cartierlove.xyz
cartierlovestore.xyz cartierlovestore.xyz
cartierreplica.pw cartierring.xyz
casinorewards.info casite-513049.cloudaccess.net
cbcseward.com cctva.tv
cbox.ws cdn.walkme.com
cheapjerseysa.com celejihad.info
cheapjerseysap.com cenoval.ru
chorus.terakeet.com cerev.info
christianlouboutinreplica.win charmstroy.info
cicaki.net cheapestjerseysonwholesale.com
clickbank.net cheapmarmot.online
cognitiveseo.com chimiver.info
colehaanoutlet.store choice-credit.ru
compliance-alexa.xyz cleanmypc.com
compliance-checker.info cleansearch.net
compliance-ivan.xyz clickzzs.nl
compliance-jack.top cmhomestayagency.com
compliance-willy.top community.allhiphop.com
contentsexpress.com compliance-alexa.top
coop-gamers.ru compliance-walt.top
coprofam.org computernetworksonline.com
cornerstone-countertops.com confib.ifmo.ru
counciltally.com cool-wedding.net
countercrazy.com countercrazy.com
credit-online.ws courtshipgift.com
credit-respect.ru cowlmash.com
cpabegins.ru
credit.co.ua
crystalslot.com
cs-passion.pl
cubook.supernew.org cubook.supernew.org
customchocolate.business-for-home.com cubs-jerseys.us
cxpromote.com culad.com
ddlmega.net d2jsp.org
demon-tweeks.com damedingel.ya.ru
dentfidemountpreach.tk data.vtc.pw
datsun-do.com
deda-moroza-zakaz.ru
dentalpearls.com.au
depositfiles-porn.ga depositfiles-porn.ga
dev-seo.blog detskie-zabavi.ru
dgfitness.co dojki-hd.com
diarioaconcagua.com dokfilms.net
digital-video-processing.com dolohen.com
div.as domain-tracker.com
dnepropetrovsk.mistr-x.org domainsatcost.com
docstoc.com domik-derevne.ru
doctormakarova.ru dominterior.org
dojki365.online dood.live
domainanalyzing.xyz dostavka-v-krym.com
domaincdn.xyz dosug-lux.ru
dosugrostov.site download-wallpaper.net
doublepimp.com dumpsmania24.com
download-of-the-warez.blogspot.com
downloadme.life
driving.kiev.ua
drugstoreforyou.com
dvrlists.com
e-commerce-seo.com e-commerce-seo.com
econom.co edge.sharethis.com
educontest.net eduinfosite.com
egvar.net el-nation.com
ekobata.ru elektir.ru
eldiariodeguadalajara.com elektrozigarette1.dreamwidth.org
elektronischezigarettekaufen1.myblog.de elektrozigarette2.wordpressy.pl
elvenar.com elvenar.com
eng-lyrics.com energy-ua.com
eralph.tk engines-usa.com
eropho.net englishgamer.com
erot.co ertelecom.ru
erotag.com estelight.ru
erotiktreff24.info etm-consult.de
errorfixing.space evening-dating-club.info
europeanwatches.ru exchanges-bet.com
euroskat.ru exlarseva.webblog.es
evaashop.ru exoneration-project.us
expertblog.info ez8motelseaworldsandiego.com
eyessurgery.ru ezaz.info
factorynightclub.com ezrvrentals.com
fandlr.com faptitans.com
fantasticpixcool.com feel-planet.com
fapgon.com ferieboligkbh.dk
fashion-mk.net fickenprivat.info
felizporno.com filkhbr.com
finemanteam.com filmania-x.ru
finstroy.net filmci.pro
fivedwld.cf firstdrugmall.ru
fordsonmajbor.cf fitfloponline.store
freenode.info fiuxy.com
freevpn.space flashahead.info
fuckmill.com flavors.me
fugarif.ga floridamobilebillboards.com
fun2cell.net forum.doctissimo.fr
galaxy-family.ru franch.info
galaxyflowers.ru frankofficial.ru
galeria-zdjec.com free-deals.faith
gameplexcity.com free-floating-buttons.com
geileweiber.tk freelifetimefuckbook.com
genericlowlatencyasiodriverhq.aircus.com funcrushgames.com
getlaid-xxxhookupdirect.com fungamelands.com
getoutofdebtfree.org fx-brokers-review.com
glcomputers.ru game-mmorpg.net
golden-catalog.pro game300.ru
golfresa.lucania.se gembird.com
goo.ne.jp gemgrab-a.akamaihd.net
goodnightjournal.com genetworx.com
goodprotein.ru gentamicineyedrops.blogspot.com
goodwinmetals.co get-free-traffic-now.com
googglet.com getmiro.com
googlsucks.com getyourimage.club
gorabagrata.ru gidonline.one
graphicwe.org gifspics.com
gratuitbaise.com gk170.ru
gok-kasten.net
goldadpremium.com
golden-praga.ru
gourcy.altervista.org
gratuitxblcodes.com gratuitxblcodes.com
greenidesign.co greamimgo.pw
growmyfunds.ca greatzip.com
gsmtlf.ru gsbs.com.ua
gyffu.com gtopstats.com
hackers-crackers.tk gwebtools.com
hauleddes.com gz2.bbsoldes.fr
hem.passagen.se hamptonoaks.ca
high-speed1.net handicapbathtubarea.com
hollywoodactress.info handicapvantoday.com
home-task.com hao123.com
hoporno.com harmonyglen.us
hostnow.men hazardky.net
housekuba.org hebr.myddns-flir.com
howopen.ru helpmymacfaster.trade
huhn.altervista.org hexpilot.com
hildinghr.se
hobbyhuren-datenbank.com
horoshieokna.com
hotblognetwork.com
hoverboardforsaledirect.com
hully.altervista.org
hunterboots.online
hut1.ru
hydropump.su
hystersister.com hystersister.com
iboss.com i-hobot.ru
ie.57883.net i4track.net
id-forex.com
idegenvezeto.eu
ideibiznesa2015.ru
igrovyeavtomaty777.ru igrovyeavtomaty777.ru
ihc2015.info ilmen.net
ihtec2019.org imagefinder.site
imadedinner.net imagine-ex.co
imgarit.pw incolors.club
indiakino.net indiakino.net
infogame.name iptool.xyz
int.search.tb.ask.com iradiology.ru
iopeninghours.co.uk irkutsk.zrus.org
irkutsk.online-podarki.com isotoner.com
ispaniya-costa-blanca.ru istock-mebel.ru
istizanidineacontrolledsubstance.blogspot.com ivanstroi.ru
istmira.ru j-times.ru
itch.io japfm.com
iyasimasennka.com jav-fetish.site
jav-way.site javspace.net
je7.us jennyfire.ru
joingames.org jestr.org
jrcigars.com jimmychoosale.online
jyvopys.com jjbabskoe.ru
kakadu-interior.com.ua jrpmakati.com
kamalsinha.com jwss.cc
keywords-monitoring-your-success.com kaliningrad.zrus.org
kiprinform.com kamorel.com
kladrus.ru kinky-fetishes.com
knogg.net kinomaniatv.pl
krasnodar-avtolombards.ru knigonosha.net
krasnodar.zrus.org kostenlos-sexvideos.com
kreativperlen.ch kostenloser-sex.com
kutikomi.net kumuk.info
kvartira-sutochno.com kurwa.win
la-fa.ru la-fa.ru
lacapilla.info ladsup.com
law-check-nine.xyz lapolis.it
law-check-seven.xyz lazymanyoga.com
laxdrills.com letolove.ru
leadn.pl
leftofcentrist.com
legalrc.biz
lennatin.info
lidoradio.com
lietaer.com lietaer.com
lifeinsurancekb.com lionking-1994.blogspot.com
liran-locks.com liran-locks.com
liver-chintai.org liveu.infoteka.hu
liverpool.gsofootball.com lmrauction.com
loadingpages.me locustdesign.co
lol-smurfs.com low-format.ru
lombia.com lowephotos.info
lostfilm-online.ru luchshie-topcasino.ru
louisvuittonoutletstore.net m3gadownload.pl
lsp-awak-perikanan.com m5home.ru
lyngdalhudterapi.no machicon-akihabara.info
lyrster.com manuscript.su
mabdoola.blogspot.com mediawhirl.net
machicon-ueno.info meduza-consult.ru
mafa-free.com meetingrainstorm.bid
make-money-online.com megahdporno.net
malwareremovalcenter.com member-quiz.com
manualterap.roleforum.ru menhealed.net
marketland.ml metalonly.info
markjaybeefractal.com metarip.ru
marmitaco.cat michaelkorsoutlet.store
mebelcomplekt.ru
med-zdorovie.com.ua
megagrabber.ru
mesto-x.com
metabar.ru
metallosajding.ru
metascephe.com
meteocast.net
microstatic.pl
midst.eu
migente.com migente.com
minet.club mikrobiologies.ru
mixtapetorrent.com minharevisao.com
moncleroutletonline.pw misslike.ru
moscow.online-podarki.com mmofreegames.online
mobilierland.com
mojowhois.com
monclercoats.xyz
moncleroutletonline.win
most-kerch.org most-kerch.org
mostantikor.ru moxo.com
mozello.ru mp3downloadhq.com
musirc.com mrbitsandbytes.com
myfreemp3.eu mrvideospornogratis.xxx
myftpupload.com mturkcontent.com
nailsimg.com musflashtv.com
nakozhe.com musicdaddy.net
net-radar.com musicstock.me
netpics.org my-floor.in.ua
myhydros.org
myliveblog.ru
naj-filmy24.pl
narkologiya-orel.ru
naughtyconnect.com
net-profits.xyz
netcheckcdn.xyz
netfacet.net
netvouz.com netvouz.com
nfljerseysforsalewholesaler.com new7ob.com
noclegonline.info nina.az
noscrapleftbehind.co nonameread45.live
novatech.vn office2web.com
novgorod.xrus.org oklogistic.ru
nowtorrents.com oren-cats.ru
nucia.biz.ly osoznanie-narkotikam.net
o00.in otbelivanie-zubov.com
odywpjtw.bloger.index.hr oxford-book.com.ua
oecnhs.info paccohichetoti.ml
omsk.xrus.org paceform.com
onlinedomains.ru pacificair.com
onlineku.com parajumpersjakkesalgnorge.info
onlywoman.org partner-host.men
optitrade24.com partnerads.men
orion-code-access.net parvezmia.xyz
outrageousdeal-a.akamaihd.net
oxotl.com
paclitor.com
pb-dv.ru pb-dv.ru
pechikamini.ru pendelprognos.se
peekyou.com
penisvergrotendepillennl.ovh penisvergrotendepillennl.ovh
pewit.pw phrcialiled.com
picturesify.com picture-group.com
piulatte.cz pihl.se
pk-pomosch.ru pila.pl
pinwallpaper.top
piski.top
placid-rounded-coast.glitch.me
plastweb.ru
play-mp3.com play-mp3.com
pochemychka.net playfortuna-play.ru
pogodnyyeavarii.gq pochtovyi-index.ru
pons-presse.com popander.mobi
porno-dojki.net pornoforadult.com
pornoreino.com pornogratisdiario.com
pretty-mart.com pornoted.com
printdirectforless.com pornozhara.com
portside.xyz
powenlite24.ru
pr0fit-b0x.com
preparevideosafesystem4unow.space
priora-2.com priora-2.com
privacyassistant.net producm.ru
privatbank46.ru pron.pro
profitwithalex.info prosperent.com
purplestats.com prostitutki-astana.org
radiodigital.co prostitutki-kiev.org
razyboard.com prostitutki-novgoroda.org
pskcijdc.bloger.index.hr
qexyfu.bugs3.com
quick-offer.com
rankingchart.de
ranksonic.com
rapidsites.pro
rcb101.ru rcb101.ru
redbottomheels.xyz recinziireale.com
rednise.com redirectingat.com
refads.pro remont-ustanovka-tehniki.ru
renewablewealth.com renecaovilla.online
responsinator.com rewardit.com
rewardpoll.com rezeptiblud.ru
rimedia.org riders.ro
ritlweb.com riralmolamsaca.tk
rockprogblog.com
roznica.com.ua
rp9.ru rp9.ru
rucrypt.com rumamba.com
ruex.org.ua runstocks.com
ruscopybook.com
ruspdd.com ruspdd.com
rustic-quiver.win rusprostitute.com
sale-japan.com s8-nowy-wygraj.comli.com
sbtdesign.co.uk sammlungfotos.online
scanner-jack.top sanjosestartups.com
scanner-mary.top scanmyphones.com
scanner-jess.top
scanner-viktor.top scanner-viktor.top
scoopquest.com seo-2-0.com
se-welding.ru seocdvig.ru
searchinterneat-a.akamaihd.net serw.clicksor.com
seasaltwithfood.com
seinterface.com
seo-platform.com
sethrollins.net sethrollins.net
sex.hotblog.top sexpornotales.net
sexsearch.com sexxdate.net
shop-electron.ru sexy-screen-savers.com
shop.acim.org sexy.babes.frontend-stack.top
shopsellcardsdumps.com shakhtar-doneck.ru
shopvilleroyboch.com.ua shemale-sex.net
silvermature.net signx.info
simul.co site-speed-checker.site
siteimprove.com siteripz.net
slujbauborki.ru sitevaluation.com
sk.golden-praga.ru
skutecznetabletkinaporostwlosow.pl
sledstvie-veli.net
slowmac.tech
smartadserver.com smartadserver.com
smichovbike.cz smichovbike.cz
social-buttons.com soaksoak.ru
socialmadesimple.com social-button.xyz
social-s-ggg.xyz
social-s-iii.xyz
social-vestnik.ru
softlinesolutions.me
softomix.com softomix.com
softxaker.ru sonnikforme.ru
solution4u.com spasswelt.xyz
spb-plitka.ru
spb.afora.ru spb.afora.ru
spitfiremusic.com spinnerco.ca
stal-rulon.ru spravka-medosmotr.ru
streetfooduncovered.com sprttrack.com
strigkaomsk.ru srecorder.com
staynplay.net
storehouse.ua
strongholdsb.ru
stroy-portal22.ru
subj.ukr-lit.com subj.ukr-lit.com
sucsesofinspiration.com supermama.top
sudexpert66.ru svolze.com
sundrugstore.com symphonyintegratedhealthcare.com
surflinksmedical.com taihouse.ru
suture.co tattomedia.com
svarbit.com telefonsexi.com
svbur.ru telesvoboda.ru
swagbucks.com theporndude.com
swingerseiten.com thepornsex.org
tackletarts.co threecolumnblogger.com
tagil.zrus.org titslove.yopoint.in
tako3.com tizanidinedosage.blogspot.com
taranerymagesswa.blogspot.com tizanidinedosageforsleep.blogspot.com
tattoo33.ru tizanidinehcl2mgtablet.blogspot.com
td-33.ru tonerbox.kz
telefonsex-ohne0900.net top10-online-games.com
teslathemes.com torrentgamer.net
the-trader.net trafficfactory.biz
the-usa-games.blogspot.com trustaffs.com
theplacetoupdating.pw turbo-suslik.org
thepokertimer.com twelvevisionspartyofcolorado.com
thewomenlife.com tzritel.tk
tivolibasket.it ua.tc
tizanidineusedfor.blogspot.com
tizanidinewithdrawalsymptoms.blogspot.com
tizanidinezanaflex.blogspot.com
tk-assortiment.ru
tmearegion26.com
tomck.com
toursmaps.com
tozup.com
tracksurf.daooda.com
tradgardspartner.se
trafficnetzwerk.de
udsgame.online
ugogo.info ugogo.info
ukkala.xyz ukr-lit.com
ukrtvory.in.ua ukrtextbook.com
umg-stroy.ru ultramart.biz
unpredictable.ga uogonline.com
uptime-alpha.net uptime-delta.net
uptimechecker.com userequip.com
url-extractor.xyz utiblog.fr
uroki.net uzungil.com
vadimkravtcov.ru valkiria-tk.ru
vapeface.club varikozdok.ru
vegan-foods.us vegascosmetics.ru
venerologiya.com ventelnos.com
via-energy-order.com vesnatehno.ru
videooko.weebly.com vezuviy.su
vip-dom.in video-woman.com
vladhistory.com videochat.bz
vod.com.ua visa-pasport.ru
voltrknc1.com voloslove.ru
vostoktrade.info vseigry.fun
vrotike.ru vtc.pw
vroze.com wallpapersdesk.info
vse-pesni.com wcb.su
vseuznaem.com weblo.com
vykupavto-krasnodar.ru withstandingheartwarming.com
wahicbefa31.soup.io womens-journal.net
wait3sec.org wonderfulflowers.biz
wareseeker.com word-vorlagen.net
warezkeeper.com wordkeyhelper.com
warningzscaler.heraeus.com wpsecurity.website
waysbetter.cn www.timer4web.com
web-analytics.date x-diesel.biz
webalan.ru x-lime.net
webix.me x-rates.ru
website-stealer.nufaq.com xkaz.org
websiteexplorer.info xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
wesharepics.site xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
westen-z.life
westsextube.com
whatistizanidinehclusedfor.blogspot.com
wholesalejerseysgaa.com
wineration.com
wnoz.de
womensplay.net
woodyguthrie.se
wordpress-crew.net
wrz0iuebwhp5fg.freeddns.com
www.arenda-yeisk.ru
www.kabbalah-red-bracelets.com
x-lime.com
xitjw.info
xjrul.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn----9sbebi2bvzr7h.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--80aaagvmjabrs1aoc9luc.xn--p1ai xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--80abgj3a5acid6ghs.top xn--80aikhbrhr.net
xn--80aeahghtf8ac5i.xn--p1ai xxlargepop.com
xn--80ahdheogk5l.xn--p1ai xzlive.com
xn--80atua3d.xn--p1ai yogamatsexpert.com
xn--b1adccaf1bzj.xn--p1ai yubikk.info
xn--b1addnj3cah.xn--p1ai yunque.pluto.ro
xn--oogle-wmc.com zacreditom.ru
xsion.net zaimhelp.ru
xxxsiterips.xyz zajm-pod-zalog-nedvizhimosti.ru
yes-com.com zarenica.net
yoursite.com zhorapankratov7.blogspot.com
youtubedownload.org zigarettenonlinekaufen1.bloog.pl
yuweng.info znaniyapolza.ru
za-fun-offer.com zoosexart.com
zahvat.ru zs2vm.top
zaidia.xhost.ro zscaler.net
zaimite.ru zygophyceous.womanstars.site
zambini.ru
zerocash.msk.ru
zigarettenonlinekaufen2.drupalgardens.com
zixizop.net.ru
znakom.sibtest.ru
znakomstva-moskva77.ru
zoompegs.com
zskdla.site

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123any.com 0c47f8422d3f.com
24x7-server-support.site 0lovespells0.blogspot.com
2728fb936f0.com 1hwy.com
2nt.ru 6128786.com
4k-player.pl 7search.com
4kepics.com 991mostfm.co.id
5-steps-to-start-business.com aa81bf391151884adfa3dd677e41f94be1.com
888.com adloads.com
8jn.dating-app.ru adultgalls.com
aa14ab57a3339c4064bd9ae6fad7495b5f.com afora.ru
aa8b68101d388c446389283820863176e7.com an-donut.com
abovetherivernc.com anmysite.com
adpremium.org
advokateg.xyz
ameblo.jp
amung.us
animaltoplist.com
anticrawler.org
antons-transporte.de antons-transporte.de
apispringsmartne-a.akamaihd.net apple.com-webbrowsing-security.science
autonew.biz arcteryxstore.online
autowebmarket.com.ua asrv-a.akamaihd.net
awaybird.ru autobudpostach.club
azazaporn.com autodo.info
azlex.uz autogrep.ru
aviav.org
ayakino.net
babosas.com
baladur.ru
bannerbank.ru bannerbank.ru
beauby.ru bannerconnect.net
bestbrainprod.win beremenyashka.com
bestmobilityscooterstoday.com bestcoin.cc
bigcareer.info bestofferswalkmydogouteveryday.gq
bildsuche.ru bildsuche.ru
bingoporno.com bitcoinremote.com
bio-market.kz biteg.xyz
bio.trade-jp.net bluesman.nu
biznesluxe.ru braindaily.xyz
blackbabesporn.com
bloggedporn.com
blue-square.biz
boostmyppc.com
brandehk.dk
buchananshardware.com buchananshardware.com
buntube.net buehne-fuer-menschenrechte.de
calc-for-credit.ru buigas.00it.com
cartujano-pre.de buqayy0.livejournal.com
caveavins.fr buron.pw
cfcl.co.uk business-made-fun.com
cheapmoncler.win butstrap.space
cialisndbrx.com callejondelpozo.es
citysecurity.nu canyougethighofftizanidine.blogspot.com
clean-start.net carsplate.com
cocolyze.com cbb1smartlist12.click
cookie-law-enforcement-aa.xyz cdncash.net
cornerstone-countertops.com cfacarrosserie74.com
craftburg.ru cinemaenergy-hd.ru
cutt.us clickcash.com
danashop.ru compliance-alex.xyz
daretodonate.co compliance-don.xyz
date-withme.com compucelunlock.net
dawlenie.com comsysnet.com
depositfiles-porn.ga cordstrap.cc
dev78.dioniqlabb.se corta.co
divisioncore.com cutalltheshit.com
dkvorota.ru dating-time-now.com
dnepr-avtospar.com.ua dentalpearls.com.au
dojki365.online discovertreasure-a.akamaihd.net
domainanalyzing.xyz dktr.ru
domini.cat doctissimo.fr
download-wallpaper.net dprtb.com
downloader12.ru ednorxmed.com
downtuptv.gq efnor-ac.com
dscaas.website ege-essay.ru
el-nation.com excaliburfilms.com
elektronischezigarettekaufen1.myblog.de exmasters.com
es5.com ezigaretteshop.webs.com
eshop4u.jp f00kclan.de
eu-cookie-law.blogspot.com f1nder.org
ezigarettenkaufen2.dreamwidth.org familyphysician.ru
f0815.de fanrto.com
facecup.top fastcrawl.com
failingmarriege.blogspot.com filmidivx.com
fickkontaktehobbyhuren.com findacheaplawyers.com
financeloan.us finejewelryshop.ru
findthe.pet flowersbazar.com
footbalive.org forum20.smailik.org
forex-indextop20.ru gagrasector.ru
forum.doctissimo.fr gamewrath.com
freakycheats.com gaypornmovie.net
frivgame250.com gearsadspromo.club
fusoradio.info gerhardhealer.com
gallictures.com girlsatgames.ru
gamingspark.com global-ics.co.za
gazoblok.net.ua go2album.com
ge0ip.net goodhumor24.com
gentamicineyedrops.blogspot.com gta-club.ru
get-your-social-buttons.info gwebtools.com
golmau.host.sk hkladys.com
gribkovye-zabolevaniya.com hol.es
hasshe.com hotcore.info
hcate.com hotloans.ru
headpharmacy.com houdom.net
heygidday.biz houseofgaga.ru
hjaoopoa.top houston-vikings.com
homeart.space i-midias.net.br
homemade.gq ideawheel.com
hoverboard360.se inbabes.sexushost.com
hoverboardforsaledirect.com innoslicon.com
hscsscotland.com invivo.hu
hunterboots.online iopeninghours.co.uk
iccornacircri.cf iplusbit.blogspot.co.za
ilbe.club irkutsk.zrus.org
imitex-plus.ru jjbabskoe.ru
impotentik.com
inboundlinks.win
incest-ru.com
infazavr.ru
int.search.mywebsearch.com
interesnie-faktu.ru
islamtoday.co.za
itservicesthatworkforyou.com
james13prix.info
jobgirl24.ru
jobmarket.com.ua jobmarket.com.ua
jofucipiku.tk juliadiets.com
johnnyhaley.top jyrxd.com
jrpmakati.com keyword-suggestions.com
kaliningrad.zrus.org keywordteam.net
keywordhut.com kirov.zrus.org
kinocccp.net kiwe-analytics.com
klikbonus.com kosmetyki.tm.pl
knogg.net kosova.de
krafte.ru kountrylife.com
ktotut.net kovrenok.ru
languagecode.com krasivoe-hd.net
law-check-nine.xyz krasula.pp.ua
lifeinsurancekb.com ladsblue.com
linerdrilling.com lion.conboy.us
livingcanarias.com lkbennettoutlet.online
ljusihus.se lookingglassemb.com
lodki-pvh.dp.ua luckyday.world
louisvuittonoutletstore.net m292.info
marketingtechniques.info m4ever.net
mastershef.club mall.uk
medicines-choice.com manimpotence.com
medkletki.ru massage-info.nl
medosmotr-ufa.ru maxthon.com
mexicotravelnet.com mbiologi.ru
minecraft-rus.org me-ke.com
mirtorrent.net meduza-consult.ru
misandesign.se mesto-x.com
missclub.info michellblog.online
mlvc4zzw.space millionare.com
mobifunapp.weebly.com minegam.com
modnie-futbolki.net mir-limuzinov.ru
mokrayakiska.com mobsfun.net
monetizationking.net mojowhois.com
mp3s.club morlat.altervista.org
myghillie.info mrbitsandbytes.com
mylida.org multgo.ru
nac-bearings.ru musicpro.monster
mwtpludn.review
mymobilemoneypages.com
mystats.xyz
nakozhe.com nakozhe.com
neobux-bg.info naphukete.ru
oecnhs.info netvouz.com
offtime.ru new7ob.com
nextbackgroundcheck.gq
nhl17coinsforps3.gratisblog.biz
nsatc.net
nuup.info
okmedia.sk okmedia.sk
oren-cats.ru omsk.xrus.org
onload.pw
ororodnik.goodbb.ru ororodnik.goodbb.ru
paceform.com ourtherapy.ru
painting-planet.com ovirus.ru
pastaleads.com ownshop.cf
pathwhelp.org paintingplanet.ru
paydayloanslocal.com paleohub.info
phormchina.com pb-dv.ru
photorepair.ru pdns.download
piulatte.cz pensplan.com
pixell.club percin.biz.ly
pliks.pl photo.houseofgaga.ru
polska-poezja.com pizdopletka.club
pontiacsolstice.info platinumdeals.gr
pornosok.ru playfortuna-play.ru
pornotubs.com podshipniki-nsk.ru
pregnant.guru podshipniki-ntn.ru
privacylocationforloc.com pornokajf.com
productarium.com pornophoto.xyz
prombudpostach.com.ua pornoted.com
remontbiz.ru pozdrawleniya.ru
rezeptiblud.ru pr0fit-b0x.com
rockprogblog.com preconnubial.usuby.site
running-line.ru prohoster.info
sa-live.com promodj.com
sankt-peterburg.nodup.ru pyramidlitho.webs.com
santechnik.jimdo.com pyrodesigns.com.au
sayyoethe.blogspot.co.za rabotaetvse.ru
sbtdesign.co.uk rangoman.date
scanner-alex.top realtytimes.com
seoboxes.com reining.lovasszovetseg.hu
seokicks.de rent2spb.ru
serptehnika.ru rethinkwasteni.info
serving.adbetclickin.pink s.lollypopgaming.com
serw.clicksor.com savingsslider-a.akamaihd.net
sexgalleries.top sbwealthsolutions.ca
sexyteens.hol.es scanner-george.top
sfj-ror.no search-goo.com
similardeals.net searchwebknow-a.akamaihd.net
site-speed-checker.site semaltmedia.com
sitevaluation.com seorankinglinks.com
skuteczna-dieta.co.pl sexfreepornoxxx.com
stockquotes.wooeb.com shitmovs.com
strag-invest.ru shop.acim.org
support.nopeas.sk si-unique.com
taqplayer.info sklad-24.ru
timecrimea.ru soft-terminal.ru
tizanidinepain.blogspot.com songoo.wz.cz
tizanidinepills.blogspot.com spravka130.ru
tizanidinesideeffectsweightloss.blogspot.com stanthonyscatholicchurch.org
stmassage.ru
surintech.ac.th
sweepstakes.rewardit.com
tattooha.com
tattooreligion.ru
teletype.in
thin.me.pn
tizanidineandgabapentin.blogspot.com
tizanidinehcl2mg.blogspot.com
tizanidineorflexeril.blogspot.com
tkanorganizma.ru tkanorganizma.ru
todohr.com tobeyouday.win
toondinsey.com toon-families.com
torrent-to-magnet.com torrentdownloadhub.com
touchmods.fr tourismvictoria.com
trafficmonetize.org triplepanda.xyz
tubeline.biz uchil.net
twu.com.ua vadimkravtcov.ru
tyumen.xrus.org vegan-foods.us
ublaze.ru vellings.info
udsgame.online videochat.cafe
unimodemhalfduplefw.pen.io vkonche.com
uzungil.com vysigy.su
vapomnoncri.tk vzlomtw.com
vchulkah.net w3data.co
viberdownload10.com webenlace.com.ar
videochat.ph wikes.20fr.com
vltai.com wildnatureimages.com
wanker.us ws.ampower.me
web-betting.ru www.kabbalah-red-bracelets.com
webalan.ru x-rates.ru
webix.biz x-true.info
website-analyzer.info xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
wnoz.de xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
woweb.com.ua xn--l1aengat.xn--p1ai
wzgyyq.com xportvusbdriver8i.snack.ws
xlolitka.com xxlargepop.com
xn----7sbifcamovvfggw9d.xn--p1ai youjizz.vc
xn----itbkqkfiq.xn--p1ai yourserverisdown.com
ynymnwbm.bloger.index.hr yunque.pluto.ro
z-master.ru zacreditom.ru
zarenica.net zhongwenlink.com
zatjmuzu.info zigarettenonlinekaufen.tumblr.com
zigarettenonlinekaufen2.drupalgardens.com znakom.sibtest.ru
zrizvtrnpale.tk zynax.ua
zrus.org

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.11 0; 10.1.0.56 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.09.3343 ### Version: V4.2022.09.3344
### Updated: Sun Sep 11 10:03:30 UTC 2022 ### Updated: Sun Sep 11 22:01:16 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.09.3343 ### Version: V4.2022.09.3344
### Updated: Sun Sep 11 10:03:30 UTC 2022 ### Updated: Sun Sep 11 22:01:16 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##