vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3458 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-08 10:09:18 +00:00
parent 4d216cf509
commit 211871e438
47 changed files with 103267 additions and 103277 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3457 #### Version: V4.2022.11.3458
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3457 #### Version: V4.2022.11.3458
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3457 #### Version: V4.2022.11.3458
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12774
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3457 #### Version: V4.2022.11.3458
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12778
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12778
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12778
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12778
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.1.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.1.122 0;
127.0.0.1 0; 127.0.0.1 0;

12778
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.1.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.1.122 0;
127.0.0.1 0; 127.0.0.1 0;

12778
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.1.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.1.122 0;
127.0.0.1 0; 127.0.0.1 0;

12778
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3457 3458

12778
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12778
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.77.124.191 0; 172.173.248.33 0;

12778
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.1.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.1.122 0;
127.0.0.1 0; 127.0.0.1 0;

12778
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.1.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.1.122 0;
127.0.0.1 0; 127.0.0.1 0;

12778
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.1.122 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.1.122 0;
127.0.0.1 0; 127.0.0.1 0;

12778
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.77.124.191 0; 172.173.248.33 0;

12778
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.77.124.191 0; 172.173.248.33 0;

12778
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 1; 10.1.1.122 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BlowFish AfD-Verbotsverfahren
Fuzz GetWeb
Iblog Mag-Net
Larbin Name Intelligence
Re-re Nimbostratus
WiseGuys Robot Scanbot
crawler4j Sucuri
polaris version URLy Warning
trendiction.de WebEnhancer
webpros.com x22Mozilla

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 80legs
360Spider AIBOT
404enemy ASPSeek
ADmantX
Abonti
Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Aipbot
Alexibot
AllSubmitter
Anarchie Anarchie
Anarchy Anarchy
Ankit Ankit
Apexoo Anthill
Atomseobot
Attach Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BDCbot
BDFetch
BackStreet BackStreet
BacklinkCrawler BacklinkCrawler
Barkrowler
BetaBot BetaBot
Bolt Black Hole
BlackWidow
Blackboard
Brandprotect Brandprotect
Brandwatch Brandwatch
Buck Buck
BuiltWith BuiltWith
BunnySlippers CODE87
BuzzSumo Cegbfeieh
CCBot CensysInspect
CSHttp CheTeam
CherryPicker CherryPicker
ChinaClaw ChinaClaw
Chlooe
Cliqzbot Cliqzbot
Cloud mapping Cloud mapping
Cogentbot
Collector Collector
Copier Copier
Copyscape
Cosmos Cosmos
Craftbot
CrazyWebCrawler CrazyWebCrawler
CrunchBot Crescent
DBLBot DTS Agent
Deusu Devil
Digincore Digincore
Disco DigitalPebble
Discoverybot
Dispatch Dispatch
DittoSpyder
DomCopBot DomCopBot
DomainCrawler DomainCrawler
Drip DomainSigmaCrawler
EMail Wolf Domains Project
Ebingbong EasyDL
EroCrawler Ecxi
Evil Exabot
ExtLinksBot ExtLinksBot
Extractor Extractor
Extreme Picture Finder
Ezooms Ezooms
Firefox/7.0 FHscan
FemtosearchBot
Fimap
FlashGet
Foobot Foobot
Fuzz Freeuploader
Fyrebot GT::WWW
GalaxyBot Gigabot
GetRight GoZilla
GetWeb GrabNet
GrapeFX GrapeFX
HMView GridBot
HTMLparser HEADMasterSEO
HTTP::Lite
Haansoft Haansoft
Havij Harvest
Hloader Hloader
Humanlinks HybridBot
IRLbot IRLbot
IlseBot Id-search
Image Sucker Image Fetch
IndeedBot
Indy Library
InfoNaviRobot
InterGET
Internet Ninja Internet Ninja
IstellaBot InternetSeer
JOC Web Spider JOC Web Spider
JamesBOT JikeSpider
JetCar
Joomla Joomla
Jorgee
JustView
Jyxobot Jyxobot
Kenjin Spider
Keybot Translation-Search-Machine
Keyword Density Keyword Density
Lanshanbot Kinza
LNSpiderguy
LWP::Simple
Larbin Larbin
LeechFTP Leap
Lftp LeechGet
LieBaoFast LexiBot
LinkWalker LibWeb
LinkextractorPro Libwhisker
LinkpadBot Lightspeedsystems
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
MFC_Tear_Sample Lmspider
MIDown tool MIDown tool
MIIxpc
MJ12bot
MSFrontPage MSFrontPage
Majestic SEO MTRobot
Majestic-SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch MarkWatch
MauiBot Masscan
MeanPath Bot Mata Hari
Meanpathbot Mb2345Browser
MegaIndex.ru
Metauri Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Minefield
Mister PiX Mister PiX
Moblie Safari Moblie Safari
Mojeek Mojeek
Musobot Mr.4x3
NearSite NICErsPRO
NPbot
Name Intelligence
Nameprotect
Nessus
Net Vampire Net Vampire
NetLyzer NetAnts
NetMechanic NetZIP
NetSpider
Netcraft Netcraft
Nettrack Niki-bot
NextGenSearchBot
Nibbler
Nikto Nikto
NimbleCrawler Nuclei
OpenLinkProfiler Offline Navigator
OpenVAS OpenVAS
OrangeBot OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageScorer PageScorer
PageThing.com
Panscient
PeoplePal PeoplePal
Picsearch Pi-Monster
Picscout
Piepmatz Piepmatz
Pimonster
Pixray Pixray
Pockey Pockey
ProPowerBot ProPowerBot
ProWebWalker
Probethenet Probethenet
Pu_iN
Pump
PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RankActive
RankingBot2 RankActiveLinkBot
ReGet RankFlex
Re-re
Reaper Reaper
RebelMouse RebelMouse
RedesScrapy Recorder
RepoMonkey RepoMonkey
Ripper
SBIder
SEOkicks SEOkicks
SEOprofiler SEOprofiler
SMTBot
SalesIntelligent
ScanAlert
Scanbot
ScoutJet
Scrapy Scrapy
ScrepyBot Screaming
SearchmetricsBot ScreenerBot
SemrushBot Searchestate
SentiBot SentiBot
SeoSiteCheckup Seomoz
Shodan Site Sucker
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
Sogou web spider SiteLockSpider
Sottopop SiteSucker
SmartDownload
SpaceBison SpaceBison
Spammen Spammen
SpankBot SpankBot
Spinn3r
SputnikBot SputnikBot
Sqlworm
Sqworm
Stripper Stripper
Sucuri Sucuri
SuperBot SuperHTTP
Surfbot
Swiftbot
T0PHackTeam T0PHackTeam
Teleport Teleport
TeleportPro TeleportPro
Telesoft
Telesphoreo
Telesphorep
The Intraformant The Intraformant
Thumbor Toata
Toweyabot Trendiction
Trendictionbot Trendictionbot
Turingos Turingos
Twice TwengaBot
URLy Warning URLy.Warning
UnisterBot Upflow
V-BOT
VB Project VB Project
VCI VCI
Vagabondo VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
VoidEYE
Voltron Voltron
WISENutbot WBSearchBot
WWW-Mechanize WEBDAV
WWW::Mechanize WWW::Mechanize
Web Enhancer Wallpapers
Web Fetch Wallpapers/3.0
WebAuto WeSEE
WebImageCollector Web Auto
WebPix Web Collage
WebReaper Web Fuck
WebStripper WebBandit
WebZIP WebFuck
Webshag WebWhacker
Webalta
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack Who.is Bot
Whatweb
WinHTTrack WinHTTrack
WiseGuys Robot
Woobot
Wotbox Wotbox
Xaldon WebSpider Zermelo
Xaldon_WebSpider Zeus
YoudaoBot ZoomBot
Zade
Zauba
ZmEu
ZumBot
ZyBorg
adscanner
archive.org_bot archive.org_bot
check1.exe arquivo-web-crawler
demandbase-bot autoemailspider
evc-batch cah.io.community
isitwp.com coccocbot
linkdexbot cognitiveseo
crawl.sogou.com
facebookscraper
instabid
iubenda-radar
lwp-trivial lwp-trivial
magpie-crawler meanpathbot
muhstik-scan muhstik-scan
oBot oBot
pcBrowser page scorer
ripz probe-image-size
scan.lol scan.lol
sexsearcher seostar
sogouspider sp_auditbot
spyfu
trendiction.com trendiction.com
ubermetrics-technologies.com webgains-bot
webmeup-crawler
webpros.com webpros.com
zgrab x22Mozilla
zauba.io

284
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
AIBOT 404checker
404enemy
ADmantX
ALittle Client ALittle Client
ASPSeek
Aboundex
Acunetix
AhrefsBot AhrefsBot
AiHitBot
AllSubmitter AllSubmitter
AlphaBot
Anarchie Anarchie
Ankit
Anthill
Apexoo Apexoo
Aspiegel Atomseobot
Attach AwarioSmartBot
BBBike
BDCbot BDCbot
BacklinkCrawler BDFetch
BackWeb
Backlink-Ceck
Badass Badass
Battleztar Bazinga BatchFTP
Blackboard Bigfoot
BlowFish Bitacle
Boardreader Blow
Bolt
BotALot
Brandprotect
Brandwatch Brandwatch
Buddy
BuiltBotTough BuiltBotTough
Bullseye Bullseye
BunnySlippers BunnySlippers
BuzzSumo CCBot
CATExplorador CODE87
CSHttp CSHttp
Calculon Calculon
CheeseBot CazoodleBot
CensysInspect
CheTeam
ChinaClaw ChinaClaw
Citoid Citoid
Claritybot Claritybot
Cliqzbot
Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Collector Copier
Copyscape Copyscape
Craftbot Crawling at Home Project
CrazyWebCrawler
Crescent Crescent
CyotekWebCopy Custo
DIIbot DIIbot
DSearch DSearch
DataCha0s DTS Agent
Demon Demon
Dirbuster Devil
Discobot Discobot
Discoverybot Discoverybot
DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomainCrawler DomainAppender
DomainStatsBot
Domains Project Domains Project
EMail Siphon EasyDL
Ebingbong EirGrabber
Ecxi EroCrawler
ExtLinksBot Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
Firefox/7.0 FDM
FlashGet FHscan
Foobot FemtosearchBot
FrontPage Fimap
GT::WWW Flunky
GalaxyBot
Genieo Genieo
Go!Zilla GermCrawler
Getintent
Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
HEADMasterSEO HTTrack
HTMLparser
HTTP::Lite
Haansoft Haansoft
Heritrix Heritrix
Humanlinks Hloader
IDBot HonoluluBot
IRLbot IRLbot
Id-search Id-search
Image Fetch Image Fetch
InfoTekies Image Sucker
InterGET Internet Ninja
Iskanie IstellaBot
JOC Web Spider
JamesBOT
JennyBot
JetCar
JikeSpider JikeSpider
Joomla Joomla
JustView JustView
LeechFTP Jyxobot
Kenjin Spider
Keyword Density
Larbin
LexiBot LexiBot
LibWeb Lftp
Linkbot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
Lipperhey Spider Lipperhey Spider
Litemage_walker
Lmspider Lmspider
MFC_Tear_Sample
MIDown tool
MIIxpc MIIxpc
MJ12bot Majestic12
Majestic-SEO
MarkMonitor MarkMonitor
Mass Downloader
MauiBot
Mb2345Browser Mb2345Browser
MicroMessenger Mediatoolkitbot
Metauri
Microsoft URL Control
Moblie Safari
Mojeek Mojeek
MolokaiBot Mozlila
Mr.4x3 Mr.4x3
Msrabot Musobot
NICErsPRO
Name Intelligence
Nameprotect Nameprotect
NearSite
Needle
Nessus
Net Vampire Net Vampire
NetLyzer NetAnts
NetMechanic
NetSpider NetSpider
Netvibes Netcraft
NextGenSearchBot Nettrack
Nikto Nibbler
Not Niki-bot
Octopus Nimbostratus
Offline Navigator Nutch
Offline Explorer
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openvas
OrangeBot
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot OutfoxBot
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer PageScorer
PageThing.com PageThing.com
Papa Foto Papa Foto
Pavuk
PeoplePal PeoplePal
Pi-Monster Pi-Monster
Picsearch Picsearch
PictureFinder Pimonster
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey Pockey
ProPowerBot
Probethenet Probethenet
PyCurl Pu_iN
RankingBot2 PxBroker
QueryN Metasearch
Quick-Crawler
RSSingBot
RankFlex
RankingBot
Rankivabot
RankurBot RankurBot
Re-re Re-re
RepoMonkey Recorder
Ripper
Rogerbot
SEOkicks-Robot SEOkicks-Robot
SEOstats SEOstats
SISTRIX SISTRIX
SMTBot
SalesIntelligent
Scanbot Scanbot
Scrapy Scrapy
Screaming SentiBot
SearchmetricsBot Seomoz
Shodan Shodan
Siphon Siphon
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger Sitevigil
Sitebeam Snake
SmartDownload Sogou web spider
Snapbot Sosospider
Snoopy SpankBot
Sociscraper Spbot
Spammen
SputnikBot SputnikBot
Sqlworm Sqlmap
Sqworm Sqworm
Steeler Steeler
Stripper Stripper
Sucuri SuperHTTP
SuperBot Suzuran
SurveyBot
T0PHackTeam
T8Abot T8Abot
Teleport
TeleportPro
Telesoft Telesoft
Telesphoreo Telesphoreo
Toweyabot TheNomad
Trendiction Toata
True_Robot True_Robot
Twice Turnitin
Typhoeus Typhoeus
URLy Warning URLy Warning
URLy.Warning URLy.Warning
UnisterBot VB Project
VCI
Vacuum Vacuum
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
VoidEYE Virusdie
WISENutbot Voltron
WWW-Collector-E WBSearchBot
WWW::Mechanize WPScan
Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
Web Enhancer Web Collage
Web Fetch
Web Pix Web Pix
Web Sauger
WebAuto WebAuto
WebBandit
WebFetch WebFetch
WebPix WebFuck
WebStripper WebLeacher
WebSucker WebSauger
WebZIP WebWhacker
Whack Webshag
WinHTTrack WebsiteQuester
Whatweb
Widow
Wonderbot Wonderbot
Wprecon Xaldon WebSpider
Zauba Zermelo
Zeus
Zitebot
ZmEu ZmEu
ZoominfoBot
ZyBorg ZyBorg
adscanner arquivo.pt
arquivo-web-crawler autoemailspider
clark-crawler
coccocbot
crawl.sogou.com crawl.sogou.com
crawler.feedback dataforseo.com
crawler4j eCatch
domainsproject.org facebookscraper
instabid
internetVista monitor
isitwp.com isitwp.com
lwp-request iubenda-radar
linkdexbot
lwp-trivial lwp-trivial
muhstik-scan magpie-crawler
netEstate NE Crawler
oBot oBot
page scorer page scorer
pcBrowser scan.lol
polaris version seoscanners
probe-image-size
s1z.ru
satoristudio.net
scalaj-http
seocompany.store
seostar seostar
serpstatbot
sitechecker.pro sitechecker.pro
spyfu
sysscan
trendiction.com
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot
webmeup-crawler webmeup-crawler
webprosbot
x09Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
arenanews.com.ua 3-letter-domains.net
bankiem.pl adrunnr.com
big-boards.info bolitgorlo.net
earthmagic.info kichenaid.ru
hut1.ru lampokrat.ws
interior-stickers.ru privetsochi.ru
mafa-free.com scanner-donald.top
rankscanner.com shoppingmiracles.co.uk
serptehnika.ru webmonetizer.net
uginekologa.com xxxmania.top

940
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00webcams.com 24x7-server-support.site
0912701309f8ce.com
1001watch.com.ua
100searchengines.com
11235813.webzdarma.cz
300richmond.co.nz
3rbseyes.com 3rbseyes.com
4inn.ru 45en.ru
4ureyesonly.com 4kpics.rocks
57883.net 4ur.click
7zap.com 8lufu.com
8558l.jobs.net aa08daf7e13b6345e09e92f771507fa5f4.com
8jn.dating-app.ru aaa38852e886ac4af1a3cff9b47cab6272.com
aanapa.ru aaronabel.com
abcdeg.xyz acc.eu.org
abovetherivernc.com acmebtn.ml
actionnooz.com adamoads.com
adminshop.com add-add.men
adrunnr.com adserver-e7.com
adsloads.com adsref.men
adssafeprotected.com affiliate-fr.com
advocatemsk.ru allproblog.com
agardomains.com alphaforum.ru
ahhjf.com aludecor.info
alert-fjg.xyz alyeskaresort.com
alienwheel.es amateurgalls.com
alif-ba-ta.com amospalla.es
allforminecraft.ru anonymous-redirect.com
allmarketsnewdayli.gdn apartamentwroclaw.eu
amanda-porn.ga apiitsthirteende-a.akamaihd.net
amateurmatch.com appfastplay.com
amazingninja.com architecturebest.com
amigobulls.com arewater.com
an-donut.com arquivo.pt
animali.deagostinipassion.it artefakct.com
animebox.com.ua articlesdirectoryme.info
anonymousfox.co asacopaco.tk
antons-transporte.de asiengirls.net
apiappenableinfo-a.akamaihd.net asrv-a.akamaihd.net
apioutoboxnet-a.akamaihd.net atmagroup.ru
apps-analytics.net atout-energie-69.com
arabgirls.adultgalls.com aussie-prizes.com
arvut.org autonew.biz
asiavirtualsolutions.com avtointeres.ru
asociatia-tipografilor-transilvania.ro azartniy-bonus.com
australia-opening-times.com azlex.uz
auto-zapchasti.org babosas.com
auto4style.ru bank.uz
autoblogger24.info beauty-bracelet.com
avtolombard-krasnodar.ru berdasovivan.ru
b00kmarks.com bestmarriages.com
balla-boo.se beta.hotkeys.com
bankofthewext.com betwinservice.com
bannerconnect.net beyan.host.sk
bannerpower.com bigames.online
beachtoday.ru bimatoprost-careprost.com.ua
bedandbreakfast.com biznesschat.net
bestcoin.cc bjorkbacken.nu
bestdooz.com blog-hits.com
bestofupload.info bloggers.nl
bestplacetobuyeriacta.jw.lt bochemit.com.ua
binomo.kz bosman.pluto.ro
blackhatworth.com brewdom.ru
blockety.co buy-forum.ru
blue-square.biz buypuppies.ca
bluesalt.co buzzonclick.com
bonky.biz buzzzg.men
brandbucket.com bvlgariwallet.xyz
bristolhotel.com.ua cacheimages.com
budpost.com.ua cakesplus.com.au
bukleteg.xyz callawaygolfoutlet.online
bumble.cheapwebsitehoster.com calstaterealty.us
buyessayonline19.blogspot.ru canacopegdl.com
buypillsorderonline.com candy-glam-hp.com
calcularpagerank.com carabela.com.do
candypeople.se carina-sy.de
canvas.gloverid.site carloans.com
carapics.com cartierlovebracelet.xyz
cartierloveringreplica.xyz cartierreplica.pw
cartujano-pre.de cartiertrinity.xyz
cashkitten-a.akamaihd.net cbozhe.com
casinos4dummies.co.uk ccjp.eu
cdncash.net cdn.walkme.com
cenokos.ru ceotrk.com
centre-indigo.org.ua cheapbarbour.online
chatroulette.life cheapbelstaff.online
cheapwebsitehoster.com checkhit.com
chinese-amezon.com christianlouboutinshoes.xyz
christianlouboutinsaleonline.us cialischmrx.com
cialiswithoutadoctor.net circlesl.com
cienum.fr citetick.com
cineacademy.ru clickbanksites.info
classicakuhni.ru cntravelre.com
clickintext.net
cobaltpro.ru
cocyq.inwtrade.com
codysbbq.com
cometorussia.net cometorussia.net
compliance-elena.xyz
compliance-fred.xyz compliance-fred.xyz
compliance-irvin.top compliance-nelson.top
compliance-irvin.xyz compliance-viktor.top
compliance-mary.top cornomase.win
concealthyself.com cougfan.info
connectionstrenth.com crirussian.ru
continent-e.tv culmimg.pw
cool-wedding.net custodieva.ru
copypaste.traderzplanet.in d0t.ru
counciltally.com dandiyabeats.in
cranly.net data-mining.tk
credit-respect.ru dating-app.ru
creditmoney.com.ua
data-ox.com
datract.com datract.com
datsun-do.com daydream-studio.ru
demenageur.com dellalimov.com
depositfiles-porn.ga design-lands.ru
designdevise.com
detskie-zabavi.ru
dev33.dioniqlabb.se
devochki.top devochki.top
dfwu1013.info dichvuvesinhhanoi.com
digest-project.ru dildofotzen.net
dinkolove.ya.ru discountbarbour.online
djekxa.ru divatraffic.com
domain.webkeyit.com dmmspy.com
dosug-lux.ru doeco.ru
downloadeer.net dogoimage.com
drupa.com domznaniy.ru
dstroy.su doreenblog.online
doxyporno.com
drivotracker.com
drunkmoms.net
earn-from-articles.com earn-from-articles.com
easycommerce.cf education-cz.ru
ec-file.info elektrozigarette2.wordpressy.pl
econom.co empirepoker.com
edwinkonijn.com.au enhand.se
ekspresihati.info
eldiariodeguadalajara.com
election.interferencer.ru
electronicadirect.com
elektrischezigarette2.onsugar.com
elektrozigarette2.webs.com
elektrozigarettekaufen1.devhub.com
elektrozigarettekaufen2.blogse.nl
elexies.info
elite-sex-finders.com
erotag.com erotag.com
espaceinventoristes.com escort-russian.com
euronis-free.com esfchat.tk
evaashop.ru esnm.ru
evehemming.blogspot.com.au exchangeit.gq
exchanges-bet.com
exci.ru
eyemagination.com
ezigarettekaufen.myblog.de ezigarettekaufen.myblog.de
ezigarettenshop1.yolasite.com ezigarettenkaufen2.dreamwidth.org
ezigarettenshop2.postbit.com f012.de
falcoware.com facecup.top
fakehandbags.xyz
family1st.ca family1st.ca
fanoboi.com fanpagerobot.com
fashion-mk.net fettefrauen.net
fastcrawl.com fil.ru
fba-mexico.com fitfloponline.store
fellowshipoftheminds.com flash4fun.com
femmesdenudees.com floridahuntingfishingadventures.com
fidelityfunding.com foodcrafts.website
figuringmoneyout.com forzeronly.com
filmania-x.ru fotopop.club
financepoints.eu fototravel.eu
findanysex.com franklinfire.co
fishtauto.ru free-deals.faith
flipper.top free-video-tool.com
flirt4free.com free411games.com
fordsonmajbor.cf freejabs.com
forsex.info freelifetimefuckbook.com
forums.d2jsp.org freemags.cc
free-gluten.ru freeseedsonline.com
frighteningremain.cf freesitetest.com
funnymama.com fuckbuddybestgilf.info
funnypica.com fuckingawesome.com
gambarkatabaru.com fun-mobi.pl
gamebackyard.com fxtips.ru
g.starmoe.xyz
gambarkataku.co
gamevalue7.weebly.com
gaming-journal.com
gamingspark.com
gavuer.ru
gdcentre.ru
gdebestkupit.ru
gdzkurokam.ru gdzkurokam.ru
geckoandfly.com gelezki.com
getmyads24.com gentamicineyedrops.blogspot.com
girls-ufa.ru gepezz.info
given2.com get-seo-domain.com
glavprofit.ru getresponse.com
globalscam.ga getridofstretchmarks.org
goatse.ru gheus.altervista.org
gonextmedia.com glasof.es
good-mummy.ru goodhousekeeping.com
googlepositions.com gotorussia.com
gorodservis.ru grand-chlen.ru
gowreckdiving.com graphid.com
gpms.org.my greamimgo.pw
greatgrace.ru gsasearchengineranker.xyz
greatzip.com gsmlab.pl
growmyfunds.ca hatdc.org
gsmtlf.ru hawaiielectriclight.com
gtopstats.com hcate.com
guidefs.ru healbio.ru
hacktougroup.ru home-task.com
hahashka.ru homedo.fabpage.com
hasshe.com hotcore.info
hccoder.info housekuba.org
hdwallpapers-free.com hoverboard360.at
hdwalls.xyz
herokuapp.com
hesteel.pl
heygidday.biz
hikesearch.net
hitcpm.com
homik.pw
houston-vikings.com
hoverboard360.de
huhn.altervista.org
hunterboots.online
iamsport.org iamsport.org
ideashome.id ilmen.net
ifirestarter.ru ilovevitaly.com
ihc2015.info imediadesk.com
iinstalll-fii1leis.jus0wil.pp.ua imicrovision.com
iloveitaly.ru iminent.com
impisr.edunsk.ru infazavr.ru
insomniagamingfestival.com infowarcraft.ru
integritylandscapeservices.com int.search.tb.ask.com
intellektmedia.at iopeninghours.co.uk
invivo.hu iqoption-bin.com
irunfar.com
iskalko.ru iskalko.ru
iusstf.org ispaniya-costa-blanca.ru
ivanovo.zrus.org iweblist.info
jestr.org izoll.ru
joy-penguin.com javxxx18.com
jeremyeaton.co
jerseychinabizwholesale.com
jolic2.com
jumptap.com
jwcialislrt.com jwcialislrt.com
karpun-iris.ru kaac.ru
kerei.ru kamagragelusa.net
khadastoafarde.tk kchaxton.com
kichenaid.ru kevblog.top
kiinomaniak.pl kiinomaniak.pl
kinobest.pl kinohall.ru
kinozapas.com kinohit1.ru
kiwe-analytics.com kinomaniatv.pl
kinopolet.net
kladrus.ru kladrus.ru
klikbonus.com klejonka.info
klitimg.pw kollekcioner.ru
knowsitall.info kollesa.ru
kochanelli.com kountrylife.com
komputers-best.ru krasivye-devushki.net
koral.se kskjco.club
kretpicf.pw kursy-ege.ru
landreferat.ru kustanay.kz
lawyers.cafe laapp.com
ldrtrack.com laexotic.com
le-clos-des-alouettes.com lafriore.ru
liky.co.ua languagecode.com
lol-smurfs.com leadn.pl
lomb.co ledpolice.ru
lonfon.xyz levaquin750mg.blogspot.com
lost-alpha.ru lezbiyanki.net
lostfilm-online.ru lflash.ru
mabdoola.blogspot.com life-instyle.com
macotool.com likesuccess.com
macresource.co.uk linkbolic.com
make-money-online.com lion.bolegapakistan.com
mansparskats.com liupis.com
martlinker.com livesex-amateure.info
master-muznachas.ru livetsomudvekslingstudent.bloggersdelight.dk
masthopehomes.com locatellicorretor.com.br
matrixalchemy.com lonely-mature.com
me-ke.com longadventure.com
mebel-alait.ru lordzfilmz.me
mebelcomplekt.ru louisvuittonoutletstore.net
medanestesia.ru lrac.info
medi-fitt.hu lubetube.com
medispainstitute.com.au lyrics.home-task.com
medmajor.ru mactechinfo.info
mega-bony2017.pl makemoneyonline.com
methodsmarketing.com mamasuper.prom.ua
mezaruk.info manualterap.roleforum.ru
misterjtbarbers.com manyresultshub-a.akamaihd.net
mjchamonix.org marcoislandvacations.net
mmofreegames.online maslenka.kz
mmstat.com maxxtor.eu
moesen-ficken.com mebeldekor.com.ua
moinozhki.com medicinacom.ru
mojaocena.com megaindex.ru
mompussy.net metalonly.info
moncleronline.xyz metarip.ru
moncleroutletonline.pw microstatic.pl
montredemarque.nl milblueprint.com
moscow-clining.ru mindeyegames.com
mozello.ru mir-business-24.ru
muz-tracker.net misslike.ru
my-cash-bot.co mobioffertrck.com
myanyone.net mockupui.com
mydownlodablefiles.com moje-recenze.cz
mylovelibrabry.com monarhs.info
myphotopipe.com monclercoats.xyz
mystats.xyz moneymaster.ru
nac-bearings.ru montazhnic.ru
nailsimg.com morocco-nomad-excursions.com
newtechspb.ru moscow.online-podarki.com
next-dentists.tk motherboard.vice.com
nlfjjunb5.ru moviemail-online.co.uk
nmrk.ru moz.com
nofreezingmac.click mrbojikobi4.biz
normalegal.ru mrcsa.com.au
notfastfood.ru msk.afora.ru
o.light.d0t.ru musirc.com
mydoctorok.ru
myftpupload.com
najaden.no
narkologiya-peterburg.ru
nelc.edu.eg
netallergy.ru
netvouz.com
nhl17coinsforps3.gratisblog.biz
nikitsyringedrivelg.pen.io
nina.az
nippon-bearings.ru
njpalletremoval.com
no-fuel.org
noscrapleftbehind.co
notasprensa.info
nottyu.xyz
o-o-8-o-o.ru
obiavo.in obiavo.in
obiavo.kz
offer.camp
office-windows.ru office-windows.ru
okmedia.sk online-templatestore.com
okroshki.ru onload.pw
onetravelguides.com oops-cinema.ru
online-hit.info oren-cats.ru
osnova3.ru origin-my.ru
oxford-book.com.ua ourtherapy.ru
oz-offers.com outshop.ru
ozas.net partner-trustworthy.men
paceform.com partnersafe.men
paphoselectricianandplumber.com patterntrader-en.com
partners-ship.pro pendelprognos.se
pb-dv.ru pensplan.com
phpdevops.com pestomou.info
physfunc.ru petedrummond.com
pills24h.com pharmacyincity.com
piluli.info pic-re.blogspot.com
pigrafix.at
piski.top piski.top
pk-services.ru pizdopletka.club
pkr1hand.com platinumdeals.gr
pl-top.pl polska-poezja.com
pl-vouchers.com porngalleries.top
plastgran.com pornhubforum.tk
poiskzakona.ru porno-video-chati.ru
pons-presse.com
popads.net
poquoson.org
porndroids.com
porno-chaman.info
porno.simple-image.com.ua porno.simple-image.com.ua
pornofiljmi.com pornoinn.com
pornosexrolik.com
porodasobak.net
porto.abuilder.net
pourvous.info pourvous.info
pozdrawleniya.ru privatamateure.com
prchecker.info prombudpostach.com.ua
pregnant.guru pufip.com
professionalwritingservices15.blogspot.ru pyramidlitho.webs.com
proxyelite.biz qcstrtvt.bloger.index.hr
psvita.ru qor360.com
pukaporn.com
pussysaga.com
quick-seeker.com
quit-smoking.ga
r-e-f-e-r-e-r.com r-e-f-e-r-e-r.com
razleton.com rangjued.com
rapevideosmovies.com
razamicroelectronics.com
rcb101.ru
recipedays.com recipedays.com
redhotfreebies.co.uk
redirlock.com
referencemoi.com
registrysweeper.com
reklama-i-rabota.ru
remedyotc.com
remont-ustanovka-tehniki.ru remont-ustanovka-tehniki.ru
remontgruzovik.ru remybutler.fr
renewablewealth.com rentehno.ru
rentalcarnavi.info resant.ru
replicalouboutin.xyz rialp.getenjoyment.net
respublica-otel.ru roma-kukareku.livejournal.com
ringtonepartner.com ru-dety.ru
robertefuller.com ruscopybook.com
rocis.site sa-rewards.co.za
rock-cafe.info sack.net
rockingclicks.com sanidumps.com
roleforum.ru sbdl.no
rozalli.com sbt-aqua.ru
rubanners.com scanner-fred.top
sabizonline.com scenarii-1-sentyabrya.uroki.org.ua
saecsa.co schalke04fc.info
safe-app.net search-goo.com
samo-soznanie.ru searchlock.com
sanyuprojects.com seccioncontrabajo.com
saveindex.xyz seeresultshub-a.akamaihd.net
sbetodiodnye-lampy.ru sel-hoz.com
scanmarine.info semxiu.com
scanner-alex.top senger.atspace.co.uk
scanner-barak.top seo-smm.kz
scanner-willy.top seorank.info
seocheckupx.com service-core.ru
serving.adbetclickin.pink sexxdate.net
sexflirtbook.com sexy-pings.com
sexiporno.net sexyali.com
sexkrasivo.net sexystrippe.info
sharebutton.to sexytrend.ru
shitmovs.com sherlock.se
shoplvlv.us shlyahten.ru
silvercash.com sicfor.bcu.cc
sitebeam.net similardeals.net
sluganarodu.ru slomm.ru
smsactivator.ru slonechka.ru
soc-proof.su smailik.org
soft-terminal.ru sms2x2.ru
sogimlecal.tk snts.shell-pmr.ru
soochi.co soietvousmaime.fr
space4update.pw spanking.to
spidtest.space spb-plitka.ru
spl63.fr special-porn.com
spywarebegone.com spin2016.cf
stariy-baku.com sport7777.net
statistici.ro sports-supplements.us
steame.ru spywarenuker.com
sticken.co stair.registrydomainservices.com
streetfire.net stal-rulon.ru
strv.se starwars.wikia.com
stuff-about-money.com stmassage.ru
styro.ru strana-solnca.ru
stroicol.net
stroiminsk.com
success-seo.com
suggest-keywords.com
supermama.top
supermodni.com.ua supermodni.com.ua
surcentro.com surintech.ac.th
swagbucks.com tabakur77.com
swinger-mobil.net tacbelarus.ru
swinon.site taqplayer.info
syndicate.fun tattomedia.com
taboola.com td-33.ru
tako3.com tedxrj.com
tanieaukcje.com.pl
telefonsexi.com telefonsexi.com
template-kid.com texbaza.by
the-torrent-tracker.blogspot.com tfxiq.com
thefds.net thegameriders.com
theplacetoupdating.pw thelottosecrets.com
theprofitsmaker.net thenews-today.info
thetattoohut.com theporndude.com
tizanidinedrugtest.blogspot.com thetardistimes.ovh
tizanidinehcl2mgsideeffects.blogspot.com threecolumnblogger.com
tizanidinestreetvalue.blogspot.com tizanidineandhydrocodone.blogspot.com
tonerbox.kz tizanidineorflexeril.blogspot.com
tongkatmadura.info tizanidinerecreationaluse.blogspot.com
topclickguru.com tizanidinewithdrawalsymptoms.blogspot.com
totu.us top250movies.ru
touchmods.fr topanasex.com
truck-land.life topkarkas.com
tsan.net tour-line.net
tt-ipd.info trainoffend.ml
ttrraacckkrr.com trucri.me
ttsq.fr truemfilelj.gq
tuberkulezanet.ru trustaffs.com
turizmus.us trustl.life
turn-up-life.life try-rx.com
ua-company.ru twojebook.pl
uac.net.au ukkelberg.no
udayavani.com unpredictable.ga
uptime-alpha.net uptime-alpha.net
uptimebot.net uroki.net
url2image.com
urll.eu
usbggettwku.ga usbggettwku.ga
utiblog.fr usdx.us
uzporno.mobi valueclick.com
valuado.com vasileostrovsky-rayon.ru
vavilone.com video-camer.com
vegascosmetics.ru video-woman.com
videochat.tv.br videochat.mx
videochaty.ru videositename.com
videotuber.ru videotuber.ru
vietimgy.pw
vip-parfumeria.ru vip-parfumeria.ru
vipcallsgirls.com vk-mus.ru
vipsexfinders.com vltai.com
virtualbb.com vodkoved.ru
vksaver-all.ru vtc.pw
vonradio.com vykupavto-krasnodar.ru
vote-up.ru wallpapersdesk.info
vsdshnik.com watchmygf.net
vykup-avto-krasnodar.ru webinstantservice.com
walpaperlist.com website-analyzer.info
weblibrary.win webstatsdomain.org
website-speed-up.site weprik.ru
wetgames.ru wholesalejerseysgaa.com
wholesalejerseyscheapjerseys.us.com windowssearch-exp.com
wingsofrefuge.net womens-journal.net
withstandingheartwarming.com wordpress-crew.net
worst-sites.online wpsecurity.website
wosik-dach.service-for-web.de wpthemedetector.co.uk
ww1943.ru wtsindia.in
wygraj-teraz.com www.solartek.ru
x-musics.com x-lime.net
xn----itbeirbjbi7bc6bh2d.xn--p1ai xaijo.com
xn--80aafb2a.xn--p1ai xev.ru
xn--d1abj0abs9d.in.ua xgftnlrt.bloger.index.hr
xn--l1aengat.xn--p1ai xjlottery.com
yebocasino.co.za xn----btbdvdh4aafrfciljm6k.xn--p1ai
yginekologa.com xn--80ab4aa2g.xn--p1ai
youporn-ru.com xn--80ak6aa92e.com
xn--e1afanlbnfckd7c3d.xn--p1ai
xnxxandxvideos.com
xwatt.ru
xxxhdvideo.site
yaoguangdj.com
yoopsie.com
yottos.com
youresponsive.com youresponsive.com
youtoner.it yoursite.com
yugk.net yourtemplatefinder.com
zaim-pod-zalog-krasnodar.ru youthreaders.com
zakazvzloma.com zagadki.in.ua
zarabiaj-dzis.pl zed21.net
zigarettenonlinekaufen1.blox.pl zhuravlev.info
zmoda.hostreo.com znakomstva-moskva77.ru
zryydi.com zog.link
zoomovies.org
zoompegs.com

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123kuma.com 214jaluwobapef.cf
1688.com 2ads.co.uk
18ps.ru 34.gs
40cg.com 6128786.com
4kepics.com actionnooz.com
4kpics.rocks adimmix.com
aab94f698f36684c5a852a2ef272e031bb.com adman.gr
abclauncher.com adprotect.net
acads.net
admanaerofoil.com
adventureparkcostarica.com
adzerg.com adzerg.com
alltheviews.com affiliate-fr.com
alpharma.net all4invest.ru
animal36.com allboard.xobor.de
apilinkswiftco-a.akamaihd.net ameblo.top
apparel-offer.com animenime.ru
arcteryxsale.online anmysite.com
atlasvkusov.ru antons-transporte.de
aviav.ru appsaurus.com
avon-novosib.ru arclk.net
azbukadiets.ru arenanews.com.ua
beauty-bracelet.com asmxsatadriverin.aircus.com
bestmouthwash.club atlant-auto.info
bestpornuha.com autoloans.com
bestserials.com ayakino.net
bestsexyblog.com azlex.uz
beta.hotkeys.com backuperwebcam.weebly.com
betterscooter.com balitouroffice.com
bezlimitko.xyz battleforupdating.site
bigpictures.club beatifullandscape.co
bio-japan.net beetpics.pw
biographiya.com biznesluxe.ru
blackbabesporn.com blog.koreadaily.com
blogsfunky672.weebly.com bochemit.com.ua
bonanza-fortune.men boleznikogi.com
bookmark4you.com.biz brewdom.ru
breget74.ru brg8.com
btnativenav.com calc-for-credit.ru
buqayy0.livejournal.com cardul.ru
buy-loft.ru cartierloveringreplica.xyz
buyessay3.blogspot.ru cartierlovestore.com
buyparajumpers.online cartierlovestore.xyz
canacopegdl.com casinopinup-wins.com
cctva.tv cb.iphantom.com
cdncash.net certifywebsite.win
cfacarrosserie74.com cheapbarbour.online
cheapbelstaff.online chee-by.biz
cheapjerseysa.com compliance-alexa.xyz
cialiswithoutadoctor.net compliance-barak.top
claytransformations.info compliance-checker.info
clickprotects.com constantaservice.net
clickso.com cordstrap.cc
club-samodelkin.ru coslab.club
cntravelre.com counter.yadro.ru
com-onlinesupport.site countercrazy.com
compliance-checker-7.info customerpromos-a.akamaihd.net
conboy.us cutt.us
corta.co danschawbel.com
cypernhuset.se data-ox.com
dame-ns.kz deniven.1bb.ru
dangphoto.trade discountbarbour.online
dataloading.net dolohen.com
dealighted.com domcran.net
dogsrun.net dosug-lux.ru
doktoronline.no dptaughtme.com
domik-derevne.ru drpornogratisx.xxx
duawitchrarato.tk e-buyeasy.com
e-kwiaciarz.pl ebonyporn.site
earthmagic.info editors.choice6912650.hulfingtonpost.com
election.interferencer.ru egovaleo.it
elidelcream.weebly.com elidelcream.weebly.com
enskedesquashclub.se englishdictionaryfree.com
ewebarticle.info eonpal.com
exoclick.com eorogo.top
expdom.com erotikstories.ru
f00kclan.de ezigarettenkaufen1.deviantart.com
financeloan.us film-one.ru
financepoints.eu
finstroy.net finstroy.net
fireads.men forum.doctissimo.fr
flagcounter.me foto-sisek.porngalleries.top
flirt4free.com fotzen-ficken.com
fordsonmajbor.cf freshberry.com.ua
foxweber.com funnymama.com
gallery.rennlist.com fxgallery.com
gamerscorps.com gaypornmovie.net
generalporn.org gcup.ru
go2mike.ru gelezki.com
goodly.pro girlsfuckdick.com
gox.com.ua girlspicsa.com
gsmlab.pl glcomputers.ru
guitar-master.org globalsurfari.com
guruofcasino.com goatse.ru
gwebtools.com gojiberriess.apishops.ru
hcate.com goodnightjournal.com
hchha.com googst2.ru
healing-dysplasia.ru goroda-vsego-mira.ru
hotkeys.com gta-top.ru
hptwaakw.blog.fc2.com hao123.com
hundejo.com hasshe.com
icaseclub.ru hexpilot.com
ihc2015.info hobild.net
ilovevitaly.xyz host-protection.com
impotentik.com ifirestarter.ru
inboxdollars.com iinstalll-fii1leis.jus0wil.pp.ua
incanto.in.ua iminent.com
int.search.tb.ask.com invivo.hu
intermesh.net istizanidineacontrolledsubstance.blogspot.com
investpamm.ru itronics.ca
jennyfire.ru jestr.org
jerseychinabizwholesale.us jobmarket.com.ua
jjbabskoe.ru jwcialislrt.com
kadashihotel.com kerei.ru
keyword-suggestions.com kfon.eu
keywordteam.net kino-fun.ru
kinosed.net kol-energo.ru
kinostorm.net ks1234.com
koleso24.com.ua leadwayau.com
komp-pomosch.ru lescinq.com
konica.kz life-instyle.com
lemon-ade.site linkbuddies.com
lenvred.org
letsrepair.in
lifehacĸer.com
lionking-1994.blogspot.com lionking-1994.blogspot.com
lkbennettoutlet.online locatellicorretor.com.br
lrac.info lowephotos.info
madot.onlinewebshop.net macdamaged.tech
manifestyourmillion.com macotool.com
manipulyator-peterburg.ru marblestyle.ru
manve.info marketingvici.com
manyresultshub-a.akamaihd.net matpre.top
meble-bogart.info max-p.men
medicinacom.ru mediafresh.online
medictube.ru
members.ghanaweb.com
mesto-x.com
metarip.ru
militarysale.pro militarysale.pro
mindeyegames.com miniature.io
mlf.hordo.win minneapoliscopiers.com
mmstat.com morlat.altervista.org
moncleroutletonline.xyz most-kerch.org
moyaterapiya.ru muz-tracker.net
my-big-family.com myhealthcare.com
myliveblog.ru mysexpics.ru
no-rx.info najaden.no
nootrino.com nalogovyy-kodeks.ru
nsatc.net netcheckcdn.xyz
offer.wpsecurity.website nordstar.pro
onblastblog.online nowtorrents.com
onlainbesplatno.ru nudejapan.net
parajumpersoutlet.online o-o-6-o-o.com
peekyou.com obsessionphrases.com
perm-profnastil.ru offers.bycontext.com
petedrummond.com onclkads.com
piccdata.com onetravelguides.com
openmultipleurl.com
opinionreelle.com
orenburg-gsm.ru
orion-code-access.net
outpersonals.com
pay2me.pl
pelfind.me
penisvergrotendepillennl.ovh
perfectplanned.com
pharmacyincity.com
phelissota.xyz
pirateday.ru
pl.aasoldes.fr
plastjulgranar.se plastjulgranar.se
pontiacsolstice.info play-movie.pl
popmarker.com pliks.pl
poptool.net pokemongooo.ml
praisong.net popugauka.ru
printie.com porno-raskazy.ru
prizeestates.cricket pornosmola.info
portside.cc
portside.xyz
pp-budpostach.com.ua
preparevideosafesystem4unow.site
prod2016.com prod2016.com
produkto.net prostitutki-rostova.org
profitkode.com razamicroelectronics.com
projectforte.ru reklamuss.ru
proposal-engine.com resant.ru
ptr.ruvds.com roofers.org.uk
rank-checker.online s.lollypopgaming.com
rating-bestcasino.com sack.net
registratciya-v-moskve.ru santechnik.jimdo.com
renewablewealth.com scanner-john.top
reward-survey.net scanner-walter.top
robertefuller.com search-goo.com
rock-cafe.info searchaddis.com
rp9.ru searchengineranker.email
runtnc.net semp.net
saitevpatorii.com seocdvig.ru
sanjosestartups.com seojokes.net
scanner-brian.top sethrollins.net
scanner-fred.top sexiporno.net
seccioncontrabajo.com sextracker.be
security60-e.com
seorankinglinks.us
sex-tracker.com
sex-tracker.de
sexgalleries.top
shareyards.com
sibtest.ru sibtest.ru
sildenafil-tadalafil.info silktide.com
sildenafilcitratemed.com simply.net
slomm.ru siteaero.com
snjatie-geroinovoy-lomki.ru skatestick.bid
sogimlecal.tk skladvaz.ru
sohoindia.net slavia.info
spacash.com slavkokacunko.de
spamnuker.com solitaire-game.ru
sps-shop.com spbchampionat.ru
squidoo.com studentguide.ru
sribno.net studiofmp.com
stpicks.com studiokamyk.com.pl
stretchmate.net suchenindeutschland.com
stroyplus.ru taqywu51.soup.io
surflinksmedical.com tastyfoodideas.com
tabletkinaodchudzanie.com.pl telefonsex-ohne0900.net
tackletarts.co terrafootwear.us
talant-factory.ru thegioixekhach.com
technika-remont.ru tizanidine4mgstreetvalue.blogspot.com
templates.radiodigital.co tizanidineandhydrocodone.blogspot.com
thecoral.com.br tizanidineforopiatewithdrawal.blogspot.com
thefotosgratis.eu tizanidinehydrochloride2mg.blogspot.com
timhost.ru toondinsey.com
tooplay.com toyota.7zap.com
top-l2.com trafficjunky.com
torrnada.ru tvteleport.ru
traffic100.com tx41tclega.ru
trubywriting.com unimodemhalfduplefw.pen.io
turist-strani.ru urlopener.com
unlimitdocs.net
uptime.com
urlopener.blogspot.com.au
ussearche.cf
uytmaster.ru
vclicks.net
veerotech.com
vesnatehno.ru
via-gra.webstarts.com
viandpet.com viandpet.com
vitalads.net vids18.site
vitoriacabos.com vikistars.com
voltrknc1.com vip.51.la
warning.or.kr vozbujdenie.com
watchmygf.net webextract.profound.net
webix.biz webpromotion.ae
wejdz-tu.pl whereverdesperate.gq
wesharepics.site withstandingheartwarming.com
winx-play.ru wstroika.ru
word-vorlagen.xyz wwwadultcheck.com
x-lime.com x-lime.net
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xlovecam.com
yur-p.ru xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
zagadki.in.ua xxxdatinglocal.us
zerocash.msk.ru xxxhdvideo.site
yottos.com
z-master.ru
zaim-pod-zalog-krasnodar.ru
zakon-ob-obrazovanii.ru
zeroredirect12.com zeroredirect12.com
zhacker.net zeroredirect5.com
zetgie.com.pl
zirondelli.it
zlatnajesen.com zlatnajesen.com
znakom.sibtest.ru zrizvtrnpale.tk
zvuker.net zrus.org
zscalertwo.net

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,14 +581,4 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.200 0; 10.1.1.122 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3457 ### Version: V4.2022.11.3458
### Updated: Mon Nov 7 22:01:33 UTC 2022 ### Updated: Tue Nov 8 10:03:27 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3457 ### Version: V4.2022.11.3458
### Updated: Mon Nov 7 22:01:33 UTC 2022 ### Updated: Tue Nov 8 10:03:27 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##