vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.12.3525 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-12-23 10:09:38 +00:00
parent 51e691009b
commit b266f4d125
47 changed files with 108216 additions and 108219 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3524 #### Version: V4.2022.12.3525
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3524 #### Version: V4.2022.12.3525
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.12.3524 #### Version: V4.2022.12.3525
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13388
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.12.3524 #### Version: V4.2022.12.3525
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

13392
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13392
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

13392
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

13392
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

13392
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

13392
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

13392
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3524 3525

13392
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

13392
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.86.156.32 0; 104.45.204.82 0;

13392
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

13392
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

13392
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

13392
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.86.156.32 0; 104.45.204.82 0;

13392
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.86.156.32 0; 104.45.204.82 0;

13392
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.160 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BatchFTP Ankit
Flunky Chlooe
InternetSeer InfoNaviRobot
Masscan Joomla
PictureFinder Litemage_walker
ScoutJet Majestic SEO
Semrush MauiBot
Xaldon WebSpider Offline Explorer
gopher Voltron
sexsearcher polaris version

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
ADmantX 01h4x.com
AIBOT 404enemy
ASPSeek
Aboundexbot Aboundexbot
AiHitBot AfD-Verbotsverfahren
Alexibot AhrefsBot
AllSubmitter Aipbot
Alligator AlphaBot
Anarchy Anarchy
Aspiegel Anarchy99
Asterias Apexoo
Atomseobot Atomseobot
BBBike BBBike
BDCbot BackWeb
BLEXBot Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
BatchFTP BatchFTP
BlackWidow BetaBot
Blackboard Boardreader
Blow
BotALot
Brandprotect Brandprotect
Brandwatch Buck
BuiltWith
Bullseye Bullseye
BunnySlippers BuzzSumo
CODE87 CCBot
CSHttp
CazoodleBot CazoodleBot
CensysInspect Cegbfeieh
CheTeam CheTeam
ChinaClaw ChinaClaw
Chlooe Cloud mapping
Citoid
Cocolyzebot Cocolyzebot
Copier
CopyRightCheck
Copyscape
Cosmos Cosmos
Crawling at Home Project Crescent
Curious CrunchBot
Custo
CyotekWebCopy
DBLBot
DSearch
DTS Agent
DatabaseDriverMysqli DatabaseDriverMysqli
Discoverybot Deusu
Devil
Disco
DnBCrawler-Analytics DnBCrawler-Analytics
DomainAppender
DomainCrawler DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Domains Project
Drip
ECCP/1.0 ECCP/1.0
EasyDL EMail Siphon
Ecxi Ebingbong
EirGrabber EirGrabber
Evil
Exabot
Express WebPictures Express WebPictures
ExtLinksBot
Extractor Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder FDM
EyeNetIE FHscan
Firefox/7.0 FemtosearchBot
Flunky Fimap
Freeuploader
Fuzz
FyberSpider FyberSpider
GT::WWW
GalaxyBot
Genieo Genieo
GermCrawler GermCrawler
Go!Zilla Go!Zilla
GoZilla GoZilla
GrabNet Grafula
Grabber HTTP::Lite
HEADMasterSEO
HTTrack HTTrack
Heritrix Haansoft
Havij
Hloader Hloader
HonoluluBot
Humanlinks Humanlinks
IDBot IDBTE4M
Id-search IRLbot
Iblog
IlseBot
Image Fetch
Image Sucker Image Sucker
Indy Library InfoTekies
Internet Ninja Intelliseek
Iria InterGET
JamesBOT InternetSeer
Iskanie
JOC Web Spider
Jbrofuzz
JennyBot JennyBot
Jetty Jetty
Jorgee
Jyxobot Jyxobot
Keyword Density Keyword Density
Kinza Kinza
Kozmosbot
LNSpiderguy
LWP::Simple
Larbin
Leap Leap
LeechFTP LeechFTP
Lftp Lftp
LibWeb
Libwhisker Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkScan
Linkbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinkpadBot
Lmspider LinksManager
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey
Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MJ12bot
MIDown tool
MSFrontPage
MSIECrawler MSIECrawler
MTRobot
Mag-Net Mag-Net
Majestic12 Mail.RU_Bot
MauiBot MarkWatch
Meanpathbot Mb2345Browser
Mister PiX Mediatoolkitbot
Moblie Safari MicroMessenger
Mojeek Microsoft Data Access
Microsoft URL Control
Morfeus Fucking Scanner Morfeus Fucking Scanner
Msrabot Mozlila
NICErsPRO Musobot
NPbot
Nameprotect Nameprotect
Navroad
NearSite
Needle Needle
NetAnts NetZIP
Netcraft
Nettrack
Netvibes Netvibes
NextGenSearchBot
Niki-bot
Nuclei Nuclei
Nutch Offline Explorer
Octopus Offline Navigator
OpenLinkProfiler OnCrawl
Openvas OpenVAS
OrangeBot
OrangeSpider OrangeSpider
OutclicksBot Page Analyzer
OutfoxBot
PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer
PageScorer PageScorer
Pandalytics PageThing.com
Petalbot Petalbot
Pimonster Picsearch
Pixray Pixray
PleaseCrawl
ProPowerBot
Psbot Psbot
Pu_iN
Pump Pump
PxBroker QueryN Metasearch
PyCurl Quick-Crawler
RankActive RankFlex
RankActiveLinkBot
RankingBot RankingBot
RankurBot RankurBot
ReGet Re-re
RealDownload
RebelMouse RebelMouse
RedesScrapy RedesScrapy
Ripper RocketCrawler
SBIder ScreenerBot
SEOkicks
SMTBot
Scanbot
Scrapy
Screaming
ScrepyBot ScrepyBot
SearchmetricsBot
SemanticJuice
Semrush Semrush
SeoSiteCheckup SemrushBot
Seomoz SentiBot
SiteSnagger
SiteSucker
Sitebeam Sitebeam
Siteimprove
Sitevigil
SlySearch SlySearch
Sottopop SmartDownload
Spammen Snake
Snoopy
Spbot
Spinn3r Spinn3r
SputnikBot
Sqlmap Sqlmap
Sqworm Sucuri
Stripper
Sucker
SuperHTTP SuperHTTP
SurveyBot SurveyBot
Swiftbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
T8Abot
Teleport
Telesoft Telesoft
Telesphoreo Telesphorep
The Intraformant The Intraformant
TheNomad Thumbor
Titan
Toata
Tracemyfile
Trendictionbot
Turingos
TurnitinBot
URLy Warning
URLy.Warning URLy.Warning
UnisterBot
Upflow
V-BOT
VB Project VB Project
Vagabondo VCI
VidibleScraper Vacuum
Virusdie VelenPublicWebCrawler
VoidEYE WPScan
Voltron WWW-Mechanize
WASALive-Bot WWW::Mechanize
Web Auto WWWOFFLE
WeSEE
Web Collage
Web Enhancer
Web Pix Web Pix
WebCopier Web Sauger
Web Sucker
WebFetch
WebFuck
WebGo IS
WebImageCollector
WebReaper
WebStripper WebStripper
WebSucker
WebWhacker
Webalta
WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack Whack
Whacker Whacker
Widow Wotbox
Wonderbot
Xaldon WebSpider Xaldon WebSpider
Zermelo Xenu
Zeus
Zitebot Zitebot
ZmEu ZmEu
ZumBot ZoomBot
ZoominfoBot
adscanner
archive.org_bot
autoemailspider
backlink-check backlink-check
check1.exe cah.io.community
clark-crawler
coccocbot coccocbot
cognitiveseo cognitiveseo
com.plumanalytics
crawler.feedback crawler.feedback
dataforseo.com
demandbase-bot demandbase-bot
eCatch eCatch
evc-batch
gopher
instabid instabid
ips-agent internetVista monitor
linkdexbot
lwp-request lwp-request
magpie-crawler meanpathbot
mediawords mediawords
pcBrowser muhstik-scan
page scorer
ripz ripz
s1z.ru satoristudio.net
scalaj-http scalaj-http
seocompany.store scan.lol
seobility
seoscanners seoscanners
serpstatbot serpstatbot
sexsearcher sexsearcher
ubermetrics-technologies.com siteripz
sogouspider
sp_auditbot
spyfu
trendiction.com
webpros.com webpros.com
webprosbot
x22Mozilla
xpymep1.exe
zauba.io
zgrab

300
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker AIBOT
80legs ALittle Client
ASPSeek ASPSeek
Abonti Abonti
Aboundex Aboundex
AfD-Verbotsverfahren AhrefsBot
Aipbot
AllSubmitter
Alligator Alligator
AlphaBot AlphaBot
Anarchy Anarchie
Anarchy99
Ankit Ankit
Apexoo
Aspiegel Aspiegel
Asterias Atomseobot
BBBike
BDFetch BDFetch
BLEXBot BackDoorBot
BackStreet
BackWeb BackWeb
Barkrowler Backlink-Ceck
Battleztar Bazinga Badass
Blackboard BetaBot
Blow Bitacle
Black Hole
BlowFish BlowFish
Brandprotect
Brandwatch Brandwatch
BuiltWith Buddy
CODE87 BuiltBotTough
CSHttp BunnySlippers
CensysInspect BuzzSumo
CheTeam CazoodleBot
Cegbfeieh
CherryPicker CherryPicker
ChinaClaw ChinaClaw
Chlooe Citoid
Claritybot Craftbot
Cogentbot
CopyRightCheck
CrazyWebCrawler CrazyWebCrawler
Crescent
CrunchBot
Custo
DIIbot DIIbot
Demon DataCha0s
Deusu Digincore
DigitalPebble
Dirbuster
Discobot Discobot
DnBCrawler-Analytics DnBCrawler-Analytics
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Download Wonder Download Wonder
Drip ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf Ebingbong
Ecxi Ecxi
EirGrabber
EroCrawler EroCrawler
Evil Express WebPictures
Extractor FHscan
EyeNetIE Foobot
Ezooms
FemtosearchBot
Firefox/7.0
Freeuploader
Fyrebot Fyrebot
GT::WWW GT::WWW
GalaxyBot
GermCrawler GermCrawler
GetRight Go!Zilla
Getintent Go-Ahead-Got-It
GrabNet GrabNet
GrapeFX GrapeFX
GrapeshotCrawler GrapeshotCrawler
HTTP::Lite GridBot
HTTrack HEADMasterSEO
Harvest HTMLparser
HaosouSpider
Hloader
HonoluluBot HonoluluBot
Humanlinks Id-search
HybridBot IlseBot
IRLbot
Image Fetch
Image Sucker Image Sucker
IndeedBot
InfoNaviRobot InfoNaviRobot
Intelliseek
InterGET InterGET
Internet Ninja Iskanie
JamesBOT IstellaBot
Jbrofuzz
JennyBot JennyBot
Jetty Keyword Density
JustView LWP::Simple
Jyxobot Leap
Kinza LeechFTP
LNSpiderguy LeechGet
LexiBot
LibWeb
Libwhisker Libwhisker
LinkpadBot LinkScan
LinqiaMetadataDownloaderBot LinkWalker
LinkextractorPro
LinksManager
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lmspider
Ltx71 MIDown tool
MIIxpc MIIxpc
MSFrontPage MSFrontPage
MSIECrawler MSIECrawler
MTRobot MTRobot
Mag-Net Mail.RU_Bot
Majestic12 Majestic SEO
Majestic-SEO
MarkWatch MarkWatch
Mass Downloader Mass Downloader
MeanPath Bot MicroMessenger
MegaIndex.ru
Microsoft Data Access Microsoft Data Access
Minefield Microsoft URL Control
Moblie Safari Moblie Safari
Mojeek
Mojolicious Mojolicious
Mr.4x3 Mr.4x3
Msrabot Msrabot
NICErsPRO Musobot
NPbot
Name Intelligence Name Intelligence
Navroad
Needle
NetAnts
NetLyzer
NetMechanic
NetSpider
NetZIP NetZIP
Nettrack Netcraft
Nibbler Netvibes
Nikto Niki-bot
Nimbostratus Nimbostratus
Nmap Nmap
Not Not
Nuclei Nuclei
Octopus Nutch
OrangeBot OnCrawl
OutfoxBot OpenLinkProfiler
PageAnalyzer Openfind
PageGrabber OutclicksBot
PageScorer PHPCrawl
Page Analyzer
PageThing.com
Panscient Panscient
Papa Foto Papa Foto
PeoplePal Pavuk
Petalbot Picscout
Picsearch ProWebWalker
Pixray RSSingBot
ProPowerBot RankActive
Probethenet
Psbot
Pump
PxBroker
RankActiveLinkBot RankActiveLinkBot
RankFlex RankingBot2
Rankivabot Rankivabot
Re-re
ReGet ReGet
Reaper
RebelMouse
RedesScrapy RedesScrapy
Ripper SBIder
Rogerbot
SEOprofiler
SEOstats
SISTRIX SISTRIX
SMTBot
ScanAlert
ScoutJet
Scrapy
Screaming
ScreenerBot
SearchmetricsBot
Seekport Seekport
SemanticJuice SemrushBot
SentiBot Seomoz
Shodan
Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger Siteimprove
SiteSucker SmartDownload
Sitevigil
SlySearch
Snake Snake
SocialRankIOBot SpaceBison
Spammen Spammen
Spbot
Spinn3r Spinn3r
Sqworm Steeler
Sucker Sucuri
SuperBot SuperBot
Surfbot
SurveyBot SurveyBot
Suzuran Suzuran
Swiftbot Swiftbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
Teleport T8Abot
The Intraformant
TheNomad TheNomad
TightTwatBot
Titan Titan
Tracemyfile Toweyabot
Trendiction
Turnitin Turnitin
TwengaBot TurnitinBot
Twice
URLy.Warning
UnisterBot
V-BOT V-BOT
Vacuum VCI
Vagabondo
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
Voil Voltron
WASALive-Bot WASALive-Bot
WBSearchBot WEBDAV
WPScan WWWOFFLE
WWW-Collector-E Wallpapers/3.0
Web Enhancer Web Enhancer
Web Fuck Web Fetch
Web Pix Web Pix
WebEnhancer Web Sucker
WebFuck WebAuto
WebCopier
WebFetch
WebGo IS
WebImageCollector WebImageCollector
WebLeacher
WebPix WebPix
WebStripper WebReaper
WebSauger
WebWhacker WebWhacker
WebZIP
Webalta Webalta
Webshag WebmasterWorldForumBot
Webster Webster
Whack Whatweb
Whacker
Widow Widow
Wonderbot Wonderbot
Woobot Xaldon_WebSpider
Zade Zeus
Zitebot ZoominfoBot
ZoomBot
ZyBorg
adscanner
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
backlink-check autoemailspider
cah.io.community clark-crawler
coccocbot cognitiveseo
demandbase-bot com.plumanalytics
isitwp.com crawler.feedback
dataforseo.com
evc-batch
heritrix
internetVista monitor
iubenda-radar
lwp-request lwp-request
lwp-trivial lwp-trivial
meanpathbot meanpathbot
probe-image-size oBot
ripz pcBrowser
s1z.ru polaris version
satoristudio.net scan.lol
seobility
seoscanners seoscanners
seostar
serpstatbot serpstatbot
sexsearcher siteripz
sitechecker.pro
sp_auditbot
sysscan sysscan
webgains-bot trendiction.com
webmeup-crawler webmeup-crawler
webpros.com
webprosbot
x09Mozilla x09Mozilla
x22Mozilla
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
brewdom.ru arquapetrarca.info
fz139.ttk.ru freshmac.space
phpdevops.com jamiembrown.com
sa-rewards.co.za lida-ru.com
sexy.babes.frontend-stack.top maridan.com.ua
social-search.me mcadamssupplyco.com
top10registrycleaners.com mnogabukaff.net
topsiteminecraft.com nero-us.com
virtuagirl.com pornofeuer.com
webextract.profound.net themestotal.com

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
01apple.com 08800.top
03e.info 0daymusic.org
0c47f8422d3f.com 1-99seo.com
0n-line.tv 11pikachu.ru
1-free-share-buttons.com 1adult.com
123any.com 1pamm.ru
1688.com 1qingdao.com
178evakuator178.ru 214jaluwobapef.cf
3w24.com 24-ak.ru
4ur.click 28n2gl3wfyb0.ru
55wmz.ru 2daytrendingnews.com
5elementov.ru 2girls1cup-free.com
6128786.com 3dracergames.com
7makemoneyonline.com 404.mn
8lufu.com 4webmasters.org
8xv8.com 5000-cotydzien.com
a.frcls.fr 51.la
a1security.com.ua 5u.com
above.com 6hopping.com
actices.com a96527gi.beget.tech
add-add.men aa08daf7e13b6345e09e92f771507fa5f4.com
admatic.com.tr adloads.com
adobereader-free.ru admeasures.com
adprotect.net adnotbad.com
adtiger.tk ads-seo.men
advancedcleaner.com adserver-e7.com
advokateg.ru adult3dgames.com
aimayangzhi.com adultfullhd.com
airlandsea.info advancedmassagebysara.com
allfinweb.com adventureparkcostarica.com
alphavisions.net adviceforum.info
am15.net affrh2015.com
albuteroli.com
alert-jdh.xyz
alfapro.ru
allornamenti.com
alphacarolinas.com
alta-realestate.com
amehdaily.com
android-systems.ru
angkortours.vn
animalia-life.club animalia-life.club
ap.senai.br anlimebel.kiev.ua
apessay.com antons-transporte.de
apibrowsesmartne-a.akamaihd.net apelsinnik.website
apiitsthirteende-a.akamaihd.net apiappenableinfo-a.akamaihd.net
apxeo.info apple.com-cleaner.systems
arcteryxstore.online applyneedy.xyz
arius.tech artparquet.ru
aruplighting.com asrv-a.akamaihd.net
asiavirtualsolutions.com atlant-auto.info
asiengirls.net atofilms.com
auto.rusvile.lt autolombard-krasnodar.ru
autoua.top autoua.top
aviapanda.ru
aviav.eu aviav.eu
avirasecureserver.com
avtochehli.by avtochehli.by
back.dog avtointeres.ru
backlinkwatch.com azte.ch
babespcs.com
babs.com.ua
balois.worldbreak.com
balook.com
bang-hotties.com bang-hotties.com
banki76.ru bankmib.ru
battleforupdating.site becuo.com
bbsoldes.fr beremenyashka.com
bdsmgalls.net bestcurs.org
beauty-bracelet.com bestserials.com
beetpics.pw billiard-classic.com.ua
bestbuy.ca bimmerpost.com
bestdraws.com biocn.dx.am
besthoro.ru bioscorp.ru
bestprofits-there.com biteg.xyz
bestwaystogetpaid.us bitnote.co
bibys.com blackcurranthumidifiers.site
bildsuche.ru bloggerads.net
bio-japan.net bolitgorlo.net
bitcoins-live.ru bonkers.name
bizfly.info boo-arts.com
biznesluxe.ru boole.onlinewebshop.net
blogqpot.com braindaily.xyz
blogs.rediff.com brandedleadgeneration.com
bonanza-fortune.men brg8.com
boobsimge.com brus-vsem.ru
brewdom.ru bryansk.zrus.org
buketeg.xyz buypanicdisorderpill.com
bumble.cheapwebsitehoster.com buzzsumo.com
buqyxa.rincian.info buzzzg.men
business-made-fun.com bvlgariring.xyz
buy-meds24.com bycontext.com
buyantiviralwp.com cam-kontakte.org
buyessay3.blogspot.ru camdolls.com
buyviagraa.blogspot.com canoncdriverq3.pen.io
calcularpagerank.com capture-room.com
callejondelpozo.es cardsharp1.ru
callstevens.com cartierlove2u.com
camsex.xxx-cam.webcam
candcstuccoandstone.com
candlewooddentalcentre.com.au
canzoni.ru
cardiosport.com.ua
cardsdumps.com
cartierreplica.top cartierreplica.top
cascadelink.org cashkitten-a.akamaihd.net
caveavins.fr casinopinup-wins.com
cellfun.mobi casite-513049.cloudaccess.net
cheapcigarettesc.info ccbill.com
checkpagerank.net cdnanalytics.xyz
chelnytruck.ru centre-indigo.org.ua
chocolateslim-original.com chastnoeporno.com
chorus.terakeet.com check-this-out-now.online
cjmc.info chinajerseyswholesalecoupons.com
clean-virus-mac.com chocolateslim-u-srbiji.com
cleanpcnow.com christianlouboutinsaleonline.us
cigarpass.com
clash-clans.ru
clickzzs.nl clickzzs.nl
clubfashionista.com clippingphotoindia.com
compliance-andrew.top clodo.ru
compliance-olga.top clothesforcash.com
confib.ifmo.ru coin-hive.com
connectingsingles.com compliance-john.top
coolingoods.com concealthyself.com
coolwallpapers4k.info copblock.org
cosmetic.donna7753191.ru credit-respect.ru
covetnica.com
craftinsta.ru
crd.clan.su
crest-poloski.ru
crynet.cc crynet.cc
cubook.supernew.org culad.com
cxpromote.com customerguru.in
czeshop.info cv.wallhade.co
cymbaltaandtizanidine.blogspot.com
cyse.tk
daooda.com daooda.com
daretodonate.co
data-ox.com data-ox.com
db.speedup.gdn daymusam.com
dealighted.com de.zapmeta.com
dear-diary.ru demon-tweeks.com
defenderxtactical.com den-noch24.ru
depositfiles-porn.ga
descargar-musica-gratis.net descargar-musica-gratis.net
detki-opt.ru design-lands.ru
dgfitness.co detalizaciya-tut.biz
digitalassetmanagement.site detoxmed24.ru
dinosaurus.site detskie-konstruktory.ru
dogoimage.com digest-project.ru
dogsrun.net digilander.libero.it
domcran.net dimaka.info
dpihatinh.gov.vn divci-hry.info
droidlook.net dmmspy.com
dynainbox.com documentbase.net
documentsite.net
domain2008.com
domaincdn.xyz
domainroam.win
domznaniy.ru
donna7753191.ru
drugs-no-rx.info
drugspowerstore.com
e2click.com e2click.com
ebooknovel.club eandsgallery.com
ee77ee.com earn-from-articles.com
ege-essay.ru easyshoppermac.com
eleimgo.pw ed-shop01.ru
elektir.ru edge.sharethis.com
efnor-ac.com
ekobata.ru
elektrischeziga.livejournal.com
elektrischezigarette2.onsugar.com
elektronischezigarette2.mex.tl elektronischezigarette2.mex.tl
elidelcream.weebly.com elektrozigarette2.wordpressy.pl
elmifarhangi.com eng-lyrics.com
eluxer.net engines-usa.com
emailaccountlogin.co
embedle.com
empis.magix.net
energydiet24.ru
errorfixing.space
etenininrade.ga etenininrade.ga
euromasterclass.ru etm-consult.de
extractorandburner.com everflora.ru
ezigarettekaufen1.hpage.com executehosting.com
falopicm.pw extener.org
ez8motelseaworldsandiego.com
ezigarettekaufen.myblog.de
factorynightclub.com
familyphysician.ru familyphysician.ru
faracontrol.ir fapgon.com
fearcrow.com farmprofi.net
femdom.twiclub.in favoritemoney.ru
ferretsoft.com fealq.com
filmi-v.online feargames.ru
findthe.pet feorina.ru
finejewelryshop.ru fillmewithhappiness.com
firstsiteguide.com findclan.org
fishtauto.ru
fitness-video.net
flex4launch.ru
floating-share-buttons.com floating-share-buttons.com
forexgb.ru fm-upgrade.ru
forodvd.com foto-basa.com
forpostlock.ru
foto-weinberger.at foto-weinberger.at
free-share-buttons.top fotzen-ficken.com
free-social-buttons.xyz franklinfire.co
frequiry.com free-fb-traffic.com
freza-sverlo.ru fun-mobi.pl
friendflnder.com fxgallery.com
funnel.co.za gavuer.ru
gallerily.com gaypornmovie.net
gallery.rennlist.com gayxperience.com
gamevalue7.weebly.com getoutofdebtfree.org
geilehausfrauen.net gk-atlant.info
genetworx.com gktt.ru
get-seo-domain.com goodwriterssales.com
getmyads24.com
girls-ufa.ru
glall.ru
glicol.kz
glopages.ru
goatse.ru
googlemare.com googlemare.com
googleseo.com.tr gopro-online.info
gpms.org.my gq-catalog.gq
greatzip.com growmyfunds.ca
gsou.cf gsasearchengineranker.pw
guiadeserraazul.com gsasearchenginerankerhelp.com
guigyverpo.cf gsbs.com.ua
guildebzh.info happy.new.yeartwit.com
guod.me hasfun.com
haikuware.com help.tpu.ru
hasshe.com hermesbirkinhandbagoutlets.com
hdapp1008-a.akamaihd.net herokuapp.com
hdwalls.xyz hikesearch.net
heartofbeijing.blogspot.com hmmm.cz
helpmymacfaster.trade hotelcrocenzi.sm
hidemyass.com hptwaakw.blog.fc2.com
himgaws.pw ilovevitaly.com
homedecoguide.info imicrovision.com
homeinns.com imitex-plus.ru
homemade.gq impisr.ru
hotkeys.com incomekey.net
hulfingtonpost.com indiakino.net
hunterboots.online instabid.tech
ido3.com intelhdgraphicsgtdrive6w.metroblog.com
igrovyeavtomaty777.ru interferencer.ru
images-graphics-pics.com internet-apteka.ru
imperiafilm.ru investzalog.ru
infoupdate.org ipornox.xxx
insomniagamingfestival.com ipostroika.ru
investingclub.ru iradiology.ru
investmac.com isistaylorporn.info
io9.com jagg.info
iqoption.com japan-bearings.ru
iqs.biz.ua javxxx18.com
iron-age.info jeremyeaton.co
ispaniya-costa-blanca.ru joingames.org
itag.pw jongose.ninja
itrevolution.cf karusel-market.ru
ivanstroi.ru keywordsuggests.com
izhevsk.zrus.org kino-fun.ru
izoll.ru kinobest.pl
j-times.ru klitimg.pw
janavibekken.no krafte.ru
jerseyschinabizwholesale.us kreativperlen.ch
jetsli.de kursy-ege.ru
jjbabskoe.ru
jofucipiku.tk
jovencitas.gratis
jus0wil.pp.ua
jwss.cc
kalandranis.gr
kam-dom.ru
kamagragelusa.net
karaganda.xkaz.org
keenoutlet.online
keywordspay.com
konica.kz
koopilka.com
kretpicf.pw
ks1234.com
la-fa.ru la-fa.ru
lacapilla.info laptoper.net
law-check-eight.xyz leadwayau.com
lexaprogeneric.link lennatin.info
live-cam6.info letslowbefast.site
livefixer.com likesdesign.com
livejasmin.com limtu.ifmo.ru
liver-chintai.org lion.playtap.us
lmrauction.com livingroomdecor.info
loginduepunti.it localflirtbuddies.com
louboutinreplica.pw locimge.pw
lovelycraftyhome.com lonfon.xyz
lsp-awak-perikanan.com lostaruban.com
lubetube.com luchshie-topcasino.ru
m1media.net m5home.ru
mabdoola.blogspot.com macfix.life
magda-gadalka.ru machicon-akihabara.info
magicdiet.gq
mainhp.net mainhp.net
mainlinehobby.net make-money-online.com
mall.uk makedo.ru
mansiondelrio.co mb140.ru
mantramusic.ru
manve.info
marmotstore.online
master-muznachas.ru
masturbate.co.uk
mbiologi.ru
mebelcomplekt.ru
mediafresh.online mediafresh.online
medicovi.com medicines-choice.com
medizinreisen.de medictube.ru
medosmotr-ufa.ru medtherapy.ru
megakino.net mega-bony-2017.pl
menhealed.net mega-bony2017.pl
metascephe.com mejoresfotos.eu
michellblog.online mexicosleevegastrectomy.com
mil-stak.com minecraft-neo.ru
minecraft-rus.org minneapoliscopiers.com
mirtorrent.net minyetki.ru
mjchamonix.org missvietnam.org
mmog-play.ru mojpreskumanie.com
mobioffertrck.com mole.pluto.ro
modnie-futbolki.net monarchfind-a.akamaihd.net
monclerjacketsoutlet.win moneymaster.ru
money-every-day.com moneyzzz.ru
montredemarque.nl monthlywinners.com
mossmesi.com mug-na-chas-moscow.ru
mrbojikobi4.biz multgo.ru
mriyadh.com mundoaberrante.com
mydownloadengine.com music.utrolive.ru
nagdak.ru muycerdas.xxx
najaden.no muztops.ru
narkologiya-voronezh.ru mvpicton.co.uk
netanalytics.xyz myonigroup.com
newstudio.tv myxdate.info
nicefloor.co.uk narkologiya-belgorod.ru
nonameread45.live nerudlogistik.ru
o-o-6-o-o.ru netfacet.net
oecnhs.info nhl09.ru
omoikiri-japan.ru nina.az
onlythegames.com ningessaybe.me
ooomeru.ru nottyu.xyz
oops-cinema.ru o.light.d0t.ru
ops.picscout.com obiavo.kz
optom-deshevo.ru obiavo.su
orgasmus-virtual.com offergroup.info
online-templatestore.com
onlinefilmz.net
onlinemegax.com
onlyforemont.ru
openmultipleurl.com
optibuymac.com
ordernorxx.com
orsonet.ru orsonet.ru
osagonline.ru osb.se11.ru
oxford-book.com.ua outshop.ru
partner-cdn.men ozoz.it
parvezmia.xyz painting-planet.com
perfection-pleasure.ru parlament.biz
petsblogroll.com partner-host.men
piccshare.com pawli.eu
pinsdaddy.com pcboa.se
pirateday.ru perfectpracticeweb.com
pisanieprac.info perm.xrus.org
pixelrz.com pewit.pw
pizda.lol phuketscreen.com
plugingeorgia.com picanalyzer.data-ox.com
poffet.net pixgood.com
pooleroadmedicalcentre.co.uk playmsn.com
popunder.ru popcash.net
popup-hgd.xyz popup-hgd.xyz
porn555.com poquoson.org
pornoinn.com porndairy.in
pornosemki.info pornhub-forum.ga
pornosok.ru pornokajf.com
pozdrawleniya.com pornolook.net
pregnant.guru princeadvantagesales.com
princevc.com privacylocationforloc.com
privatov-zapisi.ru privatefx.all4invest.info
promgirldresses.xyz prlog.ru
promodj.com professionaldieselcare.com
progress-upakovka.ru
proposal-engine.com
pshare.biz pshare.biz
pufip.com
pukaporn.com
pushdata.sendpulse.com pushdata.sendpulse.com
ranksignals.com pussyfleet.com
realmonte.net quick-seeker.com
realnye-otzyvy.info quickbuck.com
relax.ru r-control.ru
rennlist.com realtytimes.com
resellerclub.com reelheroes.net
reversing.cc registratciya-v-moskve.ru
risparmiocasa.bz.it reining.lovasszovetseg.hu
robotixix.com reklama1.ru
rp9.ru renecaovilla.online
renecaovillasale.online
replica-watch.ru
restaurantlescampi.com
reyel1985.webnode.fr
rfserial.net
rialp.getenjoyment.net
riders.ro
rixpix.ru
ru-mediaget.ru
rucrypt.com rucrypt.com
ruscopybook.com rybalka-opt.ru
rusenvironmental.net s.lollypopgaming.com
sa-live.com s8-nowy-wygraj.comli.com
sabaapress.com sammyweaver.com
sack.net scanhub.ru
sakhboard.ru scanner-andrew.top
sbornik-zakonov.ru scanner-barak.top
scenicmissouri.us scanner-don.top
se.bnt-team.com scanner-viktor.top
seasaltwithfood.com scrapinghub.com
securityallianceservices.com scrapy.org
sel-hoz.com seansonline24.pl
seo-prof1.xyz seinterface.com
seocdvig.ru sell-fb-group-here.com
sex.hotblog.top seocheki.net
sergiorossistore.online
servethis.com
service-core.ru
sexflirtbook.com
sexs-foto.top
sexuria.net sexuria.net
sfd-chess.ru shohanb.com
shama-rc.net si-unique.com
shemalegalls.blogporn.in silktide.com
shoppingmiracles.co.uk
sibtest.ru
simply.net simply.net
site-speed-checker.site simul.co
sivs.ru sitechecker.pro
smichovbike.cz sladkoevideo.com
snow.nvr163.com slavic-magic.ru
snworks.com slowmac.tech
smailik.org
snip.tw
sobecjvuwa.com.ru sobecjvuwa.com.ru
soc-econom-problems.ru socialbookmarksubmission.org
sofit-dmd.ru socialsignals24.com
softomix.net solmarket.by
songoo.wz.cz sortthemesitesby.com
souvenir.cc
spacash.com
space4updating.win space4updating.win
spitfiremusic.com spidtest.org
sr-rekneskap.no sprttrack.com
stariy-baku.com spymac.net
stjamesschool.info stanthonyscatholicchurch.org
strag-invest.ru stop-gepatit.te.ua
strigkaomsk.ru streha-metalko.si
stroyhelp-dv.ru support.nopeas.sk
superfish.com svarbit.com
swinginwithme.ru swimpool.ca
sygraem.com swingerseiten.com
taxi-v-eisk.ru szucs.ru
td-l-market.ru tacbelarus.ru
templates.franklinfire.co textads.men
terraclicks.com the-torrent-tracker.blogspot.com
tgtclick.com theendivechronicles.com
theallgirlarcade.com thegamerznetwork.com
thebluenoodle.com timer4web.com
thelottosecrets.com titelhelden.eu
timetorelax.biz tizanidineandgabapentin.blogspot.com
titan-cloud.life tizanidinehcl4mgisitanarcotic.blogspot.com
toloka.hurtom.com tizanidineusedfor.blogspot.com
tomatis.gospartner.com tizanidinevsvalium.blogspot.com
toolsky.com tizanidinewithdrawal.blogspot.com
toon-families.com tobeyouday.win
torrent-newgames.com torture.ml
tourismvictoria.com trackmedia101.com
toursmaps.com tracksurf.daooda.com
trafficjunky.net
trafficreceiver.club trafficreceiver.club
traffmonster.info trion.od.ua
trudogolik.net ttsq.fr
turizmus.us turkeyreport.tk
twelvevisionspartyofcolorado.com
twinderbella.com
twittruth.com twittruth.com
tytoona.com
ublaze.ru ublaze.ru
uchetunet.su ufa.xrus.org
unacittaconte.org uginekologa.com
unmaroll.ya.ru ukrup.com
uptime-as.net unitexindia.com
userequip.com unpredictable.ga
uzporno.mobi uptime-gamma.net
v720hd.ru utiblog.fr
valsalud.com versaut.xxx-cam.webcam
vandrie-ict.nl videojam.tv
varikoz24.com videtubs.pl
vbikse.com vids18.site
vektorpress.ru vigrx-original.ru
velobikestock.com visa-china.ru
venerologiya.com visa-pasport.ru
vereo.eu
videokrik.net
videos-for-your-business.com
videotuber.ru
vipps.com.my
visionwell.com.cn
vita.com.hr
viveresaniesnelli.it
vizag.kharkov.ua
vksex.ru vksex.ru
vltai.com volume-pills.biz
vmnmvzsmn.over-blog.com voronezh.xrus.org
vriel.batcave.net vzlom-na-zakaz.com
warning.or.kr vzube.com
wareseeker.com
watchmyfb.pl watchmyfb.pl
watracker.net waycash.net
web.cvut.cz webalan.ru
webads.co.nz website-speed-checker.site
webkeyit.com website-speed-up.top
websiteaccountant.de
websitevaluebot.com
webtherapy.ru webtherapy.ru
westen-v.life welcomeauto.ru
winx-play.ru where-toget.com
womensplay.net whosonmyserver.com
wowcasinoonline.ooo world-mmo.com
woweb.com.ua worldtraveler.world
writingservices17.blogspot.ru worst-sites.online
wstroika.ru x-stars.ru
www.bookmaker-bets.com xgames-04.com
www.solartek.ru xingzi-vision.com
wygraj-skiny.win xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
x-diesel.biz xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
x-musics.com xn--80aafb2a.xn--p1ai
xclicks.net xn--b1addnj3cah.xn--p1ai
xev.ru xn--b1ag5cfn.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xportvusbdriver8i.snack.ws
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai xsion.net
xn--80ahvj9e.xn--p1ai xtubeporno.net
xsfetish.org xvideospornoru.com
yogamatsexpert.com xxxdatinglocal.us
yopoint.in yaoguangdj.com
yougetsignal.com youporn-ru.com
youtuhe.com youresponsive.com
yubikk.info yuarra.pluto.ro
yun56.co yugo-star.ru
yuweng.info z-master.ru
zaimhelp.ru za-fun-offer.com
zajm-pod-zalog-nedvizhimosti.ru zeroredirect10.com
zakazvzloma.com zeroredirect8.com
zambini.ru zhuravlev.info
zero1.it zigzog.ru
zeroredirect9.com znaniyapolza.ru
zigarettenonlinekaufen1.bloog.pl zoolubimets.ru
znakomstva-moskva77.ru zscalerone.net
zocaparj.kz zvetki.ru
zooggames.com
zoominfo.com
zvuker.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0912701309f8ce.com 1-99seo.com
100searchengines.com 108shot.com
1millionusd.xyz 7wind.ru
1qingdao.com 8si.ru
2015god.org 98oi.ru
21h2o.com adman.se
72-news.com adminshop.com
activepr.ru ads-cool.pro
actulite.com adsland.men
adultnet.in adzerg.com
agroeconom.kz affiliate-fr.com
albamargroup.com afmuseum.com
all4wap.ru afslankpillen2017nl.eu
allblogroll.com all-number.com
alldubai.biz allfinweb.com
anaksma.info amanda-porn.ga
anal-acrobats.com amateurgalls.com
animenime.ru apartment.ru
app5.letmacworkfaster.world apartmentratings.com
aproposde.com apiwebwebgetcom-a.akamaihd.net
asrv-a.akamoihd.net approvedlocal.co.za
auto.rusvile.lt apps-analytics.net
avek.ru artel116.ru
babyfactory.fr artparquet.ru
balois.worldbreak.com asiavirtualsolutions.com
bannerads.de atelielembrancaqueencanta.com.br
bashtime.ru atley.eu.pn
bestserials.com atmagroup.ru
besttorrentknifta.weebly.com automobile-spec.com
big-boards.info avirasecureserver.com
bimatoprost-careprost.com bestsexyblog.com
bingo8888.com betshuckclean.com
bitcoinremote.com bigpenisguide.com
bmw-ark.ru bio-optomarket.ru
bonzybuddy.com bio.trade-jp.net
bracketsmackdown.com bizlist.com.de
break-the-chains.com biznesluxe.ru
btvn.ru bjetjt.com
bupropion-sr-150-mg.us blackcurranthumidifiers.site
buqyxa.rincian.info blackwitchcraft.ru
byme.se blagovest-med.ru
calcularpagerank.com blog.koreadaily.com
calstaterealty.us bluesman.nu
candlewooddentalcentre.com.au bonux.nextview.ru
cbox.ws bonzbuddy.com
censys.io braintobe.win
ceotrk.com brillianty.info
children-learningreading.info buchananshardware.com
clickhype.com bus-offer.com
clickzzs.nl canoncdriverq3.pen.io
compliance-willy.top casite-513049.cloudaccess.net
corta.co cbb1smartlist12.click
creditmoney.com.ua cejewelry.xyz
dame-ns.kz celejihad.info
date-withme.com cfacarrosserie74.com
dichvuvesinhhanoi.com cheapestjerseys-wholesale.com
djonwatch.ru cialgenisrx.com
dlya-android.org cineacademy.ru
dnsrsearch.com cleanallvirus.com
docs4all.com clickaider.com
doyouknowtheword-flummox.ml club-samodelkin.ru
dyt.net compliance-julia.top
e-c.al compliance-margo.top
earl-brown.info cookingmeat.ru
edudocs.net coolwallpapers4k.info
ekn-art.se copblock.org
elektrozigarette2.webs.com cranly.net
eloxal.ru custodieva.ru
enthuse.computernetworksonline.com dataloading.net
epngo.bz ddlmega.net
eralph.tk dipstar.org
erotag.com downloaddy.net
ertelecom.ru droidlook.net
essay-edu.biz drugspowerstore.com
estibot.com dvr.biz.ua
etm-consult.de earthmagic.info
event-tracking.com ecxtracking.com
ewebarticle.info ekosmetyki.net.pl
fealq.com eksprognoz.ru
elektrischezi.canalblog.com
elektronischezigarette2.mex.tl
elmacho.xyz
englishgamer.com
europages.com.ru
executehosting.com
expertblog.info
extremepornos.net
ezrvrentals.com
f-online.de
fashion-mk.net
fast-wordpress-start.com
feargames.ru feargames.ru
felizporno.com filesvine.com
firstsiteguide.com flex4launch.ru
fitfloponline.store forodvd.com
fortunejack.com fortunejack.com
forum20.smailik.org forums.toucharcade.com
franecki.net foto-weinberger.at
fsakhalin.ru frcls.fr
fxgallery.com free-laptop-reward.com
game-mmorpg.net froggytube.com
gateway.zscalerone.net geoads.com
glavtral.ru go2mike.ru
googlsucks.com gsou.cf
grizzlysgrill.com hasfun.com
hd-film.pl headpress.ru
hidemyass.com hidefiles.org
highland-homes.com hit-kino.ru
hotblog.top hjaoopoa.top
hotxnights.info hledejvshopech.cz
hundejo.com hostsshop.ru
hyipmanager.in hotchatdate.com
ibmdatamanagement.co hoterika.com
idc.com.ua houseofrose.com
igtools.club ideibiznesa2015.ru
ilovevitaly.ru ilovevitaly.ru
ilte.info intellektmedia.at
indianmedicaltourismshop.com iqbazar.ru
intermesh.net istizanidineanarcoticdrug.blogspot.com
itbc.kiev.ua johnnyhaley.top
izoll.ru karachev-city.ru
je7.us kenaba.su
jjbabskoe.ru keyhantercume.com
kalandranis.gr keywordsuggests.com
kalb.ru koral.se
keywords-monitoring-your-success.com kountrylife.com
khafre.us krasula.pp.ua
kiinomaniak.pl languagecode.com
kino-filmi.com layola.biz.tc
kinocccp.net leftofcentrist.com
kleine-titten.biz lietaer.com
knogg.net lionking-1994.blogspot.com
kostenloser-sex.com lordzfilmz.me
krasnodar-avtolombards.ru makemoneyonline.com
krasnodar.zrus.org masterseek.com
kursy-ege.ru masthopehomes.com
lesbian.xyz maxxtor.eu
lineavabit.it med-zdorovie.com.ua
lirunet.ru medmajor.ru
lisque.batcave.net megavolt.net.ua
livefixer.com member-quiz.com
m1media.net michaelkorsoutletstore.net
m292.info mil-stak.com
macnewtech.com minecraft-neo.ru
mail.allnews24.in mobioffertrck.com
mainhp.net monclercheap.xyz
mainlinehobby.net monclerjacketsoutlet.pw
maju.bluesalt.co moncleroutletonline.win
make-money-online.com moxo.com
manualterap.roleforum.ru moyakuhnia.ru
marwer.info mustwineblog.com
max-p.men myprintscreen.com
mearns-tractors.co.uk nardulan.com
mediaoffers.click
mesbuta.info
more-letom.ru
morf.snn.gr
mp3ringtone.info
myfreemp3.eu
myyour.eu
narkologiya-belgorod.ru narkologiya-belgorod.ru
natali-forex.com naturalpharm.com.ua
nerudlogistik.ru nextlnk12.com
newstraveller.ru nfljerseys.online
niroo.info nicovideo.jp
nofreezingmac.work nootrino.com
nonews.co nsatc.net
nullrefer.com nufaq.com
oda.as
officedocuments.net
old-rock.com old-rock.com
olgacvetmet.com online-x.ru
online7777.com onlinewritingjobs17.blogspot.ru
origin-my.ru panchro.xyz
ororodnik.goodbb.ru perfection-pleasure.ru
osagonline.ru petitions.whitehouse.gov
paramountmarble.co.uk pewit.pw
paydayloanslocal.com physfunc.ru
pendelprognos.se pihl.se
pepperstyle.ru pistonclasico.com
peugeot-club.org playfortuna-play.ru
pinapchik.com poem-paying.gq
pinwallpaper.top pogosh.com
pipki.r.acdnpro.com pornofeuer.com
pladform.ru pornosemki.info
poisk-zakona.ru pricheski-video.com
poligon.com printingpeach.com
pravoholding.ru proekt-mos.ru
primedice.com progress-upakovka.ru
privat-girl.net prostitutki-kiev.org
producm.ru pulseonclick.com
profitsport.club qwesa.ru
promoheads.com remote-dba.de
pushdata.sendpulse.com reversing.cc
py100.ru riders.ro
quickloanbank.com risparmiocasa.bz.it
raschtextil.com.ua
registratciya-v-moskve.ru
remont-komputerov-notebook.ru
renecaovillasale.online
resultshub-a.akamaihd.net
rockma.se
romhacking.ru romhacking.ru
scanner-jane.top royal-investments.net
seocdvig.ru royalcar-ufa.ru
sergiorossistore.online rusenvironmental.net
s-forum.biz
safe-app.net
salmonfishingsacramentoriver.com
scanner-elena.top
search-goo.com
searchengineranker.email
securesmrt-dt.com
sethrollins.net sethrollins.net
sex.hotblog.top sextracker.de
shitmovs.com sindragosa.comxa.com
shmetall.com.ua site-speed-check.site
silvercash.com site-speed-checker.site
sispe.com.br spravka130.ru
site3.free-share-buttons.com ssconstruction.co
skincrate.net stariy-baku.com
snip.to sterva.cc
snjatie-geroinovoy-lomki.ru strigkaomsk.ru
sobecjvuwa.com.ru superoboi.com.ua
social-vestnik.ru tackletarts.co
spb-plitka.ru terrafootwear.us
steelmaster.lv timdreby.com
stickers-market.ru tizanidineandhydrocodone.blogspot.com
sugarlyflex.pw tizanidinecapsules.blogspot.com
superiends.org tobeyouday.win
supers.com.ua top1-seo-service.com
suture.co tovaroboom.vast.ru
teenporn18.net tracxn.com
textads.men triplepanda.xyz
theendivechronicles.com tubeoffline.com
tizanidinehclsideeffects.blogspot.com turizm.bz
torrentgamer.net tzritel.tk
torrents-tracker.com uggsale.online
traffic2cash.org uptime-gamma.net
trafficjunky.net upupa.net
triumf-realty.ru urlopener.blogspot.com.au
ukkala.xyz urzedowski.eu
ultimatesetnewfreeallsoftupgradesystems.pw viandpet.com
unitygame3d.com videogamesecrets.com
unrealcommander.com vizitki.net
usacasino.com voprosotvet24.ru
via-energy-acquistare.com websiteaccountant.de
viagraneggrx.com weddingdresses.xyz
viddyoze.com ws.ampower.me
video-chat.cn www.bookmaker-bets.com
vipsiterip.org xaijo.com
whosonmyserver.com xkaz.org
worldoffiles.ru xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
writingservices17.blogspot.ru xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
www.kabbalah-red-bracelets.com xn----ctbigni3aj4h.xn--p1ai
x-diesel.info
x-lime.com
xfluro.com
xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn--80aanaardaperhcem4a6i.com xn--80aanaardaperhcem4a6i.com
xn--80ak6aa92e.com yellowstonesafaritours.com
xn--lifehacer-1rb.com your-bearings.com
xxart.ru zajm-pod-zalog-nedvizhimosti.ru
xxxmania.top
xxxsiterips.xyz
youdao.com
yunque.pluto.ro
zakon-ob-obrazovanii.ru zakon-ob-obrazovanii.ru
zarabiaj-dzis.pl zeroredirect9.com
zed21.net zetmaster.ru
zygophyceous.womanstars.site znakomstva-piter78.ru

59
dev-tools/test_units/ratelimittest-master.txt
View file

@ -8,37 +8,6 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html>ml>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html><html> </html><html>
<head><title>503 Service Temporarily Unavailable</title></head> <head><title>503 Service Temporarily Unavailable</title></head>
<body> <body>
@ -494,6 +463,34 @@
<hr><center>nginx/1.18.0 (Ubuntu)</center> <hr><center>nginx/1.18.0 (Ubuntu)</center>
</body> </body>
</html> </html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.160 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3524 ### Version: V4.2022.12.3525
### Updated: Wed Dec 21 22:01:25 UTC 2022 ### Updated: Fri Dec 23 10:03:21 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3524 ### Version: V4.2022.12.3525
### Updated: Wed Dec 21 22:01:25 UTC 2022 ### Updated: Fri Dec 23 10:03:21 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##