vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3449 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-03 22:07:25 +00:00
parent 221ef68126
commit 6bb946914d
47 changed files with 102231 additions and 102221 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3448 #### Version: V4.2022.11.3449
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3448 #### Version: V4.2022.11.3449
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3448 #### Version: V4.2022.11.3449
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12638
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3448 #### Version: V4.2022.11.3449
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12642
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12642
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12642
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12642
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

12642
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

12642
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

12642
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3448 3449

12642
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12642
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.253.230.223 0; 104.214.92.133 0;

12642
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

12642
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

12642
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

12642
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.253.230.223 0; 104.214.92.133 0;

12642
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.253.230.223 0; 104.214.92.133 0;

12642
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Digincore ADmantX
Image Sucker EMail Siphon
LexiBot LinkWalker
PageThing.com PyCurl
RankingBot RankingBot2
Scanbot Tracemyfile
SiteSnagger Web Fetch
WiseGuys Robot ZyBorg
Xaldon_WebSpider sitechecker.pro
spyfu webprosbot

320
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker 404checker
404enemy 404enemy
ADmantX 80legs
ALittle Client ALittle Client
Aboundex Aboundex
AiHitBot Aboundexbot
Alexibot
Alligator
AlphaBot
Anarchie
Anarchy
Ankit Ankit
Anthill Anthill
Aspiegel Aspiegel
Asterias Asterias
Atomseobot
Attach Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot
BBBike BBBike
BDFetch BLEXBot
BackStreet BackDoorBot
BackWeb BacklinkCrawler
BatchFTP Badass
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bigfoot Bigfoot
Bitacle
Blackboard
Blow Blow
BlowFish Boardreader
Bolt Brandwatch
BunnySlippers Buck
BuzzSumo BuiltBotTough
Bullseye
CATExplorador
CODE87
CSHttp CSHttp
Calculon Calculon
Cegbfeieh CazoodleBot
CheTeam CensysInspect
CheeseBot CheeseBot
CherryPicker
ChinaClaw ChinaClaw
Chlooe Citoid
Claritybot
Cliqzbot
Cloud mapping
Cogentbot
Copier
CopyRightCheck CopyRightCheck
Copyscape
Cosmos Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
CrunchBot Crescent
CyotekWebCopy Curious
DBLBot Custo
Demon
Deusu
Devil Devil
Digincore Discoverybot
DigitalPebble
Disco
Dispatch Dispatch
DomainAppender DittoSpyder
DomainSigmaCrawler DnBCrawler-Analytics
DnyzBot
DomCopBot
DomainCrawler
DomainStatsBot DomainStatsBot
Dotbot Domains Project
Download Wonder
Drip Drip
EMail Siphon
Ebingbong Ebingbong
EroCrawler Ecxi
Evil EirGrabber
Exabot ExtractorPro
Extreme Picture Finder Extreme Picture Finder
FDM
FHscan FHscan
FemtosearchBot
Firefox/7.0
FlashGet FlashGet
FrontPage Fuzz
GetRight GetRight
Getintent Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GrapeshotCrawler GoZilla
GridBot GridBot
HMView HTMLparser
HTTP::Lite
HTTrack HTTrack
Haansoft Haansoft
Heritrix HybridBot
HonoluluBot IDBot
IDBTE4M Id-search
IlseBot
Image Fetch Image Fetch
Image Sucker
IndeedBot IndeedBot
Indy Library InfoNaviRobot
InfoTekies
InterGET InterGET
IstellaBot JOC Web Spider
Joomla JamesBOT
Keybot Translation-Search-Machine Jyxobot
Kinza Kinza
Kozmosbot LWP::Simple
LeechFTP Larbin
LeechGet LeechGet
LexiBot
LibWeb LibWeb
LinkWalker Lightspeedsystems
Linkbot Linkbot
LinkextractorPro LinksManager
Lipperhey Lipperhey
Lmspider Lipperhey Spider
Ltx71 MJ12bot
MIIxpc MSFrontPage
Mail.RU_Bot Mail.RU_Bot
Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch
Masscan
Mata Hari
MauiBot MauiBot
Mb2345Browser Mb2345Browser
MegaIndex.ru
Metauri Metauri
MicroMessenger Microsoft URL Control
Microsoft Data Access Mojolicious
Minefield Morfeus Fucking Scanner
Mister PiX Msrabot
Mojeek
MolokaiBot
Musobot
NICErsPRO
NPbot NPbot
Name Intelligence Nameprotect
NearSite Navroad
NetAnts Needle
Net Vampire
NetLyzer NetLyzer
Netvibes NetMechanic
Nibbler NetSpider
NetZIP
NextGenSearchBot
Niki-bot Niki-bot
NimbleCrawler Nuclei
Not
Nutch
Octopus Octopus
Offline Explorer
Offline Navigator
OpenLinkProfiler
OpenVAS
Openfind Openfind
OutclicksBot PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageScorer PageGrabber
PageThing.com Panscient
Pandalytics PeoplePal
Petalbot Piepmatz
Picscout Pimonster
ProPowerBot PleaseCrawl
Pu_iN Probethenet
Pump Psbot
PxBroker PxBroker
RSSingBot PyCurl
QueryN Metasearch
Quick-Crawler
RankActive
RankFlex RankFlex
RankingBot2
RankurBot
ReGet
RealDownload
RebelMouse RebelMouse
RocketCrawler Recorder
Rogerbot
SBIder
SEOkicks
SEOprofiler SEOprofiler
SEOstats SMTBot
SISTRIX
SalesIntelligent SalesIntelligent
ScanAlert Scrapy
SemanticJuice Screaming
SemrushBot ScreenerBot
Searchestate
Seekport
Semrush
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Siphon Siphon
SiteExplorer SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
SiteSnagger SiteSnagger
Sitebeam
Sitevigil Sitevigil
SlySearch SlySearch
SmartDownload Snapbot
Sogou web spider Snoopy
Sottopop Sociscraper
SpaceBison
Spammen Spammen
SpankBot SpankBot
Sqworm Spanner
Stripper Spbot
Sqlworm
SuperHTTP
Suzuran
Swiftbot Swiftbot
Szukacz T0PHackTeam
T8Abot T8Abot
TeleportPro Telesphoreo
Telesoft Telesphorep
The Intraformant The Intraformant
Trendiction Toata
Trendictionbot Turnitin
Turingos
TwengaBot
Twice
URLy Warning
UnisterBot
V-BOT
VB Project VB Project
Vacuum VeriCiteCrawler
WASALive-Bot WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV WEBDAV
WISENutbot WISENutbot
WWWOFFLE WWW-Mechanize
Wallpapers Wallpapers
Web Auto WeSEE
Web Collage Web Enhancer
Web Sauger
WebAuto WebAuto
WebCollage WebCollage
WebCopier WebCopier
WebFetch WebEnhancer
WebFuck
WebGo IS
WebImageCollector
WebReaper WebReaper
WebStripper
WebSucker
WebWhacker
Webshag Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
Webster Whack
Whatweb
Who.is Bot Who.is Bot
Widow Wonderbot
WinHTTrack
WiseGuys Robot
Wotbox
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu Xenu
YoudaoBot Zauba
Zade Zeus
ZmEu ZumBot
ZoomBot ZyBorg
ZoominfoBot archive.org_bot
adscanner arquivo.pt
arquivo-web-crawler autoemailspider
clark-crawler backlink-check
check1.exe
coccocbot coccocbot
crawl.sogou.com cognitiveseo
crawler4j crawler4j
eCatch dataforseo.com
gopher evc-batch
heritrix internetVista monitor
ips-agent iubenda-radar
linkdexbot mediawords
magpie-crawler oBot
meanpathbot page scorer
probe-image-size polaris version
satoristudio.net ripz
seobility scalaj-http
serpstatbot scan.lol
siteripz seocompany.store
sogouspider seoscanners
trendiction.de seostar
spyfu
tAkeOut
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com webpros.com
x09Mozilla x09Mozilla
x22Mozilla
zauba.io

334
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
404enemy 404enemy
AIBOT 80legs
ALittle Client
ASPSeek ASPSeek
Abonti
Aboundex Aboundex
Acunetix Aboundexbot
AhrefsBot AhrefsBot
Aipbot Alexibot
Alligator AlphaBot
Anarchy Anarchy99
Ankit
Apexoo
Aspiegel Aspiegel
AwarioRssBot
BBBike
BDCbot
BDFetch BDFetch
BLEXBot
BackStreet
BacklinkCrawler BacklinkCrawler
Bandit Badass
Barkrowler Battleztar Bazinga
BatchFTP BetaBot
Bigfoot Black Hole
Bitacle BlackWidow
Blow Blackboard
BlowFish Boardreader
BuiltBotTough Bolt
Bullseye Buck
BuiltWith
CATExplorador
CCBot CCBot
CODE87
CSHttp CSHttp
CazoodleBot CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
CheTeam ChinaClaw
CheeseBot Cocolyzebot
Citoid
Claritybot
Cliqzbot
Cloud mapping
Collector Collector
Crawling at Home Project Cosmos
CrazyWebCrawler CrunchBot
Crescent Curious
Custo Custo
DIIbot DIIbot
DSearch DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Deusu
Devil
Digincore Digincore
DigitalPebble DigitalPebble
Disco Dirbuster
Discoverybot Discobot
DomainCrawler DnBCrawler-Analytics
DomainAppender
DomainStatsBot
Dotbot Dotbot
Download Wonder
Drip
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Wolf
EasyDL EasyDL
Ebingbong EroCrawler
Evil Evil
Exabot Express WebPictures
Extractor ExtractorPro
Extreme Picture Finder Extreme Picture Finder
FlashGet FemtosearchBot
Freeuploader Fimap
Gigabot Foobot
FrontPage
Fuzz
Fyrebot
GT::WWW
GermCrawler
GetWeb
Go-Ahead-Got-It Go-Ahead-Got-It
Grabber Gotit
HMView GrabNet
HTMLparser Grafula
HTTP::Lite GrapeFX
GrapeshotCrawler
GridBot
HTTrack HTTrack
Haansoft Haansoft
HaosouSpider
Havij
Hloader Hloader
Humanlinks Id-search
HybridBot IlseBot
IDBTE4M IndeedBot
Iblog Indy Library
InfoNaviRobot InfoTekies
Internet Ninja Internet Ninja
JOC Web Spider Iskanie
IstellaBot
JamesBOT JamesBOT
Jbrofuzz
JennyBot JennyBot
JetCar
Joomla Joomla
Jyxobot Keyword Density
Keybot Translation-Search-Machine Kinza
LNSpiderguy LWP::Simple
Larbin Lanshanbot
LeechGet LeechFTP
LexiBot
Libwhisker
LieBaoFast LieBaoFast
LinkScan
LinkWalker
Linkbot Linkbot
LinkpadBot LinkextractorPro
LinqiaMetadataDownloaderBot
Ltx71 Ltx71
MIIxpc MIIxpc
MQQBrowser MSIECrawler
MSFrontPage Magnet
MTRobot Majestic SEO
Mag-Net Majestic12
Mail.RU_Bot
Majestic-SEO
MarkMonitor MarkMonitor
MarkWatch Mass Downloader
Masscan Mata Hari
MauiBot MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
Metauri Meanpathbot
Microsoft Data Access Mediatoolkitbot
Mister PiX Mister PiX
Mojolicious Mojeek
MolokaiBot Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Msrabot NPbot
Musobot Navroad
NICErsPRO
NearSite
Needle
Nessus Nessus
NetSpider Net Vampire
NetAnts
NetMechanic
NetZIP NetZIP
Netcraft
Nettrack Nettrack
Netvibes
NextGenSearchBot NextGenSearchBot
Niki-bot Nikto
Nimbostratus Nimbostratus
Ninja Nmap
Not
Nutch Nutch
Offline Explorer Offline Navigator
OpenLinkProfiler OpenLinkProfiler
PHPCrawl OrangeSpider
POE-Component-Client-HTTP OutclicksBot
Page Analyzer
Page Analyzer Page Analyzer
PageGrabber PageGrabber
Pandalytics PageScorer
Panscient PageThing.com
Pavuk Pavuk
PeoplePal
Petalbot Petalbot
Piepmatz Pi-Monster
Pockey Picscout
ProPowerBot PictureFinder
ProWebWalker Pimonster
Pu_iN Pixray
PleaseCrawl
Pump Pump
PxBroker
QueryN Metasearch QueryN Metasearch
RSSingBot Quick-Crawler
ReGet RankFlex
RealDownload RankingBot2
Reaper Recorder
RedesScrapy Ripper
RepoMonkey
SBIder SBIder
SEOlyticsCrawler SEOkicks-Robot
SISTRIX SEOprofiler
SEOstats
SalesIntelligent
ScanAlert
ScreenerBot ScreenerBot
ScrepyBot SemanticJuice
Searchestate Shodan
Semrush Site Sucker
SentiBot SiteCheckerBotCrawler
SeobilityBot SiteExplorer
Seomoz
SiteLockSpider
Sitebeam Sitebeam
SlySearch
SmartDownload
Snake
Snapbot Snapbot
Snoopy SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider
Sosospider Sosospider
Spammen SputnikBot
Spanner Sqlworm
Spbot Steeler
Sqlmap Stripper
Sucuri
SuperBot
SurveyBot
Suzuran Suzuran
The Intraformant Szukacz
Thumbor TeleportPro
Telesphorep
TheNomad
Titan Titan
Toweyabot Toata
Tracemyfile
Trendictionbot Trendictionbot
Turnitin True_Robot
TurnitinBot TwengaBot
UnisterBot Twice
VB Project
Vacuum Vacuum
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Voil
Voltron Voltron
WBSearchBot WBSearchBot
WWW-Mechanize WEBDAV
WISENutbot
WPScan
WWW-Collector-E
WWWOFFLE
Wallpapers Wallpapers
Web Fetch
Web Pix
Web Sucker Web Sucker
WebEnhancer WebAuto
WebCollage
WebCopier
WebFetch WebFetch
WebFuck WebFuck
WebImageCollector
WebSauger WebSauger
WebStripper WebWhacker
Webalta WebZIP
WebmasterWorldForumBot Whatweb
Webster Who.is Bot
WinHTTrack WinHTTrack
Wprecon WiseGuys Robot
Xaldon WebSpider Wotbox
Xaldon_WebSpider
Xenu Xenu
YoudaoBot Zade
Zauba
Zeus
Zitebot
ZoomBot
ZoominfoBot ZoominfoBot
ZyBorg
autoemailspider
backlink-check backlink-check
cah.io.community clark-crawler
check1.exe coccocbot
com.plumanalytics
crawler.feedback crawler.feedback
crawler4j crawler4j
facebookscraper dataforseo.com
isitwp.com demandbase-bot
iubenda-radar heritrix
internetVista monitor
linkdexbot linkdexbot
lwp-request
lwp-trivial lwp-trivial
magpie-crawler
mediawords
muhstik-scan muhstik-scan
oBot netEstate NE Crawler
ripz plumanalytics
probe-image-size
s1z.ru s1z.ru
satoristudio.net
scan.lol scan.lol
siteripz seobility
sp_auditbot sitechecker.pro
spyfu sysscan
tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler
webpros.com
webprosbot webprosbot
zgrab x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
ads.gold applyneedy.xyz
bjorkbacken.nu buytizanidineonlinenoprescription.blogspot.com
contentsexpress.com ciceros.co
distver.ru coolgamechannel.com
earn-from-articles.com dataloading.net
mesto-x.com harmonyglen.us
muz-shoes.ru ihtec2019.org
tizanidinehcl4mgisitanarcotic.blogspot.com m292.info
tizanidinehcl4mgtab.blogspot.com pospr.waw.pl
znakom.sibtest.ru writersgroup580.web.fc2.com

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0n-line.tv 00webcams.com
101lesbian.xyz 0lovespells0.blogspot.com
1millionusd.xyz 12bet.com
21h2o.com 12u.info
24-ak.ru 2015god.org
28n2gl3wfyb0.ru 214jaluwobapef.cf
40cg.com 273-fz.ru
4webmasters.com 3w24.com
8jn.dating-app.ru 4istoshop.com
7fon.ru
7makemoneyonline.com
7minuteworkout.com
7search.com
8xv8.com 8xv8.com
aadroid.net aa9bd78f328a6a41279d0fad0a88df1901.com
abclauncher.com absugars.com
academiacsmendoza.org
activecampaign.dreamhosters.com activecampaign.dreamhosters.com
addtoadd.men ad-words.ru
adhome.biz adtech.de
adnotbad.com advokateg.ru
adrenalin-stk.ru aerogo.com
adsfresh.men airlandsea.info
adultactioncam.com aktobe.xkaz.org
afslankpillen2017nl.eu alevinefesleri.com
ahrntal.verymes.xyz allkrim.com
alfa-medosmotr.ru alloysteel.ru
algarveglobal.com alpinism.ru
alldezservice.kz am15.net
allsilver925.co.il apiadanaknet-a.akamaihd.net
alot.com apiwebwebgetcom-a.akamaihd.net
amigobulls.com appleid-verification.com
amoi.tn arcadeplayhouse.com
ample-awards-today.us arenda-avtoprokat-krasnodar.ru
angel1777.kz arendakvartir.kz
ap.senai.br arquapetrarca.info
apelsinnik.website artdestshop.eu
apibetweenlinesn-a.akamaihd.net artefakct.com
apiqualitinknet-a.akamaihd.net autobudpostach.club
appfixing.space autolombard-krasnodar.ru
applicationg29.com avkzarabotok.info
aquarium-pro.ru avtointeres.ru
arvut.org azartmsl.com
as5000.com balans.shahterworld.org
asiengirls.net balook.com
asrvrep-a.akamaihd.net bangla-chat-uk-paralud.ga
atleticpharm.org bankcrediti.ru
auto-moto-elektronika.cz bashtime.ru
automobile-spec.com bbsoldes.fr
autotop.com.ua
aversis.net
avto-oligarh.ru
avtoarenda.by
babosas.com
baixar-musicas-gratis.com
baltgem.com
battlecarnival.su
bbtec.net
bdsmgalls.net bdsmgalls.net
best-seo-solution.com becuo.com
bg6s0.com bestadbid.com
bharatdefencekavach.com bestcurs.org
bigames.online bestserials.com
bimatoprost-careprost.com betwinservice.com
bio.trade-jp.net bgviagrachrx.com
bioscorp.ru bigcareer.info
bkns.vn biglistofwebsites.com
blog.koreadaily.com bimatoprost-careprost.com.ua
blogig.org binaryoptionscops.info
blogs.rediff.com bitnote.co
boltalko.xyz board.f00d.de
bongiornos.info boazpower.com
bonkers.name bombla.org
bonvillan.com bonux.nextview.ru
bonzbuddy.com brendbutik.ru
boole.onlinewebshop.net browsepulse-a.akamaihd.net
breget74.ru btvn.ru
bryansk.zrus.org bupropion-sr-150-mg.us
c1.onedmp.com buytizanidineonlinenoprescription.blogspot.com
call-of-duty.info callejondelpozo.es
calvet.altervista.org candytiner.com
candiceloves.us cardul.ru
canrioloadm.gq cartierreplica.top
canrole.com cctva.tv
carsplate.com cdncash.org
cartierbraceletsreplica.pw cenokos.ru
cartierlove2u.com chastnoeporno.com
cartujano-pre.de cheapjerseysap.com
casinorewards.info chidporn.com
caulderwoodkennels.com chinaelitecheapjerseys.com
cete.ru chiptuninger.com
cfjump.com
ciproandtizanidine.blogspot.com
cjs.com.ru
classicakuhni.ru classicakuhni.ru
clippingphotoindia.com clickbank.net
co.lumb.co clickprotects.com
cocolyze.com clips.ua.ac.be
codysbbq.com clothesforcash.com
comissionka.net cocyq.inwtrade.com
companies-catalog.com colobit.biz
compliance-jess.top com-onlinesupport.site
compliance-julianna.top compliance-alexa.top
connectionstrenth.com compliance-checker.info
cookie-law-enforcement-ee.xyz compliance-jane.top
covadhosting.biz compliance-walt.top
craftburg.ru construmac.com.mx
cubook.supernew.org copro.pw
cyprusbuyproperties.com counterbot.com
czat.wp.pl courtshipgift.com
daisye.top covid-schutzmasken.de
danceuniverse.ru crackguru.tk
dandiyabeats.in cracksplanet.com
danschawbel.com csgo4.win
data.vtc.pw dafatiri.com
data1.scopich.com dailystrength.org
db.speedup.gdn daneshetabiat.com
deda-moroza-zakaz.ru darkbooks.org
delayreferat.ru dataloading.net
deplim.com dealighted.com
detsky-nabytek.info decorazilla.com
dev-seo.blog defenderxtactical.com
dev.citetick.com dentalpearls.com.au
dfwu1019.info
dgfitness.co
diesel-parts28.ru diesel-parts28.ru
dimkino.ru
dipstar.org
directrev.com
disruptingdinnerparties.com disruptingdinnerparties.com
distver.ru divisioncore.com
doktoronline.no dokfilms.net
domik-derevne.ru domain.webkeyit.com
dominterior.org doreenblog.online
downtuptv.gq downloadeer.net
driving.kiev.ua druzhbany.ru
dstroy.su dtnlyss.com
easyukraine.com dyt.net
ecommerce-seo.org e-c.al
ekspertmed.com ednorxmed.com
ekto.ee ekobata.ru
elite-sex-finders.com electricwheelchairsarea.com
eloconcream.blogspot.com electronicadirect.com
eluxer.net elektrozigarettekaufen2.blogse.nl
embedle.com elektrozigaretten1.webs.com
emergencyneeds.org empathica.com
englishdictionaryfree.com enternet.ee
eropho.net epicbrogaming.com
eropornosex.ru erot.co
esfchat.tk escortplius.com
este-line.com.ua eshop.md
estelight.ru exchanges-bet.com
eugenevaultstorage.com exct.net
euromasterclass.ru expertnaya-ocenka.ru
event-tracking.com extener.org
extractorandburner.com
extremepornos.net extremepornos.net
facecup.top ezigarettekaufen2.yolasite.com
favornews.com f-online.de
fearcrow.com face.hostingx.eu
fefo.gdn familienzahnaerzte.com
femdom.twiclub.in fashionindeed.ml
fickenprivat.info fatfasts-4tmz.com
fickkontakte.org feorina.ru
fickluder69.com fickkontaktehobbyhuren.com
filkhbr.com fingerprintjs.com
findsexguide.com
fireads.men fireads.men
firma-legion.ru fitfloponline.store
flashahead.info flavors.me
flex4launch.ru floridamhca.org
fodelsedagspresenter.nu
footbalive.org
forex-indextop20.ru
forex21.ru
forums.toucharcade.com forums.toucharcade.com
frameimage.org
free-fb-traffic.com free-fb-traffic.com
freecamdollars.com fyxabomiw.ru
freelifetimefuckbook.com galaxyflowers.ru
fullfileaccess.com gateway.zscalerone.net
fxund.us gearsadspromo.club
gael-s.ru gemgrab-a.akamaihd.net
galaxys6manual.info gerhardhealer.com
gambarkataku.co get-seo-domain.com
gamedayhouse.com getlaid-xxxhookupdirect.com
gdzkurokam.ru getridofstretchmarks.org
ge0ip.org gg-arena.ru
geile-lelly.eu ghostvisitor.com
getlamborghini.ga gigapeta.com
girlsfuckdick.com girlspicsa.com
glavprofit.ru glass-msk.ru
globetrotting-culture.ru glastecfilms.com.my
gnuetella.com
gogalleryawesome.com
gojiberriess.apishops.ru
golaya.pw
good-mummy.ru good-mummy.ru
goodhousekeeping.com googglet.com
gotomontenegro.net googst2.ru
graphicwe.org goosefishpost.bid
greatfind-a.akamaihd.net gosarhivrt.ru
grtyi.com grand-chlen.ru
gsou.cf greatidea.marketing
guigyverpo.cf gwebtools.com
hamptonoaks.ca habermetre.com
handsandlegs.ru handicapbathtubarea.com
hanwei.us happy.new.yeartwit.com
hard-porn.mobi hausfrauensex18.com
hatedriveapart.com hd-filmy.net
hawaiielectriclight.com helpmymacfaster.trade
health-medical-portal.info himazin.info
healthcarestore.info hol.es
hebr.myddns-flir.com
helicalpile.us
hermesbirkinhandbagoutlets.com
hifidesign.ru
highland-homes.com
himgaws.pw
hkdiiohi.skyrock.com
hmmm.cz
hmywwogw.bloger.index.hr
home.myplaycity.com home.myplaycity.com
honyaku.yahoofs.jp hosting-tracker.com
horoshieokna.com hostsshop.ru
igadgetsworld.com hotgirlhdwallpaper.com
ikearugs.xyz houtings.xyz
ilbe.club howopen.ru
imagez.co howtowhitenteethfast.xyz
imperia31.ru hulfingtonpost.com
in-tandem.co humbmosquina.tk
increasewwwtraffic.info iinstalll-fii1leis.jus0wil.pp.ua
infoupdate.org ilovevitaly.xyz
insomniagamingfestival.com imagine-ex.co
intellekt21.ru infobabki.ru
investpamm.ru infokonkurs.ru
insta-add.pro
investyb.com investyb.com
ipchicken.com investzalog.ru
ivanstroi.ru iqs.biz.ua
iwantedmoney.com iqupdatetmz.win
izismile.com ivoiretechnocom.ci
jav-idol.com iwantmyfreecash.com
javidol.site javitas.info
jbl-charge.info jerseysbizwholesalecheap.com
jimmychoosale.online jmat.cn
journeydownthescale.info johannesburgsingles.co.za
kadashihotel.com johnnyhaley.top
juliadiets.com
kanimage.com
kareliatobacco.ru kareliatobacco.ru
kfon.eu karting196.ru
khadastoafarde.tk karusel-market.ru
katjimej.blog.fc2.com
keywords-monitoring-success.com
kino-key.info
kletkimehan.ru kletkimehan.ru
knigonosha.net kliksaya.com
koleso24.com.ua kollesa.ru
konpax.com konkursowo-24.pl
koptims.tiu.ru laxdrills.com
ks1234.com leadn.pl
lafourmiliaire.com
laulini.soclog.se
law-check-two.xyz
laxob.com
leadwayau.com
lecbter-relationships.ga
lechtaczka.net lechtaczka.net
lerporn.info lennatin.info
letolove.ru lidoradio.com
li-er.ru link.ac
lida-ru.com linkwithin.com
lineshops.biz liumimgx.pw
linkbux.ru livingcanarias.com
live-cam6.info lonerangergames.com
liveinternet.ro longgreen.info
livejasmin.com lostfilm-online.ru
lmrauction.com louboutinreplica.xyz
lolnex.us lovelycraftyhome.com
lowephotos.info ltvperf.com
mafa-free.com lumb.co
magnetic-bracelets.ru mackeeper-land-672695126.us-east-1.elb.amazonaws.com
manipulyator-peterburg.ru macrotek.ru
magento-crew.net
manifestyourmillion.com
mantramusic.ru
map028.com
mapquestz.us mapquestz.us
maximpartnerspr.com maslenka.kz
medicine-4u.org matsdale.com
meduza-consult.ru max-p.men
mazda-roadsters.com
mcar.in.ua
meltwater.com
mere.host.sk mere.host.sk
mesbuta.info metarip.ru
metafilter.com mexicosleevegastrectomy.com
mhtr.be mhi-systems.ru
mnogabukaff.net minet.club
mole.pluto.ro miniature.io
mompussy.net mjchamonix.org
monarhs.info mobile.ok.ru
monclercoats.xyz modnie-futbolki.net
moneyteam24.com moesonce.com
moskva.nodup.ru monclerboots.xyz
most.gov.iq monclercheap.xyz
mp3ritm.top monclerjacketsoutlet.pw
mulberryoutletonlineeu.com mrlmedia.net
music.utrolive.ru mustwineblog.com
mxgetcode.com mxgetcode.com
myanyone.net myblogregistercm.tk
myliveblog.ru mydownlodablefiles.com
mywallpaper.top myhitmp3.club
najaden.no mylovelibrabry.com
narkologiya-voronezh.ru myxdate.info
netanalytics.xyz narkologiya-orel.ru
nextconseil.com nationalbreakdown.com
nfljerseys.online negral.pluto.ro
nfljerseyscheapbiz.us netcheckcdn.xyz
nhl17coinsforps3.gratisblog.biz nextrent-crimea.ru
nofreezingmac.work nfljerseysforsalewholesaler.com
novgorod.xrus.org nightvision746.weebly.com
nowtorrents.com noumeda.com
obsessionphrases.com nufaq.com
odoratus.net o-o-8-o-o.ru
oakleyglassesonline.us
obesidadealgarve.com
oddamzadarmo.eu
odesproperty.com
offergroup.info offergroup.info
oivcvx.website offside2.5v.pl
online-templatestore.com olgacvetmet.com
openfrost.net onblastblog.online
ourville.info online-hd.pl
outclicks.net onlinebay.ru
ownshop.cf onlineporno.site
oz-offers.com openmultipleurl.com
page2rss.com osoznanie-narkotikam.net
paidonlinesites.com oxford-book.com.ua
paleohub.info pandarastore.top
partner-high.men panicatack.com
papasdelivery.ru
partner-stop.men partner-stop.men
pawli.eu partners-ship.pro
paydayonlinecom.com perosan.com
pb-dv.ru petedrummond.com
pc-virus-d0l92j2.pw photosaga.info
pdfprof.com piccdata.com
pdns.cz
pelfind.me
penisvergrotendepillennl.ovh
perfectpracticeweb.com
perm-profnastil.ru
perm.xrus.org
picsforkeywordsuggestion.com picsforkeywordsuggestion.com
pihl.se picswe.com
pillscheap24h.com pictures-and-images.com
pinkduck.ga pierrehardysale.online
platinumdeals.gr pigrafix.at
pojdelo.weebly.com pila.pl
popads.net piluli.info
pirateday.ru
pizza-tycoon.com
plastjulgranar.se
playtap.us
pochemychka.net
pogruztehnik.ru
pokemongooo.ml
pontiacsolstice.info
popcash.net popcash.net
poptool.net popup-jdh.xyz
pornoelita.info pornoforadult.com
pornonik.com poshiv-chehol.ru
pornophoto.xyz pr0fit-b0x.com
pornoplen.com prchecker.info
potolokelekor.ru presleycollectibles.com
pp-budpostach.com.ua primedice.com
preconnubial.usuby.site prizestohandle.club
privat-girl.net
producm.ru
progonrumarket.ru progonrumarket.ru
promgirldresses.xyz pronekut.com
prosperent.com prostitutki-novgoroda.org
pussyspace.net prpops.com
pxhdwsm.com quick-seeker.com
pyramidlitho.webs.com rapidgator-porn.ga
q-moto.ru rarbg.to
qitt.ru razvratnoe.org
randki-sex.com realizmobi.com
rankia.com realmonte.net
raschtextil.com.ua recipedays.com
rep-am.com refudiatethissarah.info
responsive-test.net relatodelpresente.com.ar
rossmark.ru renhacklids.tk
russia-tao.ru resellerclub.com
saletool.ru rixpix.ru
salutmontreal.com rn-to-bsn.com
sampleletters.net rniaeba.ga
scanner-george.top ru-mediaget.ru
scanner-jess.top rusexy.xyz
scanner-john.top rustic-quiver.win
scanner-mark.top rvtv.ru
scanner-marwin.top s8-nowy-wygraj.comli.com
scat.porn saddiechoua.com
scrnet.biz.ua safe-app.net
sdi-pme.com santasgift.ml
search.pch.com sbtdesign.co.uk
scanmyphones.com
scanner-nelson.top
scripted.com
secret.xn--oogle-wmc.com secret.xn--oogle-wmc.com
seinterface.com seeingmeerkat.com
seo-prof1.xyz semprofile.com
seocdvig.ru sex-tracker.com
seolab.top sex.hotblog.top
seozoom.it sexyteens.hol.es
sevenstars7.com shemalegalls.blogporn.in
sex-sex-sex5.com shlyahten.ru
sexobzor.info shymkent.xkaz.org
sherlock.se
shisha-swag.de
shop-electron.ru
shop.acim.org
shopperifymac.com
sicfor.bcu.cc sicfor.bcu.cc
sims-sims.ru simpoed.ufop.br
sindragosa.comxa.com siteaero.com
site3.free-share-buttons.com sitebeam.net
sizeplus.work sitopreferito.it
smartshoppymac.com smart-scripts.com
smokewithrabbits.com social-buttons.com
snomer1.ru socialbookmarksubmission.org
social-fun.ru sodexo.com
socialmonkee.com softxaker.ru
soft-program.com solution4u.com
soheavyblog.com soserfis.com
spb-plitka.ru space4updating.win
sports-supplements.us
spylog.com spylog.com
stats-collector.org spywareit.com
statustroll.com srdrvp.com
sticken.co starpages.net
stroiminsk.com staticfs.host
strongholdsb.ru stoki.ru
stroydetali.ru strana-solnca.ru
studentguide.ru studentguide.ru
subj.ukr-lit.com supermesta.ru
sucsesofinspiration.com surintech.ac.th
superstats.com survival.betteroffers.review
tanieaukcje.com.pl taaaak.com
tds-advert002.info taranerymagesswa.blogspot.com
techart24.com tattomedia.com
thedownloadfreeonlinegames.blogspot.com tattoo33.ru
thegameriders.com telefonsexi.com
thegamerznetwork.com telefonsexkostenlos.tk
thenetinfo.com teletype.in
theprofitsmaker.net templates.radiodigital.co
thetardistimes.ovh thedownloadfromwarez.blogspot.com
thfox.com thesmartsearch.net
time-japan.ru thiegs.reco.ws
tivolibasket.it thruport.com
tizanidine4mgstreetvalue.blogspot.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidinedosage.blogspot.com tmearegion26.com
tizanidineformigraines.blogspot.com tmm-kurs.ru
tizanidinestreetvalue.blogspot.com toloka.hurtom.com
tnaionline.org toonfamilies.net
top1-seo-service.com topappspro.com
topanasex.com topkarkas.com
topdownloads.ru torrents.cd
topflownews.com totu.us
torrent-newgames.com toy-shop.top
trafficmp.com tozup.com
trafficnetzwerk.de trackmedia101.com
trainoffend.ml tracksz.co
trustedhealthtips.com trafaret74.ru
try-rx.com traffic2money.com
tsan.net trafficmania.com
ttrraacckkrr.com trafficreceiver.club
twodollarshows.com trahvid.com
ucsol.ru trailer.cinemaflix.website
ufa.xrus.org turist-strani.ru
umg-stroy.ru tvand.ru
twittruth.com
unacittaconte.org
unimodemhalfduplefw.pen.io unimodemhalfduplefw.pen.io
uptime.com universals.com.ua
urll.eu unrealcommander.org
usacasino.com us-america.ru
vadimkravtcov.ru valmetrundan.se
vapeface.club vapomnoncri.tk
vavilone.com vashsvet.com
vegascosmetics.ru vbabule.net
venerologiya.com vclicks.net
vgoloveboli.net viagengrarx.com
viagroid.ru vielporno.net
video-chat.in vitalads.net
videochat.world vitoriacabos.com
videojam.tv vladhistory.com
videonsk.com voloo.ru
vip2ch.com vonradio.com
virtuagirl.com vtc.pw
visa-pasport.ru vvon.co.uk
viveresaniesnelli.it w-journal.ru
vseuznaem.com wallpapers-best.com
w7s.ru wanker.us
watch-movies.ru watch-movies.ru
website-audit.com.ua webcamdevochka.com
website-speed-check.site webix.me
weclipart.com webmasters.stackexchange.com
wheelchairliftsarea.com webshoppermac.com
whengirlsgowild.com websiteexplorer.info
whiteelephantwellington.com westen-z.life
whos.amung.us westermarkanjou.se
wildcattube.com whatistizanidine2mg.blogspot.com
wingsoffury2.com wholesalejerseysgaa.com
winner7777.net winner7777.net
winterclassichockeyjerseys.com wiosenny-bon-1500.pl
wordpresscore.com wma-x.com
worldinternetauthority.com woman-h.ru
worldis.me worldoffiles.ru
worldmusicfests.com wpthemedetector.co.uk
www.arenda-yeisk.ru xcc24.pl
www.bookmaker-bets.com xlolitka.com
www.labves.ru xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
x-porno.video
x69ty.ru
xclicks.net
xmnb.net
xn----itbkqkfiq.xn--p1ai xn----itbkqkfiq.xn--p1ai
xxxtubesafari.com xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xz618.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
yadro.ru xn--80ahdheogk5l.xn--p1ai
yellocloud.be xn--b1agm2d.net
you-shall-not-pass.is74.ru xn--h1ahbi.com.ua
youjizz.vc xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
youradexchange.com xnxxandxvideos.com
youtube-downloader.savetubevideo.com xxxdatinglocal.us
zaapplesales.blogspot.com xxxtube69.com
zarabotat-na-sajte.ru yeartwit.com
zdesoboi.com youandcredit.ru
zebramart.ru youradulthosting.com
zeg-distribution.com yoursite.com
zooggames.com yugo-star.ru
zscalerone.net zajm-pod-zalog-nedvizhimosti.ru
zytpirwai.net zastroyka.org
zawyna.ua
zeroredirect9.com
zoominfo.com
zverokruh-shop.cz
zvuker.net

490
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 1008.su
1688.com 57883.net
23kw.ru 5tey7463.weebly.com
2rich4bitches.com 6128786.com
5forex.ru 9med.net
8jn.dating-app.ru abiente.ru
abtasty.com academiacsmendoza.org
acortarurl.es actices.com
adidas.frwebs.fr advertisingtag.net
adpremium.org aktivator-windows10.blogspot.com
agreda.pluto.ro alert.scansafe.net
alfapro.ru all4invest.ru
alyeskaresort.com alt-servis.ru
anapa-inns.ru amateurlivechat.org
andpolice.com android-systems.ru
android-vsem.org apimegabrowsebiz-a.akamaihd.net
arcadepages.com awaybird.ru
asia-forum.ru azbukafree.com
atlant-auto.info bala.getenjoyment.net
audiofree.ru bamo.xsl.pt
baixar-musicas-gratis.com banksrf.ru
balook.com bannerconnect.net
bard-real.com.ua bashtime.ru
belstaffstore.online best-mam.ru
berlininsl.com bestkfiledxd.cf
best-deals-products.com bibys.com
bestdraws.com bloglag.com
beztuberkuleza.ru boc.kz
bg6s0.com
binomo.kz
birzha-truda.eu
bizfly.info
bkgr.se
brainzpod.win brainzpod.win
brandehk.dk
brendbutik.ru
brothers-smaller.ru brothers-smaller.ru
businescoop.men bryansk.zrus.org
callejondelpozo.es buckspar.gq
canrioloadm.gq burn-fat.ga
caveavins.fr buron.pw
cdncash.org callcafe.info
cheapestjerseys-wholesale.com camsex.xxx-cam.webcam
cheapmoncler.win carina-sy.de
cheapsergiorossi.online chastnoeporno.com
cherubinimobili.it christianlouboutinreplica.pw
cialischmrx.com
cialischstgerts.com cialischstgerts.com
cityadspix.com clipartnew.com
clevernt.com com-cleaner.systems
compliance-willy.top commoncrawl.org
cookingmeat.ru compliance-jess.top
cornerstone-countertops.com cornomase.win
credit.co.ua covetnica.com
crest3d.ru culmimg.pw
cupday.com customsua.com.ua
czeshop.info daooda.com
de.zapmeta.com davebestdeals.com
dealitright.click delitime.info
decoratinghomes.ga diarioaconcagua.com
dekorkeramik.ru dieswaene.com
digilander.libero.it div.as
distver.ru docsportal.net
dms-sw.ru dojki365.online
drillsaw.ru dood.live
eksprognoz.ru dorratex.tn
ekto.ee downloadme.life
elektir.ru dptaughtme.com
elidelcream.weebly.com e2click.com
elitedollars.com eropho.com
elitesportsadvisor.com eu-cookie-law.blogspot.com
elmifarhangi.com everytalk.tv
eropornosex.ru extlabs.io
ertelecom.ru extremez.net
es5.com ezrvrentals.com
esfchat.tk ferrotodo.com
europages.com.ru fickenprivat.info
everflora.ru fix-website-errors.com
expolicenciaslatam.co flooringinstallation-edmonton.com
facecup.top foto-basa.com
failingmarriege.blogspot.com fotopop.club
falopicm.pw freecamdollars.com
fanrto.com gardene.ru
faptitans.com gay-file.com
faracontrol.ir gembird.com
fetlifeblog.com getlaid-xxxhookupdirect.com
ff30236ddef1465f88547e760973d70a.com gheus.altervista.org
financeloan.us girlspicsa.com
freejabs.com gne8.com
freenode.info goodprotein.ru
freeuploader.com gsasearchenginerankerdiscount.com
fungirlsgames.net habermetre.com
gamingspark.com hatedriveapart.com
gateway.zscalerone.net hdimagegallery.net
gotcher.us hkladys.com
gymi.name hothor.se
hdapp1008-a.akamaihd.net hplaserjetpdriver8y.pen.io
hesteel.pl hptwaakw.blog.fc2.com
homeandhealth.ru hydropump.su
hotblognetwork.com iboss.com
hotsocialz.com idealtits.net
houtings.xyz imgata.com
hoverboard360.de imitex-plus.ru
htmlcorner.com interfucks.net
huimin764128.com iskalko.ru
humanorightswatch.org iwanttodeliver.com
ifirestarter.ru kabbalah-red-bracelets.com
impisr.edunsk.ru kakadu-interior.com.ua
infoupdate.org karusel-market.ru
intim-uslugi.info kenaba.su
investyb.com kino-ecran.ru
jasonpartington.com kinoplen.ru
jerseychinabizwholesale.com kiprinform.com
jmat.cn kollesa.ru
joingames.org lacapilla.info
justucalling32211123456789.tk lalalove.ru
ketoanhanoi.info lanasshop.ru
kfon.eu larger.io
kinogonew.ru law-check-seven.xyz
klitimg.pw life-instyle.com
konkursov.net lineavabit.it
krasivoe-hd.net linkbuddies.com
krasula.pp.ua linkdebrideur.xyz
kupiproday.com.ua lion.snapmap.us
laparfumotec.com lol-smurfs.com
lazy-z.com lotto6888.com
ledpolice.ru lowephotos.info
letolove.ru m1media.net
lietaer.com m4ever.net
liky.co.ua mabdoola.blogspot.com
longadventure.com macdamaged.tech
magicplayer-s.acestream.net mackeeper-land-672695126.us-east-1.elb.amazonaws.com
manifestation.betteroffers.review macresource.co.uk
medi-fitt.hu mansparskats.com
mirtorrent.net marwer.info
missis.top matematikus.info
mistr-x.org matsdale.com
mixx.com meet-flirt-dating.com
mobile-appster.ru misslike.ru
moesexy.com missvietnam.org
moinozhki.com mnogabukaff.net
moviezbonkerssk.cf moivestiy.biz
mundoaberrante.com moneytop.ru
mycaf.it moonci.ru
nakrutka.cc mp3ringtone.info
napalm51.nut.cc musicdaddy.net
narkologiya-belgorod.ru muycerdas.xxx
narkologiya-penza.ru mylesosibirsk.ru
naturalpharm.com.ua myliveblog.ru
nowtorrents.com nastydollars.com
obsessionphrases.com nationalbreakdown.com
offers.bycontext.com news-speaker.com
online-hd.pl niroo.info
parfusale.se o00.in
pathwhelp.org obiavo.in
pc4download.co obnal.org
perederni.net odesproperty.com
perfectplanned.com omgtnc.com
petrovka-online.com one-gear.com
pewit.pw onlinadverts.com
pflexads.com oyster-green.com
pinup-casino1.ru paintingplanet.ru
pixshark.com paperwritingservice17.blogspot.ru
pizdopletka.club pawli.eu
pk-services.ru pechikamini.ru
plugingeorgia.com pelfind.me
pokerniydom.ru picsfair.com
pornogad.com playmsn.com
pornosexrolik.com popugaychiki.com
preventheadacheguide.info porno-chaman.info
printdirectforless.com pornolook.net
prize44.com pornoplen.com
pro-tec.kz pornoxxx.com.mx
prodess.ru predominant-invent.tk
producm.ru prokotov.com
produkto.net psa48.ru
profit-opportunity.com ptr.ruvds.com
promalp-universal.ru puzzleweb.ru
prostitutki-kiev.org pyramidlitho.webs.com
prostitutki-yaroslavlya.org q-moto.ru
purchasepillsnorx.com quickloanbank.com
quickchange.cc r-e-f-e-r-e-r.com
rabot.host.sk repeatlogo.co.uk
rapidsites.pro revistaindustria.com
rcpmda.ikan1080.xyz rome2rio.com
registry-cleaner.net rusoft-zone.ru
relax.ru sack.net
restorator-msk.ru savingsslider-a.akamaihd.net
rethinkwasteni.info searchengineranker.email
russintv.fr seasonvar.ru
russkie-gorki.ru serptehnika.ru
salut-camp.ru sexgalleries.top
samoiedo.it shoesonlinebuy.xyz
scanner-walter.top signoredom.com
searchwebknow-a.akamaihd.net signx.info
seasaltwithfood.com sitiz.club
security60-e.com sledstvie-veli.net
servisural.ru social-buttons.com
sideeffectsoftizanidine.blogspot.com softlinesolutions.me
sims-sims.ru soietvousmaime.fr
socialbutton.xyz solitaire-game.ru
softomix.net sonata-arctica.wz.cz
solution4u.com spaceshipad.com
sophang8.com sport-video-obzor.ru
spb.ru spravka-medosmotr.ru
sr-rekneskap.no staff.prairiesouth.ca
startufa.ru stairliftstrue.com
stockquotes.wooeb.com statistici.ro
stromerrealty.com storehouse.ua
superfish.com strfls.com
svarbit.com stroyhelp-dv.ru
swimpool.ca syvertsen-da.no
sygraem.com technika-remont.ru
teenfuck.tv teenforporn.com
themeforest.net theguardlan.com
thetoiletpaper.com theheroes.ru
timhost.ru thin.me.pn
tizanidinehcl4mgtablet.blogspot.com tizanidineinfo.blogspot.com
tizanidinezanaflex.blogspot.com
tmm-kurs.ru tmm-kurs.ru
token-lab.org torrnada.ru
topdownloads.ru toursmaps.com
u17795.netangels.ru track112.site
uhodzalijami.ru trafficnetzwerk.de
ukrtextbook.com trumpetedextremes.com
unimodemhalfduplefw.pen.io trustedmaccleaner.com
urccvfmc.bloger.index.hr turvgori.ru
vip-file.com tvory.predmety.in.ua
vip.51.la ultimatesetnewfreeallsoftupgradesystems.pw
virtuagirl.com umekana.ru
visa-pasport.ru unionmarkt.de
visitcambridge.org univerfiles.com
vkmusics.ru unrealcommander.com
vltai.com vasileostrovsky-rayon.ru
voloslove.ru venerologiya.com
webshoppermac.com viandpet.com
videooko.weebly.com
vikistars.com
vilingstore.net
vipromoffers.com
vkontaktemusic.ru
vladimir.zrus.org
volgograd.xrus.org
vroze.com
vzglyadriv.kg
w-journal.ru
warningwar.ru
webkeyit.com
wellcome2slovenia.ru wellcome2slovenia.ru
wholesalejerseychinaoutlet.com westermarkanjou.se
wineitudes.wordpress.com word-vorlagen.net
winterclassichockeyjerseys.com xclicks.net
wnoz.de xerox-douglas.cf
wordkeyhelper.com xingzi-vision.com
wordpress-crew.net xn----7sbho2agebbhlivy.xn--p1ai
wordpresscore.com xxxnatelefon.ru
world-mmo.com yatrk.xyz
x69ty.ru youtubedownload.org
xmladserver.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xnxx-n.com
za-fun-offer.com
zajm-zalog-krasnodar.ru

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.227 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3448 ### Version: V4.2022.11.3449
### Updated: Thu Nov 3 10:03:57 UTC 2022 ### Updated: Thu Nov 3 22:01:35 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3448 ### Version: V4.2022.11.3449
### Updated: Thu Nov 3 10:03:57 UTC 2022 ### Updated: Thu Nov 3 22:01:35 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##