vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3459 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-08 22:07:09 +00:00
parent 211871e438
commit 03370ddc18
47 changed files with 95368 additions and 95358 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3458 #### Version: V4.2022.11.3459
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3458 #### Version: V4.2022.11.3459
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3458 #### Version: V4.2022.11.3459
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11784
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3458 #### Version: V4.2022.11.3459
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11788
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11788
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11788
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11788
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 1; 10.1.1.191 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 0; 10.1.1.191 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 1; 10.1.1.191 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 0; 10.1.1.191 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 1; 10.1.1.191 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 0; 10.1.1.191 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3458 3459

11788
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11788
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.173.248.33 0; 13.64.21.43 0;

11788
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 1; 10.1.1.191 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 0; 10.1.1.191 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 1; 10.1.1.191 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 0; 10.1.1.191 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 1; 10.1.1.191 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 0; 10.1.1.191 0;
127.0.0.1 0; 127.0.0.1 0;

11788
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.173.248.33 0; 13.64.21.43 0;

11788
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
172.173.248.33 0; 13.64.21.43 0;

11788
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 1; 10.1.1.191 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AfD-Verbotsverfahren Buddy
GetWeb CATExplorador
Mag-Net Deusu
Name Intelligence Discobot
Nimbostratus Domains Project
Scanbot FlashGet
Sucuri LinqiaMetadataDownloaderBot
URLy Warning Mail.RU_Bot
WebEnhancer WWW-Collector-E
x22Mozilla Xaldon_WebSpider

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
80legs 80legs
AIBOT ADmantX
ASPSeek Abonti
Acunetix Aboundexbot
AhrefsBot
Aipbot Aipbot
Alexibot
AllSubmitter
Anarchie Anarchie
Anarchy Anarchy
Ankit Anarchy99
Anthill Anthill
Atomseobot Apexoo
Attach BBBike
AwarioRssBot
AwarioSmartBot
BDCbot BDCbot
BDFetch BLEXBot
BackStreet BackDoorBot
BacklinkCrawler Backlink-Ceck
Badass
Barkrowler
Battleztar Bazinga
BetaBot BetaBot
Black Hole Bitacle
BlackWidow
Blackboard Blackboard
Bolt
Brandprotect Brandprotect
Brandwatch Buddy
Buck BuiltBotTough
BuiltWith BunnySlippers
BuzzSumo
CATExplorador
CODE87 CODE87
Cegbfeieh CazoodleBot
CensysInspect CensysInspect
CheTeam Chlooe
CherryPicker Cocolyzebot
ChinaClaw CopyRightCheck
Cliqzbot
Cloud mapping
Cogentbot
Collector
Copier
Cosmos
CrazyWebCrawler CrazyWebCrawler
Crescent CrunchBot
Custo
CyotekWebCopy
DTS Agent DTS Agent
Devil DataCha0s
Demon
Digincore Digincore
DigitalPebble Dirbuster
Disco
Dispatch Dispatch
DomCopBot DittoSpyder
DnyzBot
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Domains Project ECCP/1.0
EasyDL EasyDL
Ecxi EirGrabber
Exabot EroCrawler
ExtLinksBot
Extractor Extractor
Ezooms Extreme Picture Finder
EyeNetIE
FDM
FHscan FHscan
FemtosearchBot
Fimap
FlashGet FlashGet
Foobot FrontPage
Freeuploader G-i-g-a-b-o-t
GT::WWW Genieo
Gigabot GetWeb
GoZilla Go-Ahead-Got-It
GrabNet Gotit
GrapeFX HMView
GridBot HTTP::Lite
HEADMasterSEO
Haansoft
Harvest Harvest
Hloader Hloader
HybridBot Humanlinks
IRLbot Iblog
Id-search Id-search
Image Fetch Image Sucker
Internet Ninja InfoNaviRobot
InternetSeer InfoTekies
InterGET
JOC Web Spider JOC Web Spider
Jbrofuzz
JikeSpider JikeSpider
Joomla
Jorgee
JustView JustView
Jyxobot Kenjin Spider
Keyword Density Keyword Density
Kinza Lanshanbot
LNSpiderguy
LWP::Simple
Larbin Larbin
Leap
LeechGet
LexiBot LexiBot
LibWeb LibWeb
Libwhisker
Lightspeedsystems Lightspeedsystems
LinqiaMetadataDownloaderBot Likse
LinqiaRSSBot LinkpadBot
LinksManager
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider Lipperhey
Litemage_walker
Lmspider Lmspider
MIDown tool MIIxpc
MSFrontPage
MTRobot MTRobot
Mag-Net
Mail.RU_Bot
Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkMonitor MarkMonitor
MarkWatch MarkWatch
Masscan Mass Downloader
Mata Hari Mata Hari
Mb2345Browser MauiBot
Mediatoolkitbot
Metauri Metauri
Microsoft Data Access MicroMessenger
Minefield Minefield
Mister PiX Morfeus Fucking Scanner
Moblie Safari Msrabot
Mojeek
Mr.4x3
NICErsPRO
NPbot NPbot
Name Intelligence Name Intelligence
Nameprotect Nameprotect
Nessus Navroad
Net Vampire
NetAnts NetAnts
NetSpider
NetZIP NetZIP
Netcraft Nettrack
Niki-bot Netvibes
Nikto Nimbostratus
Nuclei Ninja
Offline Navigator Nmap
Not
Nutch
Octopus
OnCrawl
OpenVAS OpenVAS
OutfoxBot Openfind
Openvas
OrangeSpider
OutclicksBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageScorer PageAnalyzer
PeoplePal Panscient
Pi-Monster Papa Foto
Picscout Petalbot
Piepmatz Picsearch
Pimonster PleaseCrawl
Pixray
Pockey Pockey
ProPowerBot ProWebWalker
Probethenet Probethenet
Pu_iN
Pump
PxBroker PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler Quick-Crawler
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
Re-re
Reaper
RebelMouse
Recorder Recorder
RepoMonkey RepoMonkey
SEOkicks
SEOprofiler
SMTBot SMTBot
SalesIntelligent
ScanAlert
Scanbot
ScoutJet ScoutJet
Scrapy ScrepyBot
Screaming SemrushBot
ScreenerBot SeobilityBot
Searchestate
SentiBot
Seomoz Seomoz
Site Sucker Shodan
Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer Snoopy
SiteLockSpider Sosospider
SiteSucker Sottopop
SmartDownload
SpaceBison SpaceBison
Spammen Sqlmap
SpankBot
SputnikBot
Sqlworm
Sqworm Sqworm
Stripper Steeler
Sucuri Sucuri
SuperHTTP SurveyBot
Surfbot Suzuran
Swiftbot
T0PHackTeam
Teleport
TeleportPro TeleportPro
The Intraformant Telesphoreo
Telesphorep
Thumbor
Titan
Toata Toata
Tracemyfile
Trendiction Trendiction
Trendictionbot True_Robot
Turingos Turingos
TwengaBot Twice
URLy Warning
URLy.Warning URLy.Warning
UnisterBot
Upflow Upflow
VB Project VB Project
VCI VCI
VeriCiteCrawler VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
VoidEYE VoidEYE
Voltron Voltron
WBSearchBot
WEBDAV WEBDAV
WPScan
WWW::Mechanize WWW::Mechanize
Wallpapers WWWOFFLE
Wallpapers/3.0
WeSEE WeSEE
Web Auto Web Auto
Web Collage Web Sucker
Web Fuck WebAuto
WebBandit WebCopier
WebFuck WebFetch
WebImageCollector
WebLeacher
WebSucker
WebWhacker WebWhacker
Webalta Webalta
WebsiteExtractor WebmasterWorldForumBot
WebsiteQuester Whacker
Who.is Bot Whatweb
WinHTTrack Widow
Wotbox WiseGuys Robot
Zermelo Wonderbot
Wprecon
Xenu
YoudaoBot
Zade
Zeus Zeus
ZoomBot ZyBorg
adscanner
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
autoemailspider backlink-check
cah.io.community cah.io.community
clark-crawler
coccocbot coccocbot
cognitiveseo
crawl.sogou.com crawl.sogou.com
facebookscraper crawler.feedback
instabid demandbase-bot
domainsproject.org
eCatch
isitwp.com
iubenda-radar iubenda-radar
lwp-trivial lwp-request
meanpathbot mediawords
muhstik-scan netEstate NE Crawler
oBot oBot
page scorer page scorer
probe-image-size satoristudio.net
scan.lol seoscanners
seostar siteripz
sp_auditbot sp_auditbot
trendiction.com trendiction.com
webgains-bot voyagerx.com
webmeup-crawler webprosbot
webpros.com
x22Mozilla x22Mozilla
zauba.io zauba.io

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404checker
404enemy
ADmantX ADmantX
ALittle Client ASPSeek
Aboundex
Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot Aipbot
AllSubmitter
Anarchie Anarchie
Apexoo Anthill
Aspiegel
Atomseobot Atomseobot
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike
BDCbot BDCbot
BDFetch BackDoorBot
BackWeb
Backlink-Ceck
Badass Badass
BatchFTP Bandit
Bigfoot Barkrowler
Bitacle Battleztar Bazinga
Blow Black Hole
BlowFish
Brandwatch Brandwatch
Buddy Buck
BuiltBotTough BuiltBotTough
Bullseye
BunnySlippers BunnySlippers
CATExplorador
CCBot CCBot
CODE87
CSHttp CSHttp
Calculon Cegbfeieh
CazoodleBot
CensysInspect
CheTeam CheTeam
ChinaClaw CherryPicker
Citoid Chlooe
Claritybot Cliqzbot
Cogentbot Cloud mapping
Copier Collector
Copyscape Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
Crescent CrazyWebCrawler
Custo DatabaseDriverMysqli
DIIbot Deusu
DSearch
DTS Agent
Demon
Devil Devil
Digincore
Dirbuster
Disco
Discobot Discobot
Discoverybot Discoverybot
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomainAppender DomainAppender
DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot
Dragonfly
EMail Siphon
EasyDL EasyDL
EirGrabber Extractor
EroCrawler
Extreme Picture Finder
Ezooms Ezooms
FDM FDM
FHscan Freeuploader
FemtosearchBot GT::WWW
Fimap
Flunky
Genieo Genieo
GermCrawler GermCrawler
GetRight
Getintent Getintent
Gigabot Gigabot
Go-Ahead-Got-It Go!Zilla
Grabber
Grafula
GrapeFX
HEADMasterSEO
HMView
HTTP::Lite
HTTrack HTTrack
Haansoft IDBTE4M
Heritrix Iblog
Hloader
HonoluluBot
IRLbot
Id-search Id-search
Image Fetch IndeedBot
Image Sucker Indy Library
InfoTekies
InterGET
Internet Ninja Internet Ninja
IstellaBot Iskanie
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
JennyBot JennyBot
JetCar JetCar
JikeSpider
Joomla
JustView JustView
Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density Kozmosbot
Larbin Lanshanbot
LexiBot Leap
Lftp LibWeb
LinqiaMetadataDownloaderBot Libwhisker
Lipperhey Spider Lipperhey Spider
Litemage_walker
Lmspider Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MIIxpc MIIxpc
MTRobot
Mail.RU_Bot
Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkMonitor Mass Downloader
Masscan
Mb2345Browser Mb2345Browser
Mediatoolkitbot MeanPath Bot
Metauri MicroMessenger
Microsoft URL Control Microsoft URL Control
Moblie Safari Minefield
Mojeek Mojeek
Mozlila Morfeus Fucking Scanner
Mr.4x3 Mr.4x3
Musobot Msrabot
NICErsPRO NICErsPRO
Name Intelligence Name Intelligence
Nameprotect
NearSite NearSite
Needle Needle
Nessus Nessus
Net Vampire
NetAnts
NetSpider NetSpider
Netcraft Netcraft
Nettrack Netvibes
Nibbler
Niki-bot Niki-bot
Nikto
Nimbostratus Nimbostratus
Ninja
Nmap
Nuclei
Nutch Nutch
Offline Explorer Octopus
OnCrawl
OpenVAS OpenVAS
Openvas Openvas
OrangeBot OrangeBot
OrangeSpider PECL::HTTP
OutclicksBot
OutfoxBot
PHPCrawl
POE-Component-Client-HTTP
PageAnalyzer
PageScorer PageScorer
PageThing.com Panscient
Papa Foto PictureFinder
PeoplePal
Pi-Monster
Picsearch
Pimonster
Pixray Pixray
PleaseCrawl ProWebWalker
Pockey
Probethenet Probethenet
Psbot
Pu_iN Pu_iN
Pump
PxBroker PxBroker
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankFlex RankFlex
RankingBot RankingBot2
Rankivabot RealDownload
RankurBot Reaper
Re-re RebelMouse
Recorder RepoMonkey
SEOkicks-Robot RocketCrawler
SEOstats SEOkicks
SISTRIX SISTRIX
SMTBot
SalesIntelligent SalesIntelligent
ScanAlert
Scanbot Scanbot
Scrapy Scrapy
Screaming
ScrepyBot
Seekport
SemrushBot
SentiBot SentiBot
Seomoz SeobilityBot
Shodan Shodan
Siphon Site Sucker
SiteExplorer
SiteLockSpider SiteLockSpider
Sitevigil Siteimprove
Snake SlySearch
Sogou web spider SmartDownload
Sosospider Snoopy
SpankBot SpaceBison
Spammen
Spbot Spbot
SputnikBot
Sqlmap Sqlmap
Sqworm Sqworm
Steeler
Stripper Stripper
SuperHTTP Sucker
Suzuran Sucuri
Surfbot
T0PHackTeam
T8Abot T8Abot
Telesoft Telesphorep
Telesphoreo
TheNomad TheNomad
Toata Thumbor
TightTwatBot
Tracemyfile
Trendictionbot
True_Robot True_Robot
Turnitin TwengaBot
Typhoeus Upflow
URLy Warning
URLy.Warning
VB Project
Vacuum Vacuum
VidibleScraper
Virusdie
Voltron Voltron
WBSearchBot WASALive-Bot
WPScan WWW::Mechanize
Wallpapers WeSEE
Wallpapers/3.0
WallpapersHD
Web Collage Web Collage
Web Pix Web Enhancer
WebAuto WebAuto
WebFetch WebCopier
WebFuck WebFuck
WebLeacher WebReaper
WebSauger
WebWhacker WebWhacker
Webshag WebZIP
WebsiteQuester WebsiteQuester
Whatweb Whack
Widow Widow
Wonderbot Woobot
Xaldon WebSpider Wotbox
Wprecon
YoudaoBot
Zade
Zermelo Zermelo
ZmEu Zitebot
ZumBot
ZyBorg ZyBorg
arquivo.pt archive.org_bot
autoemailspider backlink-check
clark-crawler
coccocbot coccocbot
crawl.sogou.com crawler4j
dataforseo.com demandbase-bot
eCatch eCatch
evc-batch
facebookscraper facebookscraper
isitwp.com lwp-request
iubenda-radar
linkdexbot
lwp-trivial
magpie-crawler magpie-crawler
mediawords
netEstate NE Crawler netEstate NE Crawler
oBot polaris version
page scorer probe-image-size
satoristudio.net
scan.lol scan.lol
seobility
seocompany.store
seoscanners seoscanners
seostar seostar
sitechecker.pro serpstatbot
spyfu siteripz
sogouspider
sp_auditbot
sysscan sysscan
trendiction.com tAkeOut
ubermetrics-technologies.com voyagerx.com
webgains-bot
webmeup-crawler webmeup-crawler
xpymep1.exe
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
3-letter-domains.net bloggedporn.com
adrunnr.com erotag.com
bolitgorlo.net feel-planet.com
kichenaid.ru konkursov.net
lampokrat.ws porno-video-chati.ru
privetsochi.ru pornohub.me
scanner-donald.top prostitutki-almata.org
shoppingmiracles.co.uk tizanidinesideeffects.blogspot.com
webmonetizer.net tizanidinewithdrawalsymptoms.blogspot.com
xxxmania.top xn--80aanaardaperhcem4a6i.com

932
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
24x7-server-support.site 1000n1.ru
3rbseyes.com 1001watch.com.ua
45en.ru 178evakuator178.ru
4kpics.rocks 1adult.com
4ur.click 1qingdao.com
8lufu.com 23kw.ru
aa08daf7e13b6345e09e92f771507fa5f4.com 2pxg8bcf.top
aaa38852e886ac4af1a3cff9b47cab6272.com 4inn.ru
aaronabel.com 4kplayer.pl
acc.eu.org 5forex.ru
acmebtn.ml 999webdesign.com
adamoads.com a96527gi.beget.tech
add-add.men aa8780bb28a1de4eb5bff33c28a218a930.com
adserver-e7.com abusora.com
adsref.men abwa.tk
affiliate-fr.com activecampaign.dreamhosters.com
adf.ly
adktrailmap.com
adman.gr
adsloads.com
akita.kz
aktivator-windows10.blogspot.com
aktobe.xkaz.org
ald2014.org
alfapro.ru
alice.it
allproblog.com allproblog.com
alphaforum.ru allwomen.info
aludecor.info alta-realestate.com
alyeskaresort.com ameblo.jp
amateurgalls.com amyfoxfitness.com
amospalla.es anabolics.shop
anonymous-redirect.com ananumous.ru
apartamentwroclaw.eu anticrawler.org
apiitsthirteende-a.akamaihd.net antons-transporte.de
appfastplay.com apiallgeniusinfo-a.akamaihd.net
architecturebest.com apiappenableinfo-a.akamaihd.net
apilinkswiftco-a.akamaihd.net
apishops.ru
arcteryxstore.online
arenda-yeisk.ru
arewater.com arewater.com
arquivo.pt arius.tech
artefakct.com artdestshop.eu
articlesdirectoryme.info atovh.local-finders.com
asacopaco.tk autobudpostach.club
asiengirls.net aviav.co
asrv-a.akamaihd.net avkzarabotok.com
atmagroup.ru avramstroy.ru
atout-energie-69.com babespcs.com
aussie-prizes.com babieca.com
autonew.biz back.dog
avtointeres.ru baditri.com
azartniy-bonus.com bankiem.pl
azlex.uz banksrf.ru
babosas.com bannerpower.com
bank.uz bannerspace.com
beauty-bracelet.com beachpics.com
berdasovivan.ru beatifullandscape.co
bestmarriages.com belinvestdom.by
beta.hotkeys.com bestempresas.es
betwinservice.com bet-prognoz.com
beyan.host.sk beyan.host.sk
bigames.online bezlimitko.xyz
bimatoprost-careprost.com.ua bharatdefencekavach.com
bio.trade-jp.net
biznesluxe.ru
biznesschat.net biznesschat.net
bjorkbacken.nu bjgugu.net.ua
blog-hits.com blackhatworth.com
bloggers.nl blog.axant.it
bochemit.com.ua blog.koorg.ru
bosman.pluto.ro blogig.org
bmw-ark.ru
bonkers.name
bonux.nextview.ru
braindaily.xyz
brainsandeggs.blogspot.com
brandehk.dk
brewdom.ru brewdom.ru
buy-forum.ru brothers-smaller.ru
buypuppies.ca brus-vsem.ru
buzzonclick.com bryansk.zrus.org
buzzzg.men burger-tycoon.com
bvlgariwallet.xyz buyessayonline19.blogspot.ru
cacheimages.com calc-for-credit.ru
cakesplus.com.au camdolls.com
callawaygolfoutlet.online candiceloves.us
calstaterealty.us cayado.snn.gr
canacopegdl.com
candy-glam-hp.com
carabela.com.do
carina-sy.de
carloans.com
cartierlovebracelet.xyz
cartierreplica.pw
cartiertrinity.xyz
cbozhe.com
ccjp.eu
cdn.walkme.com cdn.walkme.com
ceotrk.com cezartabac.ro
cgi2.nintendo.co.jp
chatroulette.video
cheapbarbour.online cheapbarbour.online
cheapbelstaff.online cheapjerseysap.com
checkhit.com chidporn.com
christianlouboutinshoes.xyz cibpenonptib.flu.cc
cialischmrx.com ciceros.co
circlesl.com cinemaenergy-hd.ru
citetick.com cleanersoft.com
clickbanksites.info cleanmypc.com
cntravelre.com cleanpcnow.com
cometorussia.net click2pawn.com
compliance-elena.xyz club-lukojl.ru
compliance-fred.xyz compliance-alexa.top
compliance-nelson.top confib.ifmo.ru
compliance-viktor.top contentlook.co
cornomase.win cookie-law-enforcement-aa.xyz
cougfan.info cool-wedding.net
corta.co
counciltally.com
crazyprotocol.com
creams.makeforum.eu
creditmoney.com.ua
crest-poloski.ru
crirussian.ru crirussian.ru
culmimg.pw
custodieva.ru custodieva.ru
d0t.ru customergrowthsystems.com
dandiyabeats.in cypernhuset.se
data-mining.tk czeshop.info
dating-app.ru daneshetabiat.com
datract.com demenageur.com
daydream-studio.ru demon-tweeks.com
dellalimov.com detalizaciya-tut.biz
design-lands.ru dgfitness.co
devochki.top diesel-parts28.ru
dichvuvesinhhanoi.com dieta-personalna.pl
dildofotzen.net digitalassetmanagement.site
discountbarbour.online dikqlyremy.info
divatraffic.com disruptingdinnerparties.com
dmmspy.com dktr.ru
doeco.ru dneprsvet.com.ua
dogoimage.com domainsphoto.com
domznaniy.ru domznaniy.ru
doreenblog.online doublepimp.com
doxyporno.com downloads-whatsapp.com
drivotracker.com droidlook.net
drunkmoms.net e-poker-2005.com
earn-from-articles.com eas-seo.com
education-cz.ru ekaterinburg.xrus.org
elektrozigarette2.wordpressy.pl elstal.com.pl
empirepoker.com emailaccountlogin.co
enhand.se en.altezza.travel
erotag.com encodable.com
escort-russian.com ero-advertising.com
esfchat.tk essay-edu.biz
esnm.ru evening-dating-club.info
exchangeit.gq exoclick.com
ezigarettekaufen.myblog.de extstat.com
ezigarettenkaufen2.dreamwidth.org eyemagination.com
f012.de face.hostingx.eu
facecup.top
fakehandbags.xyz fakehandbags.xyz
family1st.ca familienzahnaerzte.com
fanpagerobot.com faracontrol.ir
fettefrauen.net fashion-stickers.ru
fil.ru fdzone.org
fitfloponline.store felizporno.com
flash4fun.com ferrotodo.com
floridahuntingfishingadventures.com fetroshok.ru
foodcrafts.website filefilter.weebly.com
forzeronly.com filkhbr.com
fotopop.club filmidivx.com
fototravel.eu flashahead.info
franklinfire.co forensicpsychiatry.ru
free-deals.faith forum69.info
free-video-tool.com foto-telok.net
free411games.com foxtechfpv.com
freejabs.com franch.info
freelifetimefuckbook.com free-traffic.xyz
freemags.cc freewlan.info
freeseedsonline.com friendflnder.com
freesitetest.com froggytube.com
fuckbuddybestgilf.info fsakhalin.ru
fuckingawesome.com funnypica.com
fun-mobi.pl fx-brokers-review.com
fxtips.ru galleryawesome.com
g.starmoe.xyz gate5.co.za
gambarkataku.co ge0ip.org
gamevalue7.weebly.com genericviagrasildenafiled.net
gaming-journal.com get-your-social-buttons.info
gamingspark.com gfaq.ru
gavuer.ru glogow.pl
gdcentre.ru goblacked.com
gdebestkupit.ru gobongo.info
gdzkurokam.ru goo.ne.jp
gelezki.com goodprotein.ru
gentamicineyedrops.blogspot.com goroda-vsego-mira.ru
gepezz.info
get-seo-domain.com
getresponse.com
getridofstretchmarks.org
gheus.altervista.org
glasof.es
goodhousekeeping.com
gotorussia.com
grand-chlen.ru
graphid.com
greamimgo.pw
gsasearchengineranker.xyz
gsmlab.pl gsmlab.pl
hatdc.org gsmtlf.ru
hawaiielectriclight.com guruofcasino.com
hcate.com handicapbathtubarea.com
healbio.ru hasfun.com
home-task.com hdfreeporno.net
homedo.fabpage.com hdpixion.com
hotcore.info hdwalls.xyz
housekuba.org highstairs-a.akamaihd.net
hoverboard360.at home.myplaycity.com
iamsport.org homeart.space
ilmen.net homedecorpicture.us
ilovevitaly.com homemature.net
imediadesk.com horoshieokna.com
imicrovision.com hotsocialz.com
iminent.com hplaserjetpdriver8y.pen.io
infazavr.ru humbmosquina.tk
i-hobot.ru
iframe-toloka.com
imgarcade.com
in-tandem.co
inbabes.sexushost.com
incep.imagine-ex.co
inclk.com
infowarcraft.ru infowarcraft.ru
int.search.tb.ask.com integritylandscapeservices.com
iopeninghours.co.uk interferencer.ru
iqoption-bin.com investyb.com
iskalko.ru islamtoday.co.za
ispaniya-costa-blanca.ru jagg.info
iweblist.info japan-bearings.ru
izoll.ru javcoast.com
javxxx18.com javrip.net
jeremyeaton.co jennyfire.ru
jerseychinabizwholesale.com jerseychinabizwholesale.com
jolic2.com jerseychinabizwholesale.us
jumptap.com jerseywholesaleelitestore.com
jwcialislrt.com jillepille.com
kaac.ru job.icivil.ir
kamagragelusa.net joessmogtestonly.com
kchaxton.com joy-penguin.com
jrpmakati.com
kalb.ru
kevblog.top kevblog.top
kiinomaniak.pl kihi.gdn
kinohall.ru kimcurlrvsms.com
kinohit1.ru kinocccp.net
kinomaniatv.pl kinostorm.net
kinopolet.net kit-opt.ru
kladrus.ru klin3952.ru
klejonka.info klosetkitten.com
kollekcioner.ru knigonosha.net
kollesa.ru komputers-best.ru
kountrylife.com kongruan.com
krasivye-devushki.net kw21.org
kskjco.club
kursy-ege.ru
kustanay.kz
laapp.com
laexotic.com
lafriore.ru lafriore.ru
languagecode.com lazymanyoga.com
leadn.pl le-clos-des-alouettes.com
ledpolice.ru lennatin.info
levaquin750mg.blogspot.com lifehacĸer.com
lezbiyanki.net lion.playtap.us
lflash.ru lost-alpha.ru
life-instyle.com lottospring.com
likesuccess.com
linkbolic.com
lion.bolegapakistan.com
liupis.com
livesex-amateure.info
livetsomudvekslingstudent.bloggersdelight.dk
locatellicorretor.com.br
lonely-mature.com
longadventure.com
lordzfilmz.me
louisvuittonoutletstore.net
lrac.info lrac.info
lubetube.com lubetube.com
lyrics.home-task.com m.b00kmarks.com
mactechinfo.info magiadinamibia.blogspot.com
makemoneyonline.com mahnwachen-helfen.info
mamasuper.prom.ua malls.com
manualterap.roleforum.ru mansiondelrio.co
manyresultshub-a.akamaihd.net matrixalchemy.com
marcoislandvacations.net max-eclat.men
maslenka.kz mebelcomplekt.ru
maxxtor.eu medanestesia.ru
mebeldekor.com.ua medi-fitt.hu
medicinacom.ru medicines-choice.com
megaindex.ru medosmotr-ufa.ru
metalonly.info mega-bony2017.pl
metarip.ru members.ghanaweb.com
microstatic.pl mensandals.xyz
milblueprint.com mhi-systems.ru
mindeyegames.com mixed-wrestling.ru
mir-business-24.ru mmoguider.ru
misslike.ru mnimmigrantrights.net
mobioffertrck.com mobplayer.ru
mockupui.com moneytop.ru
moje-recenze.cz moneyzzz.ru
monarhs.info more-letom.ru
monclercoats.xyz moviezbonkerssk.cf
moneymaster.ru mp3ringtone.info
montazhnic.ru mrinsidesales.com
morocco-nomad-excursions.com musicvidz.ru
moscow.online-podarki.com muycerdas.xxx
motherboard.vice.com my-big-family.com
moviemail-online.co.uk my-cash-bot.co
moz.com myseoconsultant.com
mrbojikobi4.biz nalogovyy-kodeks.ru
mrcsa.com.au namecrumilchlet.tk
msk.afora.ru naphukete.ru
musirc.com netfacet.net
mydoctorok.ru nextbackgroundcheck.gq
myftpupload.com
najaden.no
narkologiya-peterburg.ru
nelc.edu.eg
netallergy.ru
netvouz.com
nhl17coinsforps3.gratisblog.biz
nikitsyringedrivelg.pen.io
nina.az nina.az
nippon-bearings.ru oballergiya.ru
njpalletremoval.com
no-fuel.org
noscrapleftbehind.co
notasprensa.info
nottyu.xyz
o-o-8-o-o.ru
obiavo.in
obiavo.kz obiavo.kz
offer.camp obiavo.su
office-windows.ru oconto.ru
online-templatestore.com odesproperty.com
onload.pw onko-24.com
oops-cinema.ru ooo-gotovie.ru
oren-cats.ru openfrost.com
origin-my.ru ororodnik.goodbb.ru
ourtherapy.ru osoznanie-narkotikam.net
outshop.ru ossmalta.com
partner-trustworthy.men ownshop.win
partnersafe.men palvira.com.ua
patterntrader-en.com partnerline.men
pendelprognos.se pc-services.ru
pensplan.com pcimforum.com
pestomou.info pechikamini.ru
petedrummond.com pekori.to
pharmacyincity.com petsblogroll.com
pic-re.blogspot.com photorepair.ru
pigrafix.at phrcialiled.com
piski.top pictures-and-images.net
pizdopletka.club picturesify.com
platinumdeals.gr pinapchik.com
polska-poezja.com pinup-casino1.ru
porngalleries.top pizdeishn.com
pornhubforum.tk pogruztehnik.ru
porno-video-chati.ru popserve.adscpm.net
porno.simple-image.com.ua pornoelita.info
pornoinn.com pornstartits.xblog.in
pornosexrolik.com poshiv-chehol.ru
porodasobak.net potoideas.us
porto.abuilder.net pr-ten.de
pourvous.info prefersurvey.net
privatamateure.com presleycollectibles.com
prombudpostach.com.ua primedice.com
pufip.com privatbank46.ru
pyramidlitho.webs.com productarium.com
qcstrtvt.bloger.index.hr professionaldieselcare.com
prosmibank.ru
pts163.ru
purchasepillsnorx.com
q-moto.ru
qor360.com qor360.com
queerspace.com
quickloanbank.com
qwesa.ru
r-e-f-e-r-e-r.com r-e-f-e-r-e-r.com
rangjued.com randalljhoward.com
rapevideosmovies.com rankingchart.de
razamicroelectronics.com
rcb101.ru rcb101.ru
recipedays.com realmonte.net
remont-ustanovka-tehniki.ru realnye-otzyvy.info
remybutler.fr realtytimes.com
rentehno.ru recipedays.ru
resant.ru recordpage-a.akamaihd.net
rialp.getenjoyment.net remont-mobile-phones.ru
roma-kukareku.livejournal.com responsinator.com
ru-dety.ru restorator-msk.ru
ruscopybook.com ric.info
sa-rewards.co.za rixpix.ru
sack.net rossmark.ru
sanidumps.com royalads.net
sbdl.no rozalli.com
sbt-aqua.ru runtnc.net
scanner-fred.top rus-teh.narod.ru
scenarii-1-sentyabrya.uroki.org.ua russintv.fr
schalke04fc.info s-forum.biz
search-goo.com s.lollypopgaming.com
searchlock.com sadaholding.com
seccioncontrabajo.com saecsa.co
sale-japan.com
saletool.ru
santasgift.ml
scanner-marwin.top
scrapy.org
search.pch.com
seemoreresultshu-a.akamaihd.net
seeresultshub-a.akamaihd.net seeresultshub-a.akamaihd.net
sel-hoz.com sei80.com
semp.net
semxiu.com semxiu.com
senger.atspace.co.uk
seo-smm.kz
seorank.info
service-core.ru service-core.ru
sexxdate.net serviporno.com
sexy-pings.com sex-watch.com
sexyali.com sexcamamateurchat.com
sexystrippe.info sexiporno.net
sexytrend.ru sexpornotales.com
sherlock.se shama-rc.net
shlyahten.ru sigmund-freud.co.uk
sicfor.bcu.cc signoredom.com
similardeals.net simple-share-buttons.com
slomm.ru sinestesia.host.sk
slonechka.ru sixcooler.de
slimcdn.com
smailik.org smailik.org
sms2x2.ru snjack.info
snts.shell-pmr.ru snow.nvr163.com
soietvousmaime.fr soft-program.com
spanking.to softtor.com
sortthemesitesby.com
space4updating.win
spamnuker.com
spb-plitka.ru spb-plitka.ru
special-porn.com spb.ru
spin2016.cf speeddream.xyz
sport7777.net spidtest.space
spinnerco.ca
sports-supplements.us sports-supplements.us
spywarenuker.com spy-app.info
stair.registrydomainservices.com spyfu.com
stal-rulon.ru steelmaster.lv
starwars.wikia.com stopnarco.ru
stmassage.ru store-rx.com
strana-solnca.ru stpolice.com
stroicol.net strag-invest.ru
stroiminsk.com strongsignal-a.akamaihd.net
success-seo.com suhanpacktech.com
suggest-keywords.com sundrugstore.com
supermama.top swsociety.se
supermodni.com.ua szucs.ru
surintech.ac.th teenporn18.net
tabakur77.com
tacbelarus.ru
taqplayer.info
tattomedia.com
td-33.ru
tedxrj.com
telefonsexi.com telefonsexi.com
texbaza.by telefonsexsofort.tk
tfxiq.com theheroes.ru
thegameriders.com thetattoohut.com
thelottosecrets.com timecrimea.ru
thenews-today.info titan-cloud.life
theporndude.com tizanidineandgabapentin.blogspot.com
thetardistimes.ovh tizanidinedruginteractions.blogspot.com
threecolumnblogger.com tizanidinewithdrawal.blogspot.com
tizanidineandhydrocodone.blogspot.com tongkatmadura.info
tizanidineorflexeril.blogspot.com top-karkas.ru
tizanidinerecreationaluse.blogspot.com top10registrycleaners.com
tizanidinewithdrawalsymptoms.blogspot.com traffic2money.com
top250movies.ru trafficmp.com
topanasex.com trahic.ru
topkarkas.com trichizobswiv.agddns.net
tour-line.net triumf-realty.ru
trainoffend.ml trustedmaccleaner.com
trucri.me tuckpointingmasonrysystems.com
truemfilelj.gq tupper-posuda.ru
trustaffs.com turist-strani.ru
trustl.life tversvet.ru
try-rx.com typer.one
twojebook.pl uac.net.au
ukkelberg.no ufa.xrus.org
ugogo.info
ukrobstep.com
unmaroll.ya.ru
unpredictable.ga unpredictable.ga
uptime-alpha.net unrealcommander.com
uroki.net uptime-gamma.net
usbggettwku.ga urll.eu
usdx.us uroffer.link
valueclick.com velen.io
vasileostrovsky-rayon.ru vepad.com
viandpet.com
viddyoze.com
video-camer.com video-camer.com
video-woman.com videokrik.net
videochat.mx vielporno.net
videositename.com vod.com.ua
videotuber.ru voloomoney.com
vip-parfumeria.ru volume-pills.biz
vk-mus.ru vzlom-na-zakaz.com
vltai.com vzube.com
vodkoved.ru w3javascript.com
vtc.pw w7s.ru
vykupavto-krasnodar.ru wallpapersist.com
wallpapersdesk.info waysbetter.cn
watchmygf.net web-analytics.date
webinstantservice.com webadvance.club
webscouter.net
website-analyzer.info website-analyzer.info
webstatsdomain.org website-speed-checker.site
weprik.ru websiteaccountant.de
wholesalejerseysgaa.com while.cheapwebsitehoster.com
windowssearch-exp.com whiteproduct.com
womens-journal.net wholesalejerseyscheapjerseys.us.com
wordpress-crew.net wineration.com
wpsecurity.website wingsoffury2.com
wpthemedetector.co.uk winterclassichockeyjerseys.com
wtsindia.in wnoz.de
www.solartek.ru womama.ru
x-lime.net workle.website
xaijo.com wurr.voila.net
xev.ru www.888.com
x5market.ru
xgftnlrt.bloger.index.hr xgftnlrt.bloger.index.hr
xjlottery.com xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn----btbdvdh4aafrfciljm6k.xn--p1ai xn----9sbebi2bvzr7h.xn--p1ai
xn--80ab4aa2g.xn--p1ai xn----ctbbcjd3dbsehgi.xn--p1ai
xn--80ak6aa92e.com xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--e1afanlbnfckd7c3d.xn--p1ai xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xnxxandxvideos.com xn--80abgj3a5acid6ghs.top
xwatt.ru xn--b1addnj3cah.xn--p1ai
xxxhdvideo.site xn--b1ag5cfn.xn--p1ai
yaoguangdj.com xolodremont.ru
yoopsie.com xxx-treker.ru
yottos.com yaderenergy.ru
youresponsive.com ynymnwbm.bloger.index.hr
yoursite.com youdao.com
yourtemplatefinder.com yun56.co
youthreaders.com zaapplesales.blogspot.com
zagadki.in.ua zatjmuzu.info
zed21.net zero1.it
zhuravlev.info zeroredirect2.com
znakomstva-moskva77.ru zeroredirect6.com
zog.link zirondelli.it
zoomovies.org zs2vm.top
zoompegs.com

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
214jaluwobapef.cf 2015god.org
2ads.co.uk 2728fb936f0.com
34.gs 5000-cotydzien.com
6128786.com 66cpwgln.space
actionnooz.com 7xc4n.com
adimmix.com adcash.com
adman.gr addblueoff.com.ua
adprotect.net admatic.com.tr
adzerg.com adrunnr.com
affiliate-fr.com ads.gold
all4invest.ru adservme.com
adsref.men
adverster.com
agroeconom.kz
all4invest.info
allboard.xobor.de allboard.xobor.de
ameblo.top amazingpic.net
animenime.ru anabolics.shop
anmysite.com anaseracresar.tk
antons-transporte.de animaltoplist.com
appsaurus.com apishops.ru
arclk.net app5.letmacworkfaster.world
arenanews.com.ua articlesdirectoryme.info
asmxsatadriverin.aircus.com atley.eu.pn
atlant-auto.info ausergrubhof.info
autoloans.com aviapanda.ru
ayakino.net avkzarabotok.com
azlex.uz avto-oligarh.ru
backuperwebcam.weebly.com azartniy-bonus.com
balitouroffice.com belinka.com.ua
battleforupdating.site best-businessman.ru
beatifullandscape.co best-way.men
beetpics.pw betterhdporn.com
biznesluxe.ru bigames.online
blog.koreadaily.com bigpenisguide.com
bochemit.com.ua bio-japan.net
boleznikogi.com biznesschat.net
brewdom.ru blagovest-med.ru
brg8.com blue-square.biz
calc-for-credit.ru bluesalt.co
cardul.ru boc.kz
cartierloveringreplica.xyz brandimensions.com
cartierlovestore.com businesn.men
cartierlovestore.xyz businesxxl.com
casinopinup-wins.com c-gainsbourg.com
cb.iphantom.com cartierbraceletsreplica.pw
certifywebsite.win cfcl.co.uk
cheapbarbour.online chatroulette.si
chee-by.biz cherrypointplace.ca
compliance-alexa.xyz chiblackhawks-jerseys.com
citysecurity.nu
cleanersoft.com
clicksor.net
compiko.info
compliance-barak.top compliance-barak.top
compliance-checker.info compliance-nelson.top
constantaservice.net contentsexpress.com
cordstrap.cc cool-wedding.net
coslab.club cougfan.info
counter.yadro.ru csgo4.win
countercrazy.com cymbaltaandtizanidine.blogspot.com
customerpromos-a.akamaihd.net cypernhuset.se
cutt.us dafatiri.com
danschawbel.com dailyrank.net
data-ox.com dinosaurus.site
deniven.1bb.ru dogclothing.store
discountbarbour.online domik-derevne.ru
dolohen.com
domcran.net
dosug-lux.ru dosug-lux.ru
dptaughtme.com downvids.net
drpornogratisx.xxx eduserver.net
e-buyeasy.com eloconcream.blogspot.com
ebonyporn.site eloxal.ru
editors.choice6912650.hulfingtonpost.com essayservicewriting.org
egovaleo.it estelight.ru
elidelcream.weebly.com europages.com.ru
englishdictionaryfree.com event-tracking.com
eonpal.com exonline.info
eorogo.top extremepornos.net
erotikstories.ru fakehandbags.xyz
ezigarettenkaufen1.deviantart.com filefilter.weebly.com
film-one.ru fingerprintjs.com
financepoints.eu
finstroy.net
forum.doctissimo.fr forum.doctissimo.fr
foto-sisek.porngalleries.top free-share-buttons.blogspot.com
fotzen-ficken.com funcrushgames.com
freshberry.com.ua fusoradio.info
funnymama.com futbolkisales.ru
fxgallery.com g33.org
gaypornmovie.net gameprimary.com
gcup.ru generousdeal-a.akamaihd.net
gelezki.com gentamicineyedrops.blogspot.com
girlsfuckdick.com geoads.com
girlspicsa.com getlamborghini.ga
glcomputers.ru gigixo.com
globalsurfari.com gktt.ru
goatse.ru gopro-online.info
gojiberriess.apishops.ru hard-porn.mobi
goodnightjournal.com hawaiielectriclight.com
googst2.ru hccoder.info
goroda-vsego-mira.ru hdpixion.com
gta-top.ru heatpower.ru
hao123.com hentai-manga.porn
hasshe.com hermesreplica.pw
hexpilot.com historichometeam.com
hobild.net hmywwogw.bloger.index.hr
host-protection.com howtowhitenteethfast.xyz
ifirestarter.ru ibb.com.ua
iinstalll-fii1leis.jus0wil.pp.ua ibmdatamanagement.co
iminent.com ido3.com
invivo.hu ie.57883.net
istizanidineacontrolledsubstance.blogspot.com impotentik.com
itronics.ca io9.com
jestr.org iscblog.info
jobmarket.com.ua isoveti.ru
jwcialislrt.com ivearchenceinflu.cf
kerei.ru iwanttodeliver.com
kfon.eu janettabridal.com
kino-fun.ru jikoman.info
kol-energo.ru johnrobertsoninc.com
ks1234.com kalandranis.gr
leadwayau.com karadene.com
lescinq.com katadhin.co
life-instyle.com keki.info
linkbuddies.com kerch.site
lionking-1994.blogspot.com kinky-fetishes.com
locatellicorretor.com.br kiprinform.com
lowephotos.info kometars.xyz
macdamaged.tech lasvegaslockandsafe.com
macotool.com liver-chintai.org
marblestyle.ru lomza.info
marketingvici.com low-format.ru
matpre.top m3gadownload.pl
max-p.men mackeeper-land-672695126.us-east-1.elb.amazonaws.com
mediafresh.online mcnamaratech.com
militarysale.pro medosmotr-ufa.ru
miniature.io megakino.net
minneapoliscopiers.com members.ghanaweb.com
morlat.altervista.org metascephe.com
most-kerch.org microsearch.ru
muz-tracker.net mir-business-24.ru
myhealthcare.com more-letom.ru
mysexpics.ru mural.co
najaden.no musezone.su
nalogovyy-kodeks.ru musirc.com
netcheckcdn.xyz myanyone.net
nordstar.pro mywallpaper.top
nowtorrents.com nastroyke.net
nudejapan.net news-readers.ru
o-o-6-o-o.com next-dentists.tk
obsessionphrases.com nfljerseys.online
offers.bycontext.com nhl17coins.exblog.jp
onclkads.com niki-mlt.ru
onetravelguides.com offer.wpsecurity.website
openmultipleurl.com offtopic.biz
opinionreelle.com ooo-gotovie.ru
orenburg-gsm.ru ostrovtaxi.ru
orion-code-access.net ownshop.cf
outpersonals.com oyster-green.com
pay2me.pl panouri-solare-acoperis.com
pelfind.me parfusale.se
penisvergrotendepillennl.ovh percin.biz.ly
perfectplanned.com picsfair.com
pharmacyincity.com pigrafix.at
phelissota.xyz pl-vouchers.com
pirateday.ru podshipniki-ntn.ru
pl.aasoldes.fr pony-business.com
plastjulgranar.se pornhub-ru.com
play-movie.pl pornoblood.com
pliks.pl privetsochi.ru
pokemongooo.ml pro-poly.ru
popugauka.ru produkto.net
porno-raskazy.ru promodj.com
pornosmola.info prosmibank.ru
portside.cc prpops.com
portside.xyz quality-traffic.com
pp-budpostach.com.ua queerspace.com
preparevideosafesystem4unow.site qwarckoine.com
prod2016.com rebelmouse.com
prostitutki-rostova.org remmling.de
razamicroelectronics.com replica-watch.ru
reklamuss.ru sa-live.com
resant.ru sakhboard.ru
roofers.org.uk sanatorrii.ru
s.lollypopgaming.com sarahmilne.top
sack.net search-error.com
santechnik.jimdo.com serptehnika.ru
scanner-john.top sevendays.com.ua
scanner-walter.top sexblog.pw
search-goo.com sharpchallenge.com
searchaddis.com shitting.pro
searchengineranker.email sientalyric.co
semp.net simpoed.ufop.br
seocdvig.ru siteonomy.com
seojokes.net sky-mine.ru
sethrollins.net slujbauborki.ru
sexiporno.net smartbalanceworld.com
sextracker.be sms2x2.ru
sibtest.ru sonnikforme.ru
silktide.com space-worry.ml
simply.net space4updating.win
siteaero.com spb.afora.ru
skatestick.bid spy-app.info
skladvaz.ru sugarkun.com
slavia.info tabletkinaodchudzanie.com.pl
slavkokacunko.de tackletarts.co
solitaire-game.ru tiens2010.ru
spbchampionat.ru timhost.ru
studentguide.ru tizanidine4mgstreetprice.blogspot.com
studiofmp.com tizanidine4mgtablets.blogspot.com
studiokamyk.com.pl tizanidinepain.blogspot.com
suchenindeutschland.com tizanidinesleepaid.blogspot.com
taqywu51.soup.io tomck.com
tastyfoodideas.com tonerbox.kz
telefonsex-ohne0900.net top250movies.ru
terrafootwear.us topflownews.com
thegioixekhach.com topvidos.ru
tizanidine4mgstreetvalue.blogspot.com touchmods.fr
tizanidineandhydrocodone.blogspot.com toy-shop.top
tizanidineforopiatewithdrawal.blogspot.com tradedeals.biz
tizanidinehydrochloride2mg.blogspot.com try-rx.com
toondinsey.com tuberkulezanet.ru
toyota.7zap.com
trafficjunky.com
tvteleport.ru tvteleport.ru
tx41tclega.ru url-extractor.xyz
unimodemhalfduplefw.pen.io usbggettwku.ga
urlopener.com ustion.ru
viandpet.com varikoz24.com
vids18.site vbabule.net
vikistars.com veerotech.com
vip.51.la vesnatehno.ru
vozbujdenie.com voronezh.xrus.org
vriel.batcave.net
vse-pesni.com
vseuznaem.com
vtcdns.com
webextract.profound.net webextract.profound.net
webpromotion.ae wedding-salon.net
whereverdesperate.gq wsgames.ru
withstandingheartwarming.com www.get-free-traffic-now.com
wstroika.ru www.kabbalah-red-bracelets.com
wwwadultcheck.com xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
x-lime.net xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xlovecam.com xn--80ahvj9e.xn--p1ai
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai xn--lifehacer-1rb.com
xnxx-n.com
xnxx699.com
xolodremont.ru
xxxdatinglocal.us xxxdatinglocal.us
xxxhdvideo.site yaminecraft.ru
yottos.com yellocloud.be
z-master.ru yellowproxy.net
zaim-pod-zalog-krasnodar.ru yuarra.pluto.ro
zakon-ob-obrazovanii.ru
zeroredirect12.com
zeroredirect5.com
zetgie.com.pl
zirondelli.it
zlatnajesen.com
zrizvtrnpale.tk
zrus.org
zscalertwo.net

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.122 0; 10.1.1.191 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3458 ### Version: V4.2022.11.3459
### Updated: Tue Nov 8 10:03:27 UTC 2022 ### Updated: Tue Nov 8 22:01:16 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3458 ### Version: V4.2022.11.3459
### Updated: Tue Nov 8 10:03:27 UTC 2022 ### Updated: Tue Nov 8 22:01:16 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##