vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2717 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-15 22:07:29 +00:00
parent b10952d2d8
commit 4ed1c84abe
47 changed files with 63075 additions and 63065 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2716 #### Version: V4.2021.11.2717
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2716 #### Version: V4.2021.11.2717
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2716 #### Version: V4.2021.11.2717
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7748
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2716 #### Version: V4.2021.11.2717
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7752
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7752
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

7752
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7752
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.0.85 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.0.85 0;
127.0.0.1 0; 127.0.0.1 0;

7752
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.0.85 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.0.85 0;
127.0.0.1 0; 127.0.0.1 0;

7752
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.0.85 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.0.85 0;
127.0.0.1 0; 127.0.0.1 0;

7752
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2716 2717

7752
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

7752
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.6.90 0; 52.191.142.30 0;

7752
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.0.85 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.0.85 0;
127.0.0.1 0; 127.0.0.1 0;

7752
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.0.85 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.0.85 0;
127.0.0.1 0; 127.0.0.1 0;

7752
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.0.85 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.0.85 0;
127.0.0.1 0; 127.0.0.1 0;

7752
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.6.90 0; 52.191.142.30 0;

7752
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.114.6.90 0; 52.191.142.30 0;

7752
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 1; 10.1.0.85 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Aboundexbot InfoTekies
BatchFTP Mojeek
Download Wonder OutfoxBot
FrontPage Page Analyzer
FyberSpider Pi-Monster
PageScorer SalesIntelligent
SEOlyticsCrawler Toata
SeoSiteCheckup Zauba
The Intraformant ZumBot
clark-crawler x09Mozilla

302
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404enemy
360Spider
404checker
80legs
ADmantX ADmantX
AIBOT
ASPSeek ASPSeek
Abonti
Aboundexbot Aboundexbot
AfD-Verbotsverfahren AiHitBot
Alligator Aipbot
AlphaBot
Anarchy
Anarchy99 Anarchy99
Ankit
Apexoo
Aspiegel Aspiegel
Asterias Asterias
Attach AwarioSmartBot
BDFetch BDFetch
BLEXBot BLEXBot
BackDoorBot
BackStreet
BackWeb
Barkrowler Barkrowler
Blackboard Battleztar Bazinga
BlowFish BetaBot
Bolt Bitacle
Brandprotect Black Hole
Buck BlackWidow
BotALot
Brandwatch
BuiltBotTough
BuiltWith BuiltWith
BunnySlippers
CATExplorador CATExplorador
CODE87 CSHttp
Calculon Calculon
CazoodleBot Cegbfeieh
CensysInspect CensysInspect
CheTeam CheTeam
Chlooe ChinaClaw
Cliqzbot Claritybot
Cloud mapping Cloud mapping
CopyRightCheck Cogentbot
Copyscape Crawling at Home Project
Crescent
Curious Curious
Custo
DBLBot
DSearch DSearch
DTS Agent DatabaseDriverMysqli
Demon Demon
Digincore Deusu
Discobot Dirbuster
DnBCrawler-Analytics DittoSpyder
DnyzBot
DomainAppender
DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Download Wonder Download Wonder
Drip Drip
EMail Siphon ECCP/1.0
EasyDL EasyDL
Ebingbong ExtLinksBot
Ecxi Ezooms
EroCrawler FDM
Exabot
Express WebPictures
Extreme Picture Finder
FemtosearchBot FemtosearchBot
Freeuploader Fimap
Fyrebot Firefox/7.0
GermCrawler FlashGet
GetRight GT::WWW
Gigabot Genieo
Gotit Grabber
GridBot Grafula
HEADMasterSEO GrapeshotCrawler
HTTP::Lite HTTrack
Harvest
Heritrix
Hloader
HybridBot HybridBot
IDBTE4M IDBTE4M
IDBot IDBot
Iblog Iblog
Id-search Id-search
IlseBot IlseBot
IndeedBot Image Fetch
Indy Library Intelliseek
Internet Ninja
InternetSeer InternetSeer
Iskanie
JamesBOT
JennyBot JennyBot
JetCar
Jetty Jetty
JikeSpider
Joomla
Jorgee Jorgee
JustView JustView
Jyxobot
Keyword Density Keyword Density
LWP::Simple LWP::Simple
Leap
LeechFTP LeechFTP
LexiBot Lftp
LibWeb LinqiaMetadataDownloaderBot
Libwhisker
LinkextractorPro
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey
Ltx71 Lipperhey Spider
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIDown tool
MJ12bot
MSFrontPage
MTRobot MTRobot
Mail.RU_Bot Mag-Net
Majestic12 Magnet
Majestic-SEO
MarkWatch
Masscan
Mata Hari Mata Hari
MauiBot MeanPath Bot
Mb2345Browser
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri
Microsoft Data Access
Microsoft URL Control
Moblie Safari Moblie Safari
Mojeek
Morfeus Fucking Scanner Morfeus Fucking Scanner
Msrabot Msrabot
NPbot Name Intelligence
Navroad
NearSite
Needle
Net Vampire Net Vampire
NetAnts
NetLyzer NetLyzer
Nettrack
Nibbler
Niki-bot
Nikto Nikto
Ninja NimbleCrawler
Nuclei
Offline Explorer
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openfind Openfind
Openvas
OrangeBot OrangeBot
OutclicksBot Page Analyzer
OutfoxBot PageGrabber
PageAnalyzer Pandalytics
PageScorer PeoplePal
Pi-Monster Petalbot
Pimonster Picscout
PleaseCrawl Piepmatz
Pockey Pixray
ProWebWalker Psbot
Probethenet
Pump
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActive RankActive
RankActiveLinkBot
RankingBot RankingBot
Re-re Rankivabot
Ripper RankurBot
Rogerbot Recorder
SEOkicks-Robot SBIder
SEOprofiler SEOkicks
SEOstats SEOlyticsCrawler
SMTBot SMTBot
Scanbot ScanAlert
ScoutJet
Screaming
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
SeobilityBot SemanticJuice
Semrush
SentiBot
Seomoz Seomoz
Siphon Shodan
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer Sitebeam
Siteimprove Siteimprove
Snake SlySearch
SpankBot Snapbot
Snoopy
SocialRankIOBot
Sociscraper
Sogou web spider
Sottopop
Spammen
Spanner Spanner
Stripper Spinn3r
Steeler
Sucker Sucker
Surfbot SuperHTTP
SurveyBot Swiftbot
T8Abot
Teleport
TeleportPro
Telesphoreo Telesphoreo
TheNomad
TightTwatBot TightTwatBot
Toweyabot Toweyabot
Tracemyfile
Trendiction Trendiction
Trendictionbot Trendictionbot
True_Robot
Turingos Turingos
Turnitin
TurnitinBot
Twice Twice
Upflow Typhoeus
VeriCiteCrawler URLy.Warning
Vacuum
VidibleScraper
Virusdie
VoidEYE
Voil Voil
Voltron Voltron
WASALive-Bot
WBSearchBot WBSearchBot
WWW-Mechanize WISENutbot
WWW::Mechanize WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
WeSEE WeSEE
Web Auto Web Enhancer
Web Pix Web Fetch
Web Sucker
WebAuto
WebBandit WebBandit
WebCopier WebEnhancer
WebFetch
WebFuck
WebGo IS
WebImageCollector
WebLeacher WebLeacher
WebPix WebPix
WebReaper WebReaper
WebSauger WebSauger
WebStripper WebmasterWorldForumBot
WebSucker
WebWhacker
WebZIP
Webalta
Webshag Webshag
WebsiteExtractor WebsiteQuester
Webster Whack
Whatweb
Who.is Bot Who.is Bot
WiseGuys Robot
Wprecon
Xaldon WebSpider
Xaldon_WebSpider
Xenu
YoudaoBot YoudaoBot
Zade
Zeus
Zitebot Zitebot
ZoomBot ZmEu
adscanner archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt backlink-check
check1.exe
clark-crawler clark-crawler
cognitiveseo cognitiveseo
com.plumanalytics com.plumanalytics
crawler4j dataforseo.com
demandbase-bot evc-batch
eCatch
facebookscraper
heritrix
isitwp.com
magpie-crawler magpie-crawler
mediawords muhstik-scan
netEstate NE Crawler netEstate NE Crawler
pcBrowser pcBrowser
plumanalytics polaris version
probe-image-size
s1z.ru
satoristudio.net satoristudio.net
scalaj-http seobility
scan.lol
seocompany.store seocompany.store
seostar sexsearcher
sitechecker.pro
siteripz
sogouspider
spyfu spyfu
trendiction.de sysscan
tAkeOut
ubermetrics-technologies.com ubermetrics-technologies.com
webmeup-crawler voyagerx.com
webpros.com
x22Mozilla x22Mozilla
xpymep1.exe zauba.io
zgrab

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy 01h4x.com
ASPSeek 80legs
Aboundexbot ADmantX
Acunetix AIBOT
AhrefsBot AhrefsBot
Aipbot AiHitBot
Alexibot Alexibot
Alligator
AlphaBot
Anarchie Anarchie
Anthill Anarchy
Apexoo Anarchy99
AwarioRssBot Asterias
Attach
BLEXBot
BackDoorBot BackDoorBot
BackWeb BackStreet
Bandit Badass
Battleztar Bazinga Barkrowler
BetaBot BatchFTP
Black Hole Blow
BlackWidow
BlowFish BlowFish
Bolt Boardreader
BuiltBotTough Brandwatch
BuiltWith BuiltWith
CATExplorador BuzzSumo
CODE87 CCBot
CazoodleBot CazoodleBot
CherryPicker CherryPicker
ChinaClaw
Chlooe
Claritybot Claritybot
Cliqzbot
Cloud mapping
Cocolyzebot
Cogentbot Cogentbot
Crawling at Home Project Copyscape
CrunchBot Cosmos
DTS Agent CyotekWebCopy
DBLBot
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Demon Demon
Deusu Devil
DigitalPebble Dirbuster
Disco Disco
Discobot
Discoverybot
Dispatch Dispatch
DomCopBot DnBCrawler-Analytics
DomainAppender DomainAppender
DomainStatsBot DomainCrawler
Domains Project Dotbot
Dragonfly EirGrabber
EMail Siphon ExtLinksBot
Ecxi
EroCrawler
Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder
EyeNetIE EyeNetIE
FHscan FDM
FlashGet Fimap
Foobot Foobot
Freeuploader
FyberSpider FyberSpider
G-i-g-a-b-o-t Fyrebot
GT::WWW
GalaxyBot
Genieo
GermCrawler GermCrawler
GetRight
GoZilla
Gotit Gotit
GrabNet
Grabber
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO
HMView HMView
HTMLparser HTTP::Lite
Harvest
Havij Havij
Heritrix Heritrix
Humanlinks Humanlinks
HybridBot IDBTE4M
IDBot Id-search
Iblog Image Fetch
Image Sucker
IndeedBot IndeedBot
Indy Library
InfoTekies InfoTekies
Intelliseek Intelliseek
InterGET
Internet Ninja Internet Ninja
Iskanie InternetSeer
JOC Web Spider Iria
JamesBOT JennyBot
Jbrofuzz
JetCar JetCar
Jetty JikeSpider
JustView
Jyxobot
Keyword Density Keyword Density
Kinza Kozmosbot
LNSpiderguy LeechFTP
Lanshanbot
LeechGet
LibWeb
Libwhisker Libwhisker
LieBaoFast LieBaoFast
LinkScan LinkScan
LinkWalker LinkpadBot
Lipperhey Spider Lmspider
Ltx71
MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc MIIxpc
MJ12bot MQQBrowser
MSIECrawler Magnet
MTRobot Majestic12
Mag-Net
Majestic SEO
Majestic-SEO
MarkMonitor MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mass Downloader
Meanpathbot Masscan
MegaIndex.ru MauiBot
Microsoft URL Control Mb2345Browser
Mister PiX Microsoft Data Access
Mojeek Mojeek
Musobot Morfeus Fucking Scanner
NICErsPRO NICErsPRO
NPbot
Name Intelligence Name Intelligence
Nameprotect
Navroad Navroad
NearSite
NetAnts NetAnts
NetLyzer NetLyzer
Netvibes NetMechanic
NetSpider
NetZIP
Netcraft
Nettrack
NextGenSearchBot NextGenSearchBot
Niki-bot Nimbostratus
Nikto
Ninja
Nmap
Nuclei
Octopus Octopus
Offline Explorer OnCrawl
Offline Navigator
OpenVAS OpenVAS
Openfind
Openvas Openvas
OrangeSpider OutclicksBot
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PECL::HTTP
Page Analyzer Page Analyzer
PageGrabber PageGrabber
PageScorer
PageThing.com PageThing.com
Pandalytics PleaseCrawl
Papa Foto Quick-Crawler
PictureFinder
Piepmatz
Pimonster
ProPowerBot
Psbot
RSSingBot RSSingBot
RankActiveLinkBot
RankFlex RankFlex
Rankivabot
RankurBot
Re-re
ReGet
Recorder Recorder
RedesScrapy RepoMonkey
Ripper Ripper
Rogerbot
SBIder SBIder
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOstats SEOprofiler
SISTRIX SISTRIX
Scanbot SMTBot
ScoutJet
ScreenerBot ScreenerBot
ScrepyBot ScrepyBot
Searchestate Searchestate
SearchmetricsBot SemanticJuice
SemrushBot Semrush
SeoSiteCheckup SentiBot
SeobilityBot
Shodan Shodan
SiteCheckerBotCrawler Siphon
SiteExplorer
SiteSnagger SiteSnagger
Sitebeam
Siteimprove Siteimprove
SlySearch Sitevigil
SmartDownload
Sosospider Sosospider
SpaceBison Sottopop
Spammen
SpankBot SpankBot
Spanner
Spinn3r
SputnikBot
Sqlmap Sqlmap
Sqlworm Sqlworm
Sqworm Sqworm
Stripper Sucker
SuperHTTP SuperHTTP
Surfbot Swiftbot
SurveyBot Teleport
Suzuran The Intraformant
T0PHackTeam Titan
T8Abot Toweyabot
Telesphoreo Tracemyfile
Telesphorep
Thumbor
Toata
Trendiction
Trendictionbot Trendictionbot
Turnitin Turnitin
TurnitinBot URLy Warning
Twice URLy.Warning
Typhoeus
UnisterBot UnisterBot
VB Project Upflow
VCI Vacuum
VelenPublicWebCrawler Vagabondo
VeriCiteCrawler VeriCiteCrawler
Virusdie Virusdie
VoidEYE
Voltron
WASALive-Bot WASALive-Bot
WISENutbot WISENutbot
WPScan Wallpapers
WWW::Mechanize Wallpapers/3.0
Web Collage Web Auto
Web Enhancer Web Fuck
Web Fetch Web Sauger
WebBandit WebAuto
WebCopier WebFetch
WebEnhancer WebGo IS
WebFuck WebPix
WebImageCollector WebSauger
Webshag WebWhacker
WebZIP
WebmasterWorldForumBot
WebsiteExtractor WebsiteExtractor
Whatweb WebsiteQuester
Whacker
Who.is Bot Who.is Bot
WiseGuys Robot WinHTTrack
Wotbox Wonderbot
Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zitebot
Zermelo ZumBot
ZoomBot ZyBorg
ZoominfoBot
adscanner adscanner
backlink-check autoemailspider
clark-crawler clark-crawler
cognitiveseo cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com crawl.sogou.com
domainsproject.org crawler4j
eCatch demandbase-bot
isitwp.com gopher
ips-agent
meanpathbot meanpathbot
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
probe-image-size
s1z.ru s1z.ru
seobility scan.lol
seocompany.store
seostar seostar
sogouspider serpstatbot
sitechecker.pro
spyfu spyfu
trendiction.com sysscan
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webmeup-crawler
webpros.com
webprosbot
x22Mozilla x22Mozilla
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
a96527gi.beget.tech advokat-grodno.by
adviceforum.info analytics-ads.xyz
frivgame250.com hostnow.men
ge0ip.com ledis.top
pornmania.pl mistr-x.org
salut-camp.ru pizdopletka.club
statistici.ro running-line.ru
wildcattube.com sitiz.club
www.wohnkabinen-shop.de strigkaomsk.ru
yaminecraft.ru zhongwenlink.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101flag.ru 007angels.com
18ps.ru 2000k.ru
3-letter-domains.net 28n2gl3wfyb0.ru
3rbseyes.com 45en.ru
4pp13.com 4kepics.com
5i2.net 8si.ru
70casino.online aa625d84f1587749c1ab011d6f269f7d64.com
7xc4n.com aadroid.net
91abcw.com aanapa.ru
999webdesign.com abclauncher.com
abbanreddy.com adclickthru.net
abcdeg.xyz adult3dgames.com
abusora.com advancedcleaner.com
admeasures.com alevinefesleri.com
adrenalin-stk.ru aliexpresscashback.club
ads-seo.men alltheviews.com
adtech.fr aloofly.com
adult-shop.com.ua amigobulls.com
advancedsoftwaresupport.com amt-k.ru
ahmedabadwebs.com animalrank.com
ahrntal.verymes.xyz anjalika.co.in
airmaxshoes-2016.com apisecretsaucebi-a.akamaihd.net
akita.kz app-ready.xyz
aksonural.ru apple.com-cleaner.systems
aktivator-windows10.blogspot.com
alekseevec.ru
alf-img.com
alive-ua.com
allboard.xobor.de
allfinweb.com
allnews.md
alpinism.ru
altamayoztourism.com
alyeskaresort.com
ameblo.jp
anlimebel.kiev.ua
anonymousfox.co
apccargo.com
apibrowsesmartne-a.akamaihd.net
apidigidocketnet-a.akamaihd.net
apishops.ru
apispringsmartne-a.akamaihd.net
appiq.mobi
apps-analytics.net
appsaurus.com
aramaicmedia.org
arate.ru
architecturebest.com
aruplighting.com
as5000.com as5000.com
asophoto.com asophoto.com
asscenihotosidea.blogspot.co.za asseenontv.ru
atdedinotuho.tk astrochicks.com
autogrep.ru atleticpharm.org
avon-novosib.ru atout-energie-69.com
avtointeres.ru autoua.top
avtolombard-krasnodar.ru avek.ru
azartclub.org avramstroy.ru
azlex.uz avtoarenda.by
b2b-lounge.com babosas.com
backgroundpictures.net babs.com.ua
backuperwebcam.weebly.com baersaratov.ru
baditri.com
baksman.com baksman.com
bannerpower.com balitouroffice.com
bannerspace.com
bavsac.com
beachpics.com beachpics.com
beetpics.pw belstaffstore.online
benchmarkcommunications.co.uk beremenyashka.com
bestadbid.com bestcalovebracelet.cn
bestmouthwash.club bestdooz.com
besttorrentknifta.weebly.com bestpornuha.com
bestwebsiteawards.com betonka.pro
betteroffers.review betune.onlinewebshop.net
bif-ru.info betwinservice.com
bigames.online big-boards.info
biglistofwebsites.com bigpenisguide.com
bitcoinremote.com bioscorp.ru
bitcoins-live.ru blackle.com
bloggerads.net blagovest-med.ru
blogporn.in blavia.00author.com
bonky.biz bluesalt.co
book-bets.com boleznikogi.com
bonvillan.com
boobsimge.com
bookmark4you.com.biz
braip.com.br braip.com.br
brimstonehillfortress.org brendbutik.ru
bristolhotel.com
brk-rti.ru brk-rti.ru
builtwith.com brus.city
buketeg.xyz buigas.00it.com
bum.com.ru bukleteg.xyz
buqyxa.rincian.info buqayy0.livejournal.com
burkesales.com burger-tycoon.com
buytizanidineonline.blogspot.com bus-offer.com
buzzurl.jp buy-cheap-pills-order-online.com
bvlgariring.xyz buyfriend.ru
bwlx.prepedu.cn buyscabiescream.com
byme.se buzzonclick.com
cacheimages.com bvlgariwallet.xyz
callawaygolfstore.online bycontext.com
candy-glam-hp.com cactussoft.biz
carapics.com capsderfudd.tk
cartujano-pre.de cars-modification.net
casite-513049.cloudaccess.net cartierwatch.xyz
cdn.walkme.com ccjp.eu
cdnnetwok.xyz cdncash.org
chatroulette.online cheapestjerseysonwholesale.com
chatroulette.si cheapjerseysa.com
chatroulette.video chinaelitecheapjerseys.com
cheapwebsitehoster.com chocolateslim-u-srbiji.com
cialisovercounteratwalmartusa.com cinemaflix.website
citetick.com
citizenclsdriveri7.pen.io
cityadspix.com cityadspix.com
clean-start.net clayvasedesigns.tk
cleanallvirus.com clubfashionista.com
clickcash.com co.lumb.co
clips.ua.ac.be compliance-elena.top
club-musics.ru compliance-julia.top
coldfilm.ru compliance-willy.top
comeondog.info connectingsingles.com
compliance-barak.top copenergo.ru
compliance-fred.top
compliance-mary.top
constantaservice.net
coolingoods.com
creams.makeforum.eu creams.makeforum.eu
credit-online.ws creditnation.ru
crest3d.ru cupday.com
custodieva.ru custom-electric-guitar.com
dairyindia.in customboxes4less.com
dandiyabeats.in cypernhuset.se
daooda.com dailyfinancefix.com
darodar.com dailystrength.org
dating-app.ru dawlenie.com
ddsoldes.fr dealighted.com
decorationspcs.com design-ideas.info
demenageur.com destinationrealestate.com
detki-opt.ru dimaka.info
dms-sw.ru dinosaurus.site
dnepr-avtospar.com.ua dividendo.ru
domashneeruporno.com djstools.com
drugs-no-rx.info dktr.ru
dvrlists.com domaineaneblanc.com
domination.ml
domini.cat
donvito.unas.cz
downloaddy.net
doxyporno.com
drupa.com
dverimegapolis.ru
dwomlink.info
eandsgallery.com
easycommerce.cf
easytuningshop.ru easytuningshop.ru
educontest.net
efkt.jp
efnor-ac.com
ege-essay.ru ege-essay.ru
eleimgo.pw ekn-art.se
elektir.ru elektrozigaretten2.yn.lt
elitedollars.com elexies.info
eluxer.net elitepcgames.com
erotikchat-24.com eloconcream.blogspot.com
estibot.com en.home-task.com
euronis-free.com energy-ua.com
executehosting.com epngo.bz
exoneration-project.us european-torches.ru
extremepornos.net exdocsfiles.com
face.hostingx.eu exhibitionplus.eu
familyphysician.ru exlarseva.webblog.es
fast-wordpress-start.com exportshop.us
feeriaclub.ru extractorandburner.com
fashionavenuegame.com
felizporno.com felizporno.com
fenoyl.batcave.net
fermersovet.ru fermersovet.ru
ferrotodo.com fifa-coins.online
fetishinside.com filefilter.weebly.com
filerockstar298.weebly.com filesdatabase.net
filmbokep69.com filmania-x.ru
filmidivx.com filmgo.ru
focalink.com finansov.info
formularz-konkurs.tk fitnesspiks.com
fortwosmartcar.pw floridamhca.org
forsex.info
forum.doctissimo.fr
fotosfotos.eu
foxjuegos.com foxjuegos.com
foxtechfpv.com frankofficial.ru
freshmac.space freelifetimefuckbook.com
freshsuperbloop.com freshberry.com.ua
freza-sverlo.ru fusoradio.info
frivgame250.com galeon.com
g7m.pl geilehausfrauen.net
gallerylisting.com
game-top.su
gameonasia.com
gaypornmovie.net
geile-lelly.eu
gemgrab-a.akamaihd.net
get-free-social-traffic.com
getdot.ru getdot.ru
getridofstretchmarks.org gheus.altervista.org
getyourimage.club
gifspics.com
gigapeta.com gigapeta.com
gigixo.com girlporn.ru
given2.com gk-atlant.info
glastecfilms.com.my glavprofit.ru
goodhousekeeping.com goforexvps.com
goldandcard.ru
googst2.ru
gosreg.amchs.ru gosreg.amchs.ru
gotwebsite1.com
greenidesign.co
gsasearchengineranker.space gsasearchengineranker.space
gwebtools.com.br gsmtlf.ru
handicapvantoday.com guarrasdelporno.xxx
hdimagelib.com gungamesz.com
helvetia.com.ua hackers-crackers.tk
hentaiheroes.com handicapbathtubarea.com
heygidday.biz hdapp1008-a.akamaihd.net
hidefiles.org heroz.fr
hotel-mkad.ru highspeed5.net
houseofrose.com hmmm.cz
hoverboard360.at hobild.net
hoverboard360.se hornymatches.com
hoverboardforsaledirect.com hotgirlhdwallpaper.com
hoztorg-opt.ru hothot.ru
ie.57883.net hully.altervista.org
iframe-toloka.com husky-shop.cz
igrovyeavtomaty777.ru hustoon.over-blog.com
iinstalll-fii1leis.jus0wil.pp.ua idealtits.net
iklysha.ml igithab.com
ilovevitaly.ru ilovevitaly.xyz
images.gyffu.com imagefinder.site
importchinacoach-teach.com immobiliaremassaro.com
inboundlinks.win impisr.ru
infobanks.ru impresagaia.it
infoupdate.org increasewwwtraffic.info
infowarcraft.ru informatiecentro.be
investingclub.ru innodgfdriverhm.aircus.com
iqoption.com inome.com.ua
iqs.biz.ua instabid.tech
istizanidinelikexanax.blogspot.com intellektmedia.at
itronics.ca iplogger.org
ivearchenceinflu.cf iptool.xyz
iwanttodeliver.com istizanidineanarcoticdrug.blogspot.com
jagg.info itbc.kiev.ua
jav-fetish.site ivanovo.zrus.org
jaxcube.info jackpotchances.com
jerseychinabizwholesale.us japfm.com
johnnyhaley.top jobmarket.com.ua
joyceblog.top jongose.ninja
kadashihotel.com jpcycles.com
kanimage.com kaliningrad.zrus.org
kazka.ru katadhin.co
kazrent.com kimcurlrvsms.com
keywesthideaways.co kino-filmi.com
kinky-fetishes.com
kinoflux.net kinoflux.net
klejonka.info kiwe-analytics.com
klumba55.ru kletkimehan.ru
knowsitall.info koopilka.com
koral.se krafte.ru
kredit-pod-zalog-krasnodar.ru
ku6.com ku6.com
lancheck.net kustanay.kz
law-check-eight.xyz kuzinsp.ru
legionalpha.com labplus.ru
lacave.ntic.fr
ladsup.com
larger.io
laserpen.club
lechtaczka.net
lemon-ade.site
letolove.ru letolove.ru
lidoradio.com life.biz.ua
likrot.com limads.men
littlesexdolls.com linerdrilling.com
locatellicorretor.com.br linkbux.ru
locksmith.jp linkdebrideur.xyz
lol-smurfs.com ljusihus.se
lovasszovetseg.hu llastbuy.ru
mabdoola.blogspot.com lombia.com
macdamaged.tech longgreen.info
makedo.ru louboutinbooties.xyz
louboutinreplica.xyz
love-baby.cz
lsp-awak-perikanan.com
machicon-ueno.info
macrotek.ru
magiadinamibia.blogspot.com
mamasuper.prom.ua mamasuper.prom.ua
masserect.com marketingtechniques.info
master-muznachas.ru matsdale.com
mastershef.club medispainstitute.com.au
max-p.men medkletki.ru
medicines-choice.com medmajor.ru
medtherapy.ru meduza-consult.ru
mega-bony-2017.pl meendo-free-traffic.ga
menetie.ru megaindex.ru
menstennisforums.com megavolt.net.ua
michaelkorsoutletstore.net message-warning.net
microsearch.ru microsearch.ru
mindeyegames.com mil-stak.com
miniature.io mirtorrent.net
misandesign.se mirzonru.net
mnogabukaff.net missis.top
mobile-appster.ru mobot.site
moetomnenie.com monclerboots.xyz
moi-glazki.ru monclercoats.xyz
monclercheap.xyz moncleroutletonline.win
moncleroutletonline.xyz mrvideospornogratis.xxx
monetizationking.net muz-shoes.ru
moneymaster.ru myhealthcare.com
moneyviking-a.akamaihd.net nalogovyy-kodeks.ru
morefastermac.trade
moxo.com
muschisexbilder.com
mvpicton.co.uk
myanyone.net
mybinaryoptionsrobot.com
mydoctorok.ru
myiptest.com
mzdish.site
narosty.com
nash-krym.info nash-krym.info
nastroyke.net nelc.edu.eg
neobux-bg.info next-dentists.tk
newstaffadsshop.club
nfvsz.com
ngps1.ru
nightvision746.weebly.com nightvision746.weebly.com
nikitabuch.com noumeda.com
niroo.info ofanda.com
njkmznnb.ru offergroup.info
normalegal.ru omsk.xrus.org
notaria-desalas.com onlyporno.ru
novosti-hi-tech.ru oops-cinema.ru
nsatc.net ourville.info
nullrefer.com outrageousdeal-a.akamaihd.net
nw-servis.ru owohho.com
nzfilecloud.weebly.com page2rss.com
okonich.com.ua panicatack.com
onefilms.net parlament.biz
onlinemeetingnow.com partner-trustworthy.men
onlinewritingjobs17.blogspot.ru pateaswing.com
orion-v.com paydayloanslocal.com
osb.se11.ru pcgroup.com.uy
ostroike.org perm.xrus.org
palocco.it perso.wanadoo.es
pewit.pw petsblogroll.com
photorepair.ru photorepair.ru
picscout.com phuketscreen.com
pillscheap24h.com picswe.com
pizza-imperia.com pirateday.ru
pkr1hand.com plastgran.com
pl-vouchers.com pokerniydom.ru
plastgranar.nu porn-w.org
play-mp3.com porngalleries.top
playfortuna-play.ru porno-raskazy.ru
podshipniki-nsk.ru portadd.men
popmarker.com portnoff.od.ua
popunder.net postclass.com
pornoted.com priora-2.com
pornoxxx.com.mx privetsochi.ru
portside.cc privhosting.com
pricheski-video.com produkto.net
printingpeach.com proekt-gaz.ru
privatbank46.ru profolan.pl
privatefx-in.ru progonrumarket.ru
professionalwritingservices15.blogspot.ru promoheads.com
pssucai.info proprostatit.com
psvita.ru prosperent.com
push-ad.com prostitutki-tolyatti.org
psa48.ru
psoriasis-file.trade
purchasepillsnorx.com
puzzleweb.ru
quickloanbank.com quickloanbank.com
rabot.host.sk rank-checker.online
rankia.com rankia.com
realting-moscow.ru ranksonic.info
registratciya-v-moskve.ru real-time-analytics.com
relatodelpresente.com.ar remont-komputerov-notebook.ru
renhacklids.tk remontgruzovik.ru
rennlist.com rennlist.com
reward-survey.net resant.ru
rialp.getenjoyment.net riralmolamsaca.tk
rockma.se risparmiocasa.bz.it
rogervivierforsale.com rostov.xrus.org
rucrypt.com rukino.org
rusexy.xyz ruscopybook.com
russia-tao.ru russia-today-video.ru
sabizonline.com saitevpatorii.com
sack.net salmonfishingsacramentoriver.com
samoiedo.it samoiedo.it
saneitconsulting.com sandhillsonline.com
sarm.tk
saveriopiazza.it saveriopiazza.it
sbdl.no scalerite.co.za
sbetodiodnye-lampy.ru scanner-elena.top
scanner-andrew.top scanner-jane.top
scanner-marwin.top scansafe.net
school-diplomat.ru scanspyware.net
screentoolkit.com scrnet.biz.ua
searchencrypt.com sdelai-prosto.ru
security60-e.com sdelatmebel.ru
seeresultshub-a.akamaihd.net search-error.com
selectads.men searchaddis.com
seorankinglinks.com seasonvar.ru
sergiorossistore.online semprofile.com
servisural.ru seorankinglinks.xyz
sevendays.com.ua sexobzor.info
sexpornotales.net
sextracker.de sextracker.de
sexyteens.hol.es sexy-pings.com
shama-rc.net sexystrippe.info
sharebutton.net sfj-ror.no
shemalegalls.blogporn.in
shikiso.info
shivafurnishings.com
shlyahten.ru shlyahten.ru
shopsellcardsdumps.com simpoed.ufop.br
si-unique.com sinestesia.host.sk
silvercash.com sirpornogratis.xxx
simon3.ru sitechecker.pro
siteexpress.co.il
sitevaluation.org
sitopreferito.it
skanninge.se
skladvaz.ru skladvaz.ru
slavic-magic.ru slomm.ru
slkrm.ru
sluganarodu.ru sluganarodu.ru
smart-balancewheel.com slujbauborki.ru
sms2x2.ru smartshoppymac.com
snip.tw snip.to
soc-econom-problems.ru social-search.me
social-button.xyz socialmonkee.com
social-s-ggg.xyz soietvousmaime.fr
soheavyblog.com solitaire-game.ru
songoo.wz.cz soserfis.com
soundfrost.org soviet-portal.do.am
spabali.org space-worry.ml
spammen.de speechfoodie.com
spinnerco.ca spywarespy.com
sports-supplements.us
srecorder.com srecorder.com
srgwebmail.nl staynplay.net
star61.de sterva.cc
stuff-about-money.com sticken.co
sugarlyflex.pw stockquotes.wooeb.com
supermama.top stockspmb.info
superstarfloraluk.com summerlinhomes411.info
surffoundation.nl superiends.org
svbur.ru svensk-poesi.com
swimpool.ca svtrd.com
szamponrevita.pl sygraem.com
tedxrj.com tech4master.com
thegioixekhach.com teletype.in
themeforest.net the-usa-games.blogspot.com
thewebsitetemplate.info theautoprofit.ml
tilido.com thecoolimages.net
tizanidinecost.blogspot.com thedownloadfreeonlinegames.blogspot.com
tizanidinefibromyalgia.blogspot.com thefotosgratis.eu
tizanidineusedfor.blogspot.com thegolfclub.info
top10-online-games.com themestotal.com
torrentgamer.net thenews-today.info
torrentred.games thepokertimer.com
traffic2money.com timeallnews.ru
traffpartners.com tizanidineandhydrocodone.blogspot.com
truck-addzilla.life tizanidinehcl4mgtab.blogspot.com
ttrraacckkrr.com tizanidinehydrochloride2mg.blogspot.com
turn-up-life.life tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidineingredients.blogspot.com
tizanidinepills.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com
tizanidinestreetvalue.blogspot.com
tnaionline.org
toloka.hurtom.com
toolsky.com
toon-families.com
top250movies.ru
topdownloads.ru
torrnada.ru
totu.us
traderzplanet.in
trafficmania.com
trichizobswiv.agddns.net
trumpetedextremes.com
tube8.com
turizm.bz
tvnewsclips.info tvnewsclips.info
twojebook.pl typimga.pw
tx41tclega.ru u555u.info
ua.tc ua.tc
unlimitdocs.net ufa.xrus.org
unrealcommander.biz uggbootsoutletsale.us
unrealcommander.com ugguk.online
upupa.net unacittaconte.org
urldelivery.com uptimechecker.com
urll.eu usiad.net
us-america.ru vabasa.inwtrade.com
vacuumcleanerguru.com vapomnoncri.tk
varikozdok.ru vchulkah.net
vbikse.com verymes.xyz
velobikestock.com
vertaform.com
viagengrarx.com
viandpet.com
videogamesecrets.com videogamesecrets.com
videojam.tv vkak.ru
vikistars.com
vip-file.com
virus-schutzmasken.de
vitanail.ru
viteonlusarezzo.it
vkonche.com vkonche.com
vksex.ru vmnmvzsmn.over-blog.com
vladimir.xrus.org voditeltrezviy.ru
vodkoved.ru vvpg.ru
voloslove.ru wapsite.me
vozbujdenie.com wdfdocando.com
vseigru.one webix.biz
vysigy.su webradiology.ru
vzlomsn.org website-analytics.online
warningzscaler.heraeus.com website-speed-check.site
webshoppermac.com website-speed-checker.site
webtherapy.ru welcomeauto.ru
williamrobsonproperty.com wemedinc.com
workle.website westen-z.life
wormix-cheats.ru white-truck.life
wpsecurity.website wholesalecheapjerseysfree.com
writersgroup580.web.fc2.com wholesalejerseychinashop.com
wsgames.ru wieseversa.no
www.get-free-traffic-now.com win-spy.com
x-diesel.org winwotgold.pl
worldinternetauthority.com
wowcasinoonline.ooo
wrona.it
ww1943.ru
www.888.com
www.pinnacle-bets.com
x69ty.ru x69ty.ru
xn----7sbahjd3btneuw1joc.xn--p1ai xmronta.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xn----ctbbcjd3dbsehgi.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--80ab4aa2g.xn--p1ai xn--80ab4aa2g.xn--p1ai
xn--h1ahbi.com.ua xn--80aikhbrhr.net
xn--oogle-wmc.com xtubeporno.net
xxx-cam.webcam xxxguitars.com
xzlive.com yebocasino.co.za
yellowfootprints.com yogamatsexpert.com
youtubologia.it yottos.com
yur-p.ru youdao.com
zapiszto.pl youporn-forum.ga
zarabiaj-dzis.pl youporn-ru.com
zeroredirect7.com your-bearings.com
zeroredirect9.com yourdesires.ru
znakomstva-moskva77.ru ypmuseum.ru
zoogdinsney.com yuweng.info
zscalerone.net zaidia.xhost.ro
zynax.ua zazagames.org
zzlgxh.com zeleznobeton.ru
zoodrawings.com
zs2vm.top
zscaler.net
zverokruh-shop.cz
zvuker.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
101flag.ru 1008.su
101lesbian.xyz 13tabs.com
4ureyesonly.com 1hwy.com
4webmasters.org 1webmaster.ml
5ws.dating-app.ru 2345.com
addbin.men 4kpics.rocks
admatic.com.tr 55wmz.ru
adminshop.com 8kisses.eu
adtiger.tk aa08daf7e13b6345e09e92f771507fa5f4.com
adbetclickin.pink
adcash.com
adman.se
adsref.men
adtech.fr
agardomains.com agardomains.com
agroeconom.kz algerianembassy.co.in
albuteroli.com allforminecraft.ru
all4invest.ru aloofly.com
allboard.xobor.de aludecor.info
ampower.me ameblo.jp
anniemation.com anal-acrobats.com
apibatbrowsecom-a.akamaihd.net animenime.ru
autobrennero.it anjalika.co.in
autolombard-krasnodar.ru appearance-cool.com
automatic-seo.com appfastplay.com
aviav.ru.com artdestshop.eu
avitocars.ru artefakct.com
awaybird.ru asacopaco.tk
backlink4u.net asociatia-tipografilor-transilvania.ro
bannerbank.ru aviav.org
battle.net azlex.uz
beauty-bracelet.com beremenyashka.com
begalka.xyz best-seo-software.xyz
best-deals-products.com
bestchoice.cf bestchoice.cf
bestdooz.com bestkfiledxd.cf
besthomemadepornsites.com bestwrinklecreamnow.com
bloggen.be bfz.biz
bloglag.com bget.ru
blue-square.biz bigcities.org
bmsco.co biznesluxe.ru
bongiornos.info bookhome.info
bonkers.name businesn.men
book-bets.com buyessay3.blogspot.ru
brainzpod.win bwlx.prepedu.cn
btvn.ru cardiosport.com.ua
bumskontakte.org carsdined.org
burger-imperia.com cascadelink.org
burger-tycoon.com cdncash.com
buyfriend.ru cfcl.co.uk
calstaterealty.us cialischstgerts.com
chatroulette.video coinsspb.com
cinemaenergy-hd.ru companies-catalog.com
cleanersoft.com compliance-mark.top
compliance-ivan.top conocer-sanabria.com
compucelunlock.net custodieva.ru
copenergo.ru dailystrength.org
crackzplanet.com davebestdeals.com
customchocolate.business-for-home.com daymusam.com
dealighted.com deda-moroza-zakaz.ru
dengi-pod-zalog-nedvizhimosti.ru deutschehobbyhuren.net
deutschland123.de dev78.dioniqlabb.se
discovertreasurenow.com dildofotzen.net
domini.cat diusyawiga.tk
drivotracker.com divatraffic.com
dynainbox.com djstools.com
earncash.com.ua doeco.ru
easyshoppermac.com donvito.unas.cz
ednorxmed.com dostavka-v-krym.com
efkt.jp downloadme.life
ekaterinburg.xrus.org drhomes.biz
elektrozigarette2.wordpressy.pl dvr.biz.ua
esnm.ru econom.co
elmacho.xyz
enbersoft.com
energydiet-info.ru
erectile.bid
eredijovon.com
ereko.ru
eu-cookie-law.blogspot.com
evehemming.blogspot.com.au evehemming.blogspot.com.au
fantasticpixcool.com extractorandburner.com
fatmaelgarny.com face.hostingx.eu
faz99.com familienzahnaerzte.com
filesclub.net favornews.com
flashbannernow.com felizporno.com
flex4launch.ru filefilter.weebly.com
footballfarrago.com filmfanatic.com
forum.doctissimo.fr frequiry.com
forumprofi.de front.to
forums.d2jsp.org fun2cell.net
free-share-buttons.blogspot.com gamerscorps.com
free-video-tool.com garciniaxt.us
freefoto.ca gearcraft.us
frighteningremain.cf
fungamelands.com
fxtips.ru
gamesprite.me
geileweiber.tk
gentamicineyedrops.blogspot.com gentamicineyedrops.blogspot.com
get-free-social-traffic.com get-your-social-buttons.info
golden-catalog.pro gnuetella.com
goodnightjournal.com googlemare.com
gosreg.amchs.ru goroda-vsego-mira.ru
graphics8.info gratuitxblcodes.com
graphicwe.org growmyfunds.ca
greatidea.marketing gta-club.ru
hawaiisurf.com guardlink.com
hdimagegallery.net handicapvantoday.com
hentai-manga.porn headpharmacy.com
hermesreplica.pw healbio.ru
hidemyass.com highspeed5.net
hoverboard360.de hit-kino.ru
https-legalrc.biz hkladys.com
hulfingtonpost.com hotdl.in
hully.altervista.org hotkeys.com
hustoon.over-blog.com howmuchdoestizanidinecost.blogspot.com
imk.com.ua ifmo.ru
insider.pro infobanks.ru
invitefashion.com itsdp3.com
iptool.xyz ivoiretechnocom.ci
isistaylorporn.info iwantedmoney.com
ispac.org jav-fetish.com
itservicesthatworkforyou.com kanimage.com
javspace.net kicknights.gq
jerseysbizwholesalecheap.com kinobaks.com
jjbabskoe.ru kinobest.pl
job.icivil.ir klikbonus.com
jwcialislrt.com klin3952.ru
kalb.ru kovrenok.ru
kandidos.com landoftracking.com
keenoutlet.online laxob.com
kiprinform.com ldrtrack.com
kiwe-analytics.com limtu.ifmo.ru
klitimg.pw lion.bolegapakistan.com
knowyournextmove.com lionking-1994.blogspot.com
krasnodar.xrus.org liveinternet.ro
lafriore.ru lizace.com
landscapebackgrounds.blogspot.com loginduepunti.it
laparfumotec.com lunamedia.co
law-check-nine.xyz mearns-tractors.co.uk
legionalpha.com mediafresh.online
linerdrilling.com medispainstitute.com.au
liveshoppersmac.com megaapteka.ru
longadventure.com megaindex.ru
losangeles-ads.com middlerush-a.akamaihd.net
m0r0zk0-krava.ru mixx.com
macdamaged.tech mlvc4zzw.space
magazin-pics.ru mmoguider.ru
manuscript.su mobplayer.ru
max-eclat.men motherboard.vice.com
meltwater.com muschisexbilder.com
midst.eu
mockupui.com
monclercheap.xyz
money-for-placing-articles.com
monitorwebsitespeed.com
mrvideospornogratis.xxx
musezone.su musezone.su
musicspire.online nalogovyy-kodeks.ru
mybinaryoptionsrobot.com net-profits.xyz
myblogregistercm.tk next-dentists.tk
na15.ru nicefloor.co.uk
netoil.no nikhilbahl.com
nmrk.ru nikitabuch.com
novgorod.xrus.org nofreezingmac.click
novosibirsk.xrus.org noscrapleftbehind.co
o-o-6-o-o.com notasprensa.info
of-ireland.info nrv.co.za
offer.camp nuker.com
ons-add.men oktube.ru
paccohichetoti.ml olvanto.ru
partner-cdn.men onlinadverts.com
partner-host.men online7777.com
partnerline.men onlythegames.com
pewit.pw parfusale.se
photochki.com partner-trustworthy.men
pigrafix.at pay2me.pl
pixgood.com playmsn.com
pk-pomosch.ru playtap.us
platezhka.net polimga.pw
plusnetwork.com pons-presse.com
pornotubexxx.name potoideas.us
portal-eu.ru promodj.com
printdirectforless.com prosmibank.ru
printingpeach.com pshare.biz
privatbank46.ru pulse33.ru
pro-poly.ru quickcashlimited.com
psa48.ru rcb101.ru
ratemodels.net realresultslist.com
realizmobi.com regionshop.biz
remorcicomerciale.ro remedyotc.com
renewablewealth.com replica-watch.ru
rfd-split.hr rock-cafe.info
ros-ctm.ru royal-investments.net
royalcar-ufa.ru runstocks.com
royalvegascasino.com rutor.group
rubanners.com sawin.beth.webd.pl
ryetaw.com sdrescher.net
s8-nowy-wygraj.comli.com securesmrt-dt.com
salut-camp.ru sethrollins.net
scanner-mark.top sexkrasivo.net
scanner-nelson.top sexobzor.info
search.1and1.com shopperifymac.com
segol.tv shtaketniki.kz
sendearnings.com site.ru
seozoom.it siteaero.com
sexyteens.hol.es sitevaluation.com
shopfishing.com.ua skatestick.bid
sibdevice.ru sklad-24.ru
sigmund-freud.co.uk smartshoppymac.com
simplepooltips.com snip.tw
siteripz.net sockshares.tv
social-fun.ru sofit-dmd.ru
speedup-my.site soochi.co
spravka130.ru spyfu.com
stevemonsen.com spymac.net
sukarame.net strana-krasoty.ru
superstarfloraluk.com stroiminsk.org
svnuppsalaorebro.se surflinksmedical.com
thelottosecrets.com the-usa-games.blogspot.com
thin.me.pn thecoolimages.net
timer4web.com thepantonpractice.co.uk
tizanidinefibromyalgia.blogspot.com thetoiletpaper.com
tizanidineoral.blogspot.com tivolibasket.it
tizanidinesleepaid.blogspot.com tizanidine6mg.blogspot.com
topquality.cf tizanidineduringpregnancy.blogspot.com
torrnada.ru tizanidinevscyclobenzaprine.blogspot.com
trackmedia101.com trailer.cinemaflix.website
traffique.net try-rx.com
tuberkuleznik.ru tt-ipd.info
tula.howotorg.ru unacittaconte.org
unionmarkt.de unitexindia.com
v-doc.co url-extractor.xyz
vapomnoncri.tk vadimkravtcov.ru
wardreapptokone.tk vapsy.com
warezaccess.com vchulkax.com
webinstantservice.com vereo.eu
webstatsdomain.org via-gra.webstarts.com
wemedinc.com viagroid.ru
wesharepics.site videochat.ph
westsextube.com vilingstore.net
whos.amung.us villacoloniale.com
workle.website vinsit.ru
worldinternetauthority.com vip-parfumeria.ru
www.ehscloud.cn volgograd.xrus.org
www.timer4web.com vzlomfb.com
www.wohnkabinen-shop.de vzlomtw.com
vzube.com
waterefficiency.co
westen-z.life
wma-x.com
womensplay.net
wygraj-skiny.win
xerox-douglas.cf xerox-douglas.cf
xn--e1afanlbnfckd7c3d.xn--p1ai xn----itbkqkfiq.xn--p1ai
yaderenergy.ru xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
yougame.biz xxxnatelefon.ru
youghbould.wordpress.com za-fun-offer.com
yoursite.com za-music.mymobiplanet.com
youtube-downloader.savetubevideo.com zolotoy-lis.ru
zaobao.com.sg
zatjmuzu.info
zigarettenonlinekaufen.tumblr.com
zirondelli.it
zoominfo.com
zrizvtrnpale.tk
zzlgxh.com zzlgxh.com

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.75 0; 10.1.0.85 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2716 ### Version: V4.2021.11.2717
### Updated: Mon Nov 15 10:03:18 UTC 2021 ### Updated: Mon Nov 15 22:01:37 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2716 ### Version: V4.2021.11.2717
### Updated: Mon Nov 15 10:03:18 UTC 2021 ### Updated: Mon Nov 15 22:01:37 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##