vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.11.2718 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-11-16 10:09:25 +00:00
parent 4ed1c84abe
commit f02cb62658
47 changed files with 66270 additions and 66260 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2717 #### Version: V4.2021.11.2718
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.11.2717 #### Version: V4.2021.11.2718
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.11.2717 #### Version: V4.2021.11.2718
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8148
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.11.2717 #### Version: V4.2021.11.2718
#### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7077](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [627](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8152
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8152
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8152
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8152
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

8152
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

8152
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

8152
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2717 2718

8152
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8152
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.191.142.30 0; 52.175.203.137 0;

8152
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

8152
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

8152
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

8152
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.191.142.30 0; 52.175.203.137 0;

8152
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.191.142.30 0; 52.175.203.137 0;

8152
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 1; 10.1.1.35 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
InfoTekies Barkrowler
Mojeek Bolt
OutfoxBot Crawling at Home Project
Page Analyzer DomainAppender
Pi-Monster FDM
SalesIntelligent LinqiaScrapeBot
Toata NetSpider
Zauba Nibbler
ZumBot Spinn3r
x09Mozilla plumanalytics

304
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 01h4x.com
80legs
ADmantX ADmantX
ASPSeek AIBOT
Abonti AhrefsBot
Aboundexbot
AiHitBot AiHitBot
Aipbot
AlphaBot AlphaBot
Anarchy Anarchy
Anarchy99
Ankit Ankit
Apexoo
Aspiegel
Asterias Asterias
AwarioSmartBot Attach
BBBike
BDCbot
BDFetch BDFetch
BLEXBot
BackDoorBot
BackStreet
BackWeb BackWeb
BacklinkCrawler
Bandit
Barkrowler Barkrowler
Battleztar Bazinga BatchFTP
BetaBot
Bitacle Bitacle
Black Hole Black Hole
BlackWidow Blow
BlowFish
Boardreader
Bolt
BotALot BotALot
Brandprotect
Brandwatch Brandwatch
BuiltBotTough Buck
BuiltWith Buddy
CATExplorador CCBot
CODE87
CSHttp CSHttp
Calculon Calculon
CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
CheTeam CheeseBot
ChinaClaw CherryPicker
Claritybot Chlooe
Cloud mapping Cloud mapping
Cogentbot Cocolyzebot
Collector
CopyRightCheck
Craftbot
Crawling at Home Project Crawling at Home Project
Crescent Crescent
Curious CyotekWebCopy
DSearch
DatabaseDriverMysqli
Demon Demon
Deusu Deusu
Dirbuster Digincore
DigitalPebble
Disco
Discobot
Discoverybot
DittoSpyder DittoSpyder
DomainSigmaCrawler DomCopBot
DomainStatsBot
Download Wonder Download Wonder
Drip Dragonfly
ECCP/1.0 EMail Siphon
EasyDL EMail Wolf
ExtLinksBot Ebingbong
Ezooms EirGrabber
Evil
Exabot
Extractor
Extreme Picture Finder
FDM FDM
FemtosearchBot FemtosearchBot
Fimap
Firefox/7.0 Firefox/7.0
FlashGet Flunky
GT::WWW Freeuploader
Genieo FrontPage
Grabber Fuzz
Fyrebot
Go!Zilla
Go-Ahead-Got-It
GoZilla
Grafula Grafula
GrapeshotCrawler GrapeshotCrawler
HTTrack GridBot
HEADMasterSEO
HTMLparser
HaosouSpider
Harvest
HybridBot HybridBot
IDBTE4M
IDBot IDBot
Iblog IRLbot
Id-search Image Sucker
IlseBot InfoNaviRobot
Image Fetch
Intelliseek
Internet Ninja Internet Ninja
InternetSeer InternetSeer
Iskanie Jyxobot
JamesBOT Kenjin Spider
JennyBot
JetCar
Jetty
Jorgee
JustView
Keyword Density Keyword Density
Kozmosbot
LWP::Simple LWP::Simple
Leap Lanshanbot
LeechFTP LibWeb
Lftp Lightspeedsystems
LinqiaMetadataDownloaderBot LinkextractorPro
LinqiaRSSBot LinksManager
LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
MFC_Tear_Sample
MIDown tool
MJ12bot
MSFrontPage
MTRobot
Mag-Net Mag-Net
Magnet Magnet
Majestic SEO
Majestic-SEO Majestic-SEO
MarkWatch MarkWatch
Masscan Masscan
Mata Hari MauiBot
MeanPath Bot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru Microsoft Data Access
Moblie Safari Mister PiX
Morfeus Fucking Scanner Mozlila
Msrabot Musobot
NPbot
Name Intelligence Name Intelligence
Net Vampire Nameprotect
NetAnts Nessus
NetLyzer NetLyzer
Nikto Netcraft
NimbleCrawler Nettrack
Nuclei Nimbostratus
Offline Explorer Nmap
Offline Navigator Nutch
OnCrawl
OpenLinkProfiler OpenLinkProfiler
Openfind OpenVAS
Openvas
OrangeBot OrangeBot
Page Analyzer OutclicksBot
PageGrabber OutfoxBot
PECL::HTTP
Pandalytics Pandalytics
PeoplePal
Petalbot Petalbot
Picscout
Piepmatz Piepmatz
Pixray Pixray
Psbot ProWebWalker
Quick-Crawler Probethenet
Pu_iN
Pump
QueryN Metasearch
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankingBot RankingBot
Rankivabot RankingBot2
RankurBot RankurBot
Recorder ReGet
RealDownload
Reaper
SBIder SBIder
SEOkicks SEOkicks-Robot
SEOlyticsCrawler SEOstats
SMTBot SalesIntelligent
ScanAlert ScanAlert
ScoutJet Scanbot
Screaming
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
SemanticJuice SemanticJuice
Semrush Semrush
SemrushBot
SentiBot SentiBot
Seomoz SeoSiteCheckup
Shodan Shodan
SiteCheckerBotCrawler SiteSucker
Sitebeam Sitebeam
Siteimprove Siteimprove
SlySearch Sitevigil
SmartDownload
Snapbot Snapbot
Snoopy Sosospider
SocialRankIOBot SpaceBison
Sociscraper
Sogou web spider
Sottopop
Spammen Spammen
Spanner SpankBot
Spbot
Spinn3r Spinn3r
Steeler Sqlworm
Stripper
Sucker Sucker
SuperBot
SuperHTTP SuperHTTP
SurveyBot
Suzuran
Swiftbot Swiftbot
T8Abot
Teleport Teleport
TeleportPro TeleportPro
Telesphoreo Telesphorep
TheNomad
TightTwatBot TightTwatBot
Toweyabot
Trendiction
Trendictionbot
True_Robot True_Robot
Turingos Turingos
Twice
Typhoeus Typhoeus
URLy.Warning UnisterBot
Vacuum Upflow
V-BOT
VB Project
Vagabondo
VidibleScraper VidibleScraper
Virusdie
VoidEYE
Voil
Voltron
WASALive-Bot WASALive-Bot
WBSearchBot WBSearchBot
WISENutbot WEBDAV
WWWOFFLE WWW-Collector-E
WWW-Mechanize
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Fetch
WebBandit Web Fuck
Web Pix
Web Sucker
WebCopier
WebEnhancer WebEnhancer
WebFetch
WebFuck
WebGo IS
WebImageCollector
WebLeacher
WebPix
WebReaper
WebSauger WebSauger
WebmasterWorldForumBot WebSucker
WebWhacker
Webalta
Webshag Webshag
WebsiteQuester WebsiteQuester
Whack Whacker
Whatweb Whatweb
Who.is Bot Who.is Bot
WinHTTrack
WiseGuys Robot WiseGuys Robot
Wprecon
Xaldon WebSpider
Xaldon_WebSpider
Xenu Xenu
YoudaoBot Zade
Zitebot Zermelo
ZmEu ZoomBot
archive.org_bot ZoominfoBot
adscanner
arquivo-web-crawler arquivo-web-crawler
autoemailspider
backlink-check backlink-check
clark-crawler cah.io.community
cognitiveseo check1.exe
coccocbot-web
com.plumanalytics com.plumanalytics
dataforseo.com crawl.sogou.com
evc-batch evc-batch
magpie-crawler ips-agent
isitwp.com
muhstik-scan muhstik-scan
netEstate NE Crawler scan.lol
pcBrowser
polaris version
satoristudio.net
seobility seobility
seocompany.store seocompany.store
seoscanners
seostar
sexsearcher sexsearcher
spyfu trendiction.de
sysscan webmeup-crawler
tAkeOut webpros.com
ubermetrics-technologies.com webprosbot
voyagerx.com
x22Mozilla
zauba.io
zgrab

294
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
80legs 404enemy
ADmantX ADmantX
AIBOT AIBOT
AhrefsBot ASPSeek
AiHitBot Aipbot
Alexibot AlphaBot
Anarchie Anarchie
Anarchy Apexoo
Anarchy99
Asterias Asterias
Attach AwarioRssBot
BLEXBot BDCbot
BackDoorBot Backlink-Ceck
BackStreet
Badass Badass
Barkrowler Bandit
BatchFTP BatchFTP
Blow BetaBot
BlowFish Bigfoot
Boardreader Bitacle
Brandwatch Blackboard
BuiltWith BotALot
BuzzSumo Brandprotect
CCBot Buddy
CazoodleBot Bullseye
CATExplorador
CSHttp
Calculon
Cegbfeieh
CheeseBot
CherryPicker CherryPicker
ChinaClaw
Chlooe
Claritybot Claritybot
Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Collector
Copyscape Copyscape
Cosmos Craftbot
CyotekWebCopy
DBLBot DBLBot
DataCha0s DTS Agent
DatabaseDriverMysqli Deusu
Demon
Devil Devil
Dirbuster Digincore
Disco Discoverybot
Dispatch
DnBCrawler-Analytics DnBCrawler-Analytics
DomainAppender DomainSigmaCrawler
DomainCrawler DomainStatsBot
Dotbot Domains Project
EirGrabber Ebingbong
ExtLinksBot EroCrawler
ExtractorPro Exabot
EyeNetIE Express WebPictures
FDM
Fimap Fimap
Firefox/7.0
FlashGet
Foobot Foobot
FyberSpider Freeuploader
Fuzz
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot GalaxyBot
Genieo Genieo
GermCrawler Getintent
GetRight Go-Ahead-Got-It
GoZilla GoZilla
Gotit Gotit
GrabNet
Grabber
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
HMView HTMLparser
HTTP::Lite HTTP::Lite
Harvest HTTrack
Havij Haansoft
Heritrix Heritrix
Humanlinks Hloader
IDBTE4M
Id-search Id-search
Image Fetch Image Fetch
Image Sucker
IndeedBot IndeedBot
InfoTekies Indy Library
Intelliseek
Internet Ninja
InternetSeer InternetSeer
Iria Iria
JennyBot IstellaBot
Jbrofuzz
JetCar JetCar
JikeSpider JikeSpider
Keyword Density Kenjin Spider
Kozmosbot Kozmosbot
LeechFTP LeechFTP
Libwhisker Lftp
LieBaoFast LinkWalker
LinkScan Linkdexbot
LinkpadBot LinkpadBot
Lmspider LinqiaMetadataDownloaderBot
LinqiaScrapeBot
Lipperhey Spider
Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample
MIDown tool
MIIxpc MIIxpc
MQQBrowser MJ12bot
MTRobot
Mag-Net
Magnet Magnet
Majestic12 Mata Hari
MarkMonitor MicroMessenger
MarkWatch Mr.4x3
Mass Downloader Musobot
Masscan Nameprotect
MauiBot Needle
Mb2345Browser
Microsoft Data Access
Mojeek
Morfeus Fucking Scanner
NICErsPRO
NPbot
Name Intelligence
Navroad
NetAnts NetAnts
NetLyzer
NetMechanic
NetSpider NetSpider
NetZIP
Netcraft Netcraft
Nettrack Nettrack
NextGenSearchBot Nikto
Nimbostratus Ninja
Nmap
Octopus Octopus
Offline Explorer
OnCrawl OnCrawl
OpenVAS Openfind
Openvas Openvas
OutclicksBot OrangeSpider
OutfoxBot OutfoxBot
PECL::HTTP POE-Component-Client-HTTP
Page Analyzer PageScorer
PageGrabber Pandalytics
PageThing.com PeoplePal
Picscout
Pixray
PleaseCrawl PleaseCrawl
Quick-Crawler Pockey
RSSingBot ProPowerBot
Psbot
Pu_iN
RankFlex RankFlex
Rankivabot RebelMouse
RankurBot RedesScrapy
Re-re
ReGet
Recorder
RepoMonkey
Ripper
Rogerbot
SBIder
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SISTRIX
SMTBot SMTBot
ScoutJet ScanAlert
ScreenerBot Scrapy
ScrepyBot Screaming
Searchestate Searchestate
SemanticJuice SemanticJuice
Semrush Semrush
SentiBot SemrushBot
Shodan Seomoz
Siphon Site Sucker
SiteSnagger SiteExplorer
Sitebeam SiteLockSpider
Siteimprove SiteSucker
Sitevigil Sitevigil
SmartDownload SmartDownload
Snapbot
Snoopy
SocialRankIOBot
Sociscraper
Sosospider Sosospider
Sottopop Sottopop
Spammen
SpankBot
Spanner Spanner
Spinn3r Steeler
SputnikBot SuperBot
Sqlmap
Sqlworm
Sqworm
Sucker
SuperHTTP SuperHTTP
Surfbot
Suzuran
Swiftbot Swiftbot
Szukacz
Teleport Teleport
TeleportPro
Telesoft
The Intraformant The Intraformant
Titan TheNomad
Toweyabot Toata
Tracemyfile
Trendictionbot Trendictionbot
Turnitin Turnitin
URLy Warning URLy Warning
URLy.Warning
UnisterBot
Upflow Upflow
VB Project
Vacuum Vacuum
Vagabondo VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie Virusdie
VoidEYE
Voltron Voltron
WASALive-Bot WBSearchBot
WEBDAV
WISENutbot WISENutbot
WPScan
WWW-Mechanize
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
Web Auto Web Auto
Web Fuck Web Enhancer
Web Fetch
Web Sauger Web Sauger
WebAuto WebAuto
WebFetch WebCopier
WebFuck
WebGo IS WebGo IS
WebImageCollector
WebPix WebPix
WebSauger WebSauger
WebWhacker WebWhacker
WebZIP WebZIP
WebmasterWorldForumBot Webalta
Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester Webster
Whacker
Who.is Bot
WinHTTrack WinHTTrack
Wonderbot
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot YoudaoBot
Zitebot Zade
ZumBot ZoomBot
ZoominfoBot
ZyBorg ZyBorg
adscanner adscanner
archive.org_bot
autoemailspider autoemailspider
clark-crawler cah.io.community
check1.exe
coccocbot-web
cognitiveseo cognitiveseo
com.plumanalytics
crawl.sogou.com
crawler4j crawler4j
dataforseo.com
demandbase-bot demandbase-bot
gopher gopher
instabid
internetVista monitor
ips-agent ips-agent
meanpathbot isitwp.com
magpie-crawler
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
probe-image-size plumanalytics
s1z.ru ripz
scan.lol satoristudio.net
seostar scalaj-http
seobility
seocompany.store
seoscanners
serpstatbot serpstatbot
sitechecker.pro sitechecker.pro
spyfu siteripz
sogouspider
sysscan sysscan
tAkeOut
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com voyagerx.com
webmeup-crawler x09Mozilla
webpros.com
webprosbot
x22Mozilla
xpymep1.exe
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
advokat-grodno.by 01apple.com
analytics-ads.xyz apiholdingmypage-a.akamaihd.net
hostnow.men appsecurityr.com
ledis.top custom-product-labels.com
mistr-x.org elektronischezigarettekaufen1.beeplog.com
pizdopletka.club hit-men.men
running-line.ru mesto-x.com
sitiz.club mini.7zap.com
strigkaomsk.ru sexystrippe.info
zhongwenlink.com tengohydar.tk

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 0daymusic.org
2000k.ru 123movies.love
28n2gl3wfyb0.ru 18ps.ru
45en.ru 2015god.org
4kepics.com 2daytrendingnews.com
8si.ru 4pp13.com
aa625d84f1587749c1ab011d6f269f7d64.com 5kstore.com
aadroid.net 8lufu.com
aanapa.ru a14download.com
abclauncher.com aac500b7a15b2646968f6bd8c6305869d7.com
adclickthru.net absolutelyanalog.com
adult3dgames.com acc.eu.org
advancedcleaner.com acheterviagrafr24.com
alevinefesleri.com addtoadd.men
aliexpresscashback.club adsland.men
alltheviews.com affordablewebsitesandmobileapps.com
aloofly.com ageofclones.com
amigobulls.com aihelen.net
amt-k.ru alert-hgd.xyz
animalrank.com all4invest.info
anjalika.co.in allboard.xobor.de
apisecretsaucebi-a.akamaihd.net allornamenti.com
app-ready.xyz allwomen.info
apple.com-cleaner.systems alot.com
as5000.com ameblo.jp
amung.us
animal-drawings.com
apartmentratings.com
apimossnetinfo-a.akamaihd.net
appfixing.space
apps-analytics.net
aramaicmedia.org
arcadeplayhouse.com
arendakvartir.kz
asdfz.pro
asophoto.com asophoto.com
asseenontv.ru auto-zapchasti.org
astrochicks.com autobudpostach.club
atleticpharm.org autolombard-krasnodar.ru
atout-energie-69.com availit.weebly.com
autoua.top aviav.eu
avek.ru avon-novosib.ru
avramstroy.ru avon-ukraine.com
avtoarenda.by avtolombard-krasnodar.ru
babosas.com backlinkwatch.com
babs.com.ua balook.com
baersaratov.ru banki76.ru
baksman.com bannerbank.ru
balitouroffice.com bannerspace.com
beachpics.com becuo.com
belstaffstore.online bensbargains.net
beremenyashka.com berdasovivan.ru
bestcalovebracelet.cn bestdraws.com
bestdooz.com bestofferswalkmydogouteveryday.gq
bestpornuha.com bestofpicture.com
betonka.pro bestssaker.com
betune.onlinewebshop.net betteroffers.review
betwinservice.com
big-boards.info
bigpenisguide.com bigpenisguide.com
bioscorp.ru billiard-classic.com.ua
blackle.com binaryoptionscops.info
blagovest-med.ru biznesluxe.ru
blavia.00author.com bjetjt.com
bluesalt.co blpmovies.com
boleznikogi.com bluejays-jerseys.us
bonvillan.com bluerobot.info
boobsimge.com bluesman.nu
bookmark4you.com.biz bmwhighperformers.com
braip.com.br boc.kz
brendbutik.ru boo-arts.com
bristolhotel.com boom-celebs.com
brk-rti.ru bouda.kvalitne.cz
brus.city brainsandeggs.blogspot.com
buigas.00it.com brewdom.ru
bukleteg.xyz brianhenry.co
buqayy0.livejournal.com briomotor.co
burger-tycoon.com burger-tycoon.com
bus-offer.com burn-fat.ga
buy-cheap-pills-order-online.com buron.pw
buyfriend.ru businesxxl.com
buyscabiescream.com buy-cheap-online.info
buzzonclick.com buybest1.biz
bvlgariwallet.xyz buzzzg.men
bycontext.com byme.se
cactussoft.biz c-english.ru
capsderfudd.tk camsex.xxx-cam.webcam
cars-modification.net candiceloves.us
cartierwatch.xyz cardsdumps.com
ccjp.eu carloans.com
cdncash.org cctva.tv
cheapestjerseysonwholesale.com celejihad.info
cercacamion.it
certifywebsite.win
cheap-trusted-backlinks.com
cheapjerseysa.com cheapjerseysa.com
chinaelitecheapjerseys.com cheapjerseysfootballshop.com
chocolateslim-u-srbiji.com chimiver.info
cinemaflix.website ciekawinki.pl
citetick.com
citizenclsdriveri7.pen.io
cityadspix.com
clayvasedesigns.tk clayvasedesigns.tk
clubfashionista.com cleanmypc.com
co.lumb.co cleansearch.net
compliance-elena.top cocyq.inwtrade.com
compliance-julia.top compiko.info
compliance-willy.top compliance-walt.top
connectingsingles.com constantaservice.net
copenergo.ru cookie-law-enforcement-ff.xyz
coolbar.pro
copypaste.traderzplanet.in
cornerstone-countertops.com
covetnica.com
cracksplanet.com
creams.makeforum.eu creams.makeforum.eu
creditnation.ru custodieva.ru
cupday.com customerpromos-a.akamaihd.net
custom-electric-guitar.com daretodonate.co
customboxes4less.com darkbooks.org
cypernhuset.se de.zapmeta.com
dailyfinancefix.com dealitright.click
dailystrength.org decorationspcs.com
dawlenie.com deshevo-nedorogo.ru
dealighted.com detective01.ru
design-ideas.info dichvuvesinhhanoi.com
destinationrealestate.com dignitasdata.se
dimaka.info dimaka.info
dinosaurus.site dipstar.org
dividendo.ru discountliv.com
djstools.com discovertreasure-a.akamaihd.net
dktr.ru disruptingdinnerparties.com
domaineaneblanc.com divci-hry.info
domination.ml dneprsvet.com.ua
domini.cat dolohen.com
domaincheck.io
domik-derevne.ru
donvito.unas.cz donvito.unas.cz
downloaddy.net downloadeer.net
doxyporno.com druzhininevgeniy63.blogspot.com
drupa.com dstroy.su
dverimegapolis.ru dyt.net
dwomlink.info eaplay.ru
eandsgallery.com ec-file.info
easycommerce.cf econom.co
easytuningshop.ru egvar.net
ege-essay.ru electro-prom.com
ekn-art.se elektir.ru
elektrozigaretten2.yn.lt elektrischeziga.livejournal.com
elexies.info elektrischezigarette1.onsugar.com
elitepcgames.com elkacentr.ru
eloconcream.blogspot.com eluxer.net
en.home-task.com elvenar.com
energy-ua.com enhand.se
epngo.bz epicbrogaming.com
european-torches.ru eropornosex.ru
exdocsfiles.com erotag.com
exhibitionplus.eu erotische-geschichten-xxl.com
exlarseva.webblog.es escortslet.net
exportshop.us essay-edu.biz
extractorandburner.com eurosamodelki.ru
excitacion.info
ezigarettekaufen2.mpbloggar.se
f00kclan.de
f012.de
farm26.ru
farmprofi.net
fashion-stickers.ru
fashionavenuegame.com fashionavenuegame.com
felizporno.com feel-planet.com
fenoyl.batcave.net fertilitetsradgivningen.se
fermersovet.ru filmetricsasia.com
fifa-coins.online financehint.eu
filefilter.weebly.com flashbannernow.com
filesdatabase.net flashlarevista.com
filmania-x.ru flowersbazar.com
filmgo.ru footballfarrago.com
finansov.info foto-sisek.porngalleries.top
fitnesspiks.com fr.netlog.com
floridamhca.org free-stock-illustration.com
forsex.info fres-news.com
forum.doctissimo.fr frivgame250.com
fotosfotos.eu fun2cell.net
foxjuegos.com fyxabomiw.ru
frankofficial.ru gaming-journal.com
freelifetimefuckbook.com gdzkurokam.ru
freshberry.com.ua ge0ip.net
fusoradio.info ge0ip.org
galeon.com gearsadspromo.club
geilehausfrauen.net generalporn.org
getdot.ru generousdeal-a.akamaihd.net
gheus.altervista.org get-free-traffic-now.com
gigapeta.com
girlporn.ru girlporn.ru
gk-atlant.info golden-praga.ru
glavprofit.ru gopixdatabase.com
goforexvps.com
goldandcard.ru
googst2.ru
gosreg.amchs.ru gosreg.amchs.ru
gsasearchengineranker.space gotcher.us
gsmtlf.ru gourcy.altervista.org
guarrasdelporno.xxx gq-catalog.gq
gungamesz.com greatfind-a.akamaihd.net
hackers-crackers.tk greenzaim.ru
handicapbathtubarea.com gwagka.com
hdapp1008-a.akamaihd.net gwebtools.com
heroz.fr haikuware.com
highspeed5.net hamacapty.com
hmmm.cz handicapvantoday.com
hobild.net hanink.biz.ly
hornymatches.com hdxnxxtube.mobi
hotgirlhdwallpaper.com healing-dysplasia.ru
hothot.ru hem.passagen.se
hully.altervista.org hidefiles.org
husky-shop.cz hit-kino.ru
hitmuzik.ru
hotblognetwork.com
howlongdoestizanidinestayinyoursystem.blogspot.com
hscsscotland.com
hustoon.over-blog.com hustoon.over-blog.com
idealtits.net hut1.ru
igithab.com idc.com.ua
ilovevitaly.xyz imk.com.ua
imagefinder.site inbabes.sexushost.com
immobiliaremassaro.com incolors.club
impisr.ru innoslicon.com
impresagaia.it intellekt21.ru
increasewwwtraffic.info ipostroika.ru
informatiecentro.be iqbazar.ru
innodgfdriverhm.aircus.com iqoption.pro
inome.com.ua iradiology.ru
instabid.tech istizanidineanopiate.blogspot.com
intellektmedia.at ivoiretechnocom.ci
iplogger.org
iptool.xyz
istizanidineanarcoticdrug.blogspot.com
itbc.kiev.ua
ivanovo.zrus.org
jackpotchances.com jackpotchances.com
japfm.com japfm.com
jobmarket.com.ua jerseychinabizwholesale.us
jongose.ninja jikoman.info
jpcycles.com jrcigars.com
kaliningrad.zrus.org juliaworld.net
katadhin.co justbcause.com
kimcurlrvsms.com ketoanhanoi.info
kino-filmi.com keyword-suggestions.com
kinoflux.net kfon.eu
kiwe-analytics.com kinomaniatv.pl
kletkimehan.ru klosetkitten.com
koopilka.com kollesa.ru
krafte.ru kotaku.com
ku6.com krasnodar-avtolombards.ru
kustanay.kz krasnodar.zrus.org
krassh.ru
kredit-pod-zalog-krasnodar.ru
kuzinsp.ru kuzinsp.ru
labplus.ru kvartir-remont.biz
lacave.ntic.fr ladov.ru
ladsup.com laexotic.com
larger.io lafourmiliaire.com
laserpen.club landoftracking.com
lechtaczka.net legionalpha.com
lemon-ade.site lenpipet.ru
letolove.ru levaquin750mg.blogspot.com
life.biz.ua lightinghomes.net
limads.men linkbolic.com
linerdrilling.com
linkbux.ru
linkdebrideur.xyz linkdebrideur.xyz
ljusihus.se linkpulse.com
llastbuy.ru littleberry.ru
littlesunraiser.com
liumimgx.pw
lombia.com lombia.com
longgreen.info longgreen.info
lotto6888.com
louboutinbooties.xyz louboutinbooties.xyz
louboutinreplica.xyz luchshie-topcasino.ru
love-baby.cz luxup.ru
lsp-awak-perikanan.com mac-shield.com
machicon-ueno.info mafa-free.com
macrotek.ru mafcards.ru
magiadinamibia.blogspot.com manipulyator-peterburg.ru
mamasuper.prom.ua maranbrinfo.com.br
marketingtechniques.info marinetraffic.com
matsdale.com marketland.ml
medispainstitute.com.au masserect.com
medkletki.ru maxxtor.eu
medmajor.ru mebelcomplekt.ru
meduza-consult.ru medi-fitt.hu
meendo-free-traffic.ga mex-annushka.ru
megaindex.ru mexicotravelnet.com
megavolt.net.ua micasainvest.com
message-warning.net michaelkorsoutletstore.net
microsearch.ru mirmedinfo.ru
mil-stak.com missclub.info
mirtorrent.net mnogabukaff.net
mirzonru.net
missis.top
mobot.site
monclerboots.xyz
monclercoats.xyz monclercoats.xyz
moncleroutletonline.win moncleroutletonline.win
mrvideospornogratis.xxx moscow-region.ru
muz-shoes.ru mrlmedia.net
myhealthcare.com mustwineblog.com
mydirtyhobby.com
mylesosibirsk.ru
myliveblog.ru
mysexpics.ru
na-telefon.biz
nalogovyy-kodeks.ru nalogovyy-kodeks.ru
nash-krym.info narkologiya-orel.ru
nelc.edu.eg naughtyconnect.com
next-dentists.tk notebook-pro.ru
nightvision746.weebly.com onescreen.cc
noumeda.com onko-24.com
ofanda.com online7777.com
offergroup.info onthemarch.co
omsk.xrus.org oralsexfilme.net
onlyporno.ru orsonet.ru
oops-cinema.ru pammik.ru
ourville.info parajumpersjakkesalgnorge.info
outrageousdeal-a.akamaihd.net
owohho.com
page2rss.com
panicatack.com
parlament.biz parlament.biz
partner-trustworthy.men partnerline.men
pateaswing.com pcimforum.com
paydayloanslocal.com pepperstyle.ru
pcgroup.com.uy petedrummond.com
perm.xrus.org
perso.wanadoo.es
petsblogroll.com
photorepair.ru photorepair.ru
phuketscreen.com phuketscreen.com
picswe.com pic2fly.com
pirateday.ru piccdata.com
plastgran.com picsfair.com
pokerniydom.ru picturesfrom.com
porn-w.org pigrafix.at
porngalleries.top plastgranar.nu
porno-raskazy.ru platinumdeals.gr
playmsn.com
plugingeorgia.com
pobeiranie.pl
podshipniki-ntn.ru
poem-paying.gq
polcin.de
popunder.ru
pornobrazzers.biz
pornohd1080.online
pornoklad.net
portadd.men portadd.men
portnoff.od.ua pozvonim.com
postclass.com pregnant.guru
priora-2.com preparevideosafesystem4unow.site
privetsochi.ru presleycollectibles.com
privhosting.com printie.com
produkto.net privatbank46.ru
prizefestival.mobi
prizestohandle.club
pro-tec.kz
proekt-gaz.ru proekt-gaz.ru
profolan.pl
progonrumarket.ru
promoheads.com promoheads.com
proprostatit.com prostitutki-astana.org
prosperent.com proxyradar.com
prostitutki-tolyatti.org pst2017.onlinewebshop.net
psa48.ru psvita.ru
psoriasis-file.trade pussyspace.net
purchasepillsnorx.com qcstrtvt.bloger.index.hr
puzzleweb.ru qexyfu.bugs3.com
quickloanbank.com qitt.ru
rank-checker.online quizzitch.net
rankia.com rangjued.com
ranksonic.info redirect.trafficreceiver.club
real-time-analytics.com rednise.com
remont-komputerov-notebook.ru remmling.de
remontgruzovik.ru remont-comp-pomosh.ru
rennlist.com renecaovillasale.online
resant.ru rentehno.ru
respublica-otel.ru
reward-survey.net
ringtonepartner.com
riralmolamsaca.tk riralmolamsaca.tk
risparmiocasa.bz.it roll123.com
rostov.xrus.org rossanasaavedra.net
rukino.org royal-betting.net
ruscopybook.com royalads.net
russia-today-video.ru rus-pornuha.com
saitevpatorii.com rutor.vip
salmonfishingsacramentoriver.com ryetaw.com
samoiedo.it sadaholding.com
sandhillsonline.com sbetodiodnye-lampy.ru
saveriopiazza.it sbtdesign.co.uk
scalerite.co.za scanner-alexa.top
scanner-elena.top scanner-john.top
scanner-jane.top scanner-willy.top
scansafe.net
scanspyware.net
scrnet.biz.ua
sdelai-prosto.ru
sdelatmebel.ru
search-error.com search-error.com
searchaddis.com seasaltwithfood.com
seasonvar.ru seokicks.de
semprofile.com sergiorossistore.online
seorankinglinks.xyz sex-tracker.de
sexobzor.info sexiporno.net
sextracker.de sherlock.se
sexy-pings.com shiksabd.com
sexystrippe.info shopwme.ru
sfj-ror.no sideeffectsoftizanidine.blogspot.com
shemalegalls.blogporn.in sierraapps.com
shikiso.info simple-image.com.ua
shivafurnishings.com simple-share-buttons.com
shlyahten.ru sisiynas.ru
simpoed.ufop.br site.ru
sinestesia.host.sk sitevalued.com
sirpornogratis.xxx sixcooler.de
sitechecker.pro sladkoevideo.com
siteexpress.co.il slim.sellany.ru
sitevaluation.org smartpet.ru
sitopreferito.it soc-proof.su
skanninge.se social-button.xyz
skladvaz.ru social-buttons.xyz
slomm.ru social-s-ggg.xyz
sluganarodu.ru social-s-iii.xyz
slujbauborki.ru softlinesolutions.me
smartshoppymac.com softonicads.com
snip.to softtor.com
social-search.me sophang8.com
socialmonkee.com spaceshipad.com
soietvousmaime.fr specialfinanceoffers.com
solitaire-game.ru sports-supplements.us
soserfis.com sprttrack.com
soviet-portal.do.am sribno.net
space-worry.ml start.myplaycity.com
speechfoodie.com startufa.ru
spywarespy.com staticfs.host
srecorder.com stretchingabuckblog.com
staynplay.net stretchmate.net
sterva.cc studentguide.ru
sticken.co superoboi.com.ua
stockquotes.wooeb.com surflinksmedical.com
stockspmb.info swinger-mobil.net
summerlinhomes411.info taqplayer.info
superiends.org taxi-v-eisk.ru
svensk-poesi.com telemetryverification.net
svtrd.com teslathemes.com
sygraem.com testbotprocessor44.com
tech4master.com thebestphotos.eu
teletype.in theguardlan.com
the-usa-games.blogspot.com
theautoprofit.ml
thecoolimages.net
thedownloadfreeonlinegames.blogspot.com
thefotosgratis.eu
thegolfclub.info
themestotal.com
thenews-today.info
thepokertimer.com thepokertimer.com
timeallnews.ru thepornsex.org
tizanidineandhydrocodone.blogspot.com thetoiletpaper.com
tizanidinehcl4mgtab.blogspot.com thexart.club
tizanidinehydrochloride2mg.blogspot.com timetorelax.biz
tizanidinehydrochloride4mgstreetvalue.blogspot.com titangel-vietnam.com
tizanidineingredients.blogspot.com tizanidine4mgstreetprice.blogspot.com
tizanidinepills.blogspot.com tizanidine6mg.blogspot.com
tizanidinesideeffectsweightloss.blogspot.com tizanidineandcipro.blogspot.com
tizanidinestreetvalue.blogspot.com
tnaionline.org
toloka.hurtom.com toloka.hurtom.com
toolsky.com tooplay.com
toon-families.com top10-way.com
top250movies.ru torrent-to-magnet.com
topdownloads.ru
torrnada.ru
totu.us totu.us
traderzplanet.in tovaroboom.vast.ru
trafficmania.com track-rankings.online
trichizobswiv.agddns.net triumf-realty.ru
trumpetedextremes.com tula.mdverey.ru
tube8.com turbo-suslik.org
turizm.bz turist-strani.ru
tvnewsclips.info turkeyreport.tk
typimga.pw twelvevisionspartyofcolorado.com
u555u.info u-cheats.ru
ua.tc
ufa.xrus.org ufa.xrus.org
uggbootsoutletsale.us uhod-za-sobakoj.ru
ugguk.online uptime-as.net
unacittaconte.org url-img.link
uptimechecker.com utrolive.ru
usiad.net vadimkravtcov.ru
vabasa.inwtrade.com valuado.com
vapomnoncri.tk vashsvet.com
vchulkah.net vduplo.ru
verymes.xyz vegascosmetics.ru
videogamesecrets.com viandpet.com
videochat.ph
videochat.world
vkak.ru vkak.ru
vkonche.com vladimir.zrus.org
vmnmvzsmn.over-blog.com vozbujdenie.com
voditeltrezviy.ru vpnmouse.com
vvpg.ru vseigru.one
wapsite.me vsexkontakte.net
wdfdocando.com vtcdns.com
webix.biz vzubah.com
webradiology.ru walkme.com
website-analytics.online wdrake.com
website-speed-check.site websitevaluebot.com
website-speed-checker.site weburlopener.com
welcomeauto.ru weightatraining.com
wemedinc.com westermarkanjou.se
westen-z.life whatistizanidine2mg.blogspot.com
white-truck.life wholesalejerseysgaa.com
wholesalecheapjerseysfree.com
wholesalejerseychinashop.com
wieseversa.no
win-spy.com win-spy.com
winwotgold.pl winterclassichockeyjerseys.com
worldinternetauthority.com wpsecurity.website
wowcasinoonline.ooo
wrona.it
ww1943.ru
www.888.com
www.pinnacle-bets.com www.pinnacle-bets.com
x69ty.ru www.solartek.ru
xmronta.com wygraj-skiny.win
xn----ctbbcjd3dbsehgi.xn--p1ai x-rates.ru
xn--80ab4aa2g.xn--p1ai xblognetwork.com
xn--80aikhbrhr.net xfluro.com
xtubeporno.net xmladserver.com
xxxguitars.com xn----7sbho2agebbhlivy.xn--p1ai
yebocasino.co.za xn----7sbifcamovvfggw9d.xn--p1ai
yogamatsexpert.com xn--80ahdheogk5l.xn--p1ai
yottos.com xn--b1adccaf1bzj.xn--p1ai
youdao.com xsion.net
youporn-forum.ga xxxrus.org
youporn-ru.com yellowproxy.net
your-bearings.com yopoint.in
yourdesires.ru yunque.pluto.ro
ypmuseum.ru zaobao.com.sg
yuweng.info zastroyka.org
zaidia.xhost.ro zoomovies.org
zazagames.org zscalertwo.net
zeleznobeton.ru
zoodrawings.com
zs2vm.top
zscaler.net
zverokruh-shop.cz zverokruh-shop.cz
zvuker.net

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1008.su 101billion.com
13tabs.com 2000k.ru
1hwy.com 24videos.tv
1webmaster.ml 4senses.co
2345.com aadroid.net
4kpics.rocks aaronabel.com
55wmz.ru abcguru.xyz
8kisses.eu abctoppictures.net
aa08daf7e13b6345e09e92f771507fa5f4.com above.com
adbetclickin.pink adclickservice.com
adcash.com add-add.men
adman.se advertisingtag.net
adsref.men aero2.ru
adtech.fr affiliate-fr.com
agardomains.com agecheckadult.com
algerianembassy.co.in ahrefs.com
allforminecraft.ru alert-hgd.xyz
aloofly.com alevinefesleri.com
aludecor.info aliexpresscashback.club
ameblo.jp all-streaming-media.com
anal-acrobats.com alldubai.biz
animenime.ru allornamenti.com
anjalika.co.in apimountainbikei-a.akamaihd.net
appearance-cool.com apple.com-webbrowsing-security.science
appfastplay.com aramaicmedia.org
artdestshop.eu arendakvartir.kz
artefakct.com asdfg.pro
asacopaco.tk
asociatia-tipografilor-transilvania.ro asociatia-tipografilor-transilvania.ro
aviav.org ausergrubhof.info
azlex.uz ausmepa.org.au
beremenyashka.com auto.rusvile.lt
best-seo-software.xyz autoua.top
bestchoice.cf avirasecureserver.com
bestkfiledxd.cf azartmix.com
bestwrinklecreamnow.com bad-stars.net
bfz.biz bannerswap.com
barbourjackets.online
battlecarnival.su
battleforupdating.site
beauty-b0x.pl
benchmarkcommunications.co.uk
bestmarriages.com
beta.hotkeys.com
bget.ru bget.ru
bigcities.org bitcoin-ua.top
biznesluxe.ru blog-hits.com
bookhome.info blog.koorg.ru
businesn.men bosefux.esy.es
buyessay3.blogspot.ru budilneg.xyz
bwlx.prepedu.cn buy-meds24.com
cardiosport.com.ua buyessaynow.biz
carsdined.org buypanicdisorderpill.com
cascadelink.org buypillsorderonline.com
cdncash.com buzzzg.men
cfcl.co.uk cablecar.us
cialischstgerts.com cactussoft.biz
coinsspb.com carmuffler.net
companies-catalog.com charmstroy.info
compliance-mark.top cheapsergiorossi.online
conocer-sanabria.com co.lumb.co
custodieva.ru compliance-alexa.xyz
dailystrength.org compliance-don.xyz
davebestdeals.com compliance-olga.top
daymusam.com cookie-law-enforcement-ee.xyz
coolwallpapers4k.info
cubook.supernew.org
cyclobenzaprinevstizanidine.blogspot.com
d2jsp.org
ddpills.com
deadlinkchecker.com
decoratinghomes.ga
deda-moroza-zakaz.ru deda-moroza-zakaz.ru
deutschehobbyhuren.net dikx.gdn
dev78.dioniqlabb.se
dildofotzen.net dildofotzen.net
diusyawiga.tk doctissimo.fr
divatraffic.com documentbase.net
djstools.com domain-tracker.com
doeco.ru domain.webkeyit.com
donvito.unas.cz domaineaneblanc.com
dostavka-v-krym.com dorratex.tn
downloadme.life dptaughtme.com
drhomes.biz duawitchrarato.tk
dvr.biz.ua elektir.ru
econom.co elvenar.com
elmacho.xyz
enbersoft.com
energydiet-info.ru
erectile.bid
eredijovon.com
ereko.ru
eu-cookie-law.blogspot.com
evehemming.blogspot.com.au evehemming.blogspot.com.au
extractorandburner.com exct.net
face.hostingx.eu eyemagination.com
familienzahnaerzte.com ferretsoft.com
favornews.com financepoints.eu
felizporno.com find1friend.com
filefilter.weebly.com finejewelryshop.ru
filmfanatic.com free-laptop-reward.com
frequiry.com fsalas.com
front.to geile-lelly.eu
fun2cell.net geileweiber.tk
gamerscorps.com generic-pills-online.com
garciniaxt.us germes-trans.com
gearcraft.us girlgamerdaily.com
gentamicineyedrops.blogspot.com go2mike.ru
get-your-social-buttons.info gq-catalog.gq
gnuetella.com guge.io
googlemare.com guitar-master.org
goroda-vsego-mira.ru gwagka.com
gratuitxblcodes.com hdimagelib.com
growmyfunds.ca heartofbeijing.blogspot.com
gta-club.ru highland-homes.com
guardlink.com
handicapvantoday.com
headpharmacy.com
healbio.ru
highspeed5.net
hit-kino.ru
hkladys.com hkladys.com
hotdl.in hollywoodactress.info
hotkeys.com housediz.com
howmuchdoestizanidinecost.blogspot.com housekuba.org
ifmo.ru hvd-store.com
infobanks.ru hyhj.info
itsdp3.com i4track.net
ivoiretechnocom.ci images-graphics-pics.com
iwantedmoney.com iqoption-bin.com
jav-fetish.com istizanidineanopiate.blogspot.com
kanimage.com itis4you.com
kicknights.gq iwanttodeliver.com
kinobaks.com johnnyhaley.top
kinobest.pl kabbalah-red-bracelets.com
klikbonus.com keywords-monitoring-success.com
klin3952.ru kihi.gdn
kiwe-analytics.com
kliksaya.com
kochanelli.com
kovrenok.ru kovrenok.ru
landoftracking.com kredit-pod-zalog-krasnodar.ru
laxob.com laxdrills.com
ldrtrack.com letsrepair.in
limtu.ifmo.ru linkpulse.com
lion.bolegapakistan.com liupis.com
lionking-1994.blogspot.com locimge.pw
liveinternet.ro lombardfinder.ru
lizace.com louboutinshoes.xyz
loginduepunti.it love-baby.cz
lunamedia.co luckyshop.net.ua
mearns-tractors.co.uk magicdiet.gq
mediafresh.online marinetraffic.com
medispainstitute.com.au me-ke.com
megaapteka.ru mebelcomplekt.ru
megaindex.ru medizinreisen.de
middlerush-a.akamaihd.net mere.host.sk
mixx.com methodsmarketing.com
mlvc4zzw.space minegam.com
mmoguider.ru mojpregled.com
mobplayer.ru monarchfind-a.akamaihd.net
motherboard.vice.com moonci.ru
muschisexbilder.com moscow-region.ru
musezone.su myliveblog.ru
nalogovyy-kodeks.ru myshopmatemac.com
net-profits.xyz naturalbreakthroughsresearch.com
next-dentists.tk nodding-passion.tk
nicefloor.co.uk novodigs.com
nikhilbahl.com nudepatch.net
nikitabuch.com oneminutesite.it
nofreezingmac.click onko-24.com
noscrapleftbehind.co onlineku.com
notasprensa.info oynat.info
nrv.co.za paparazzistudios.com.au
nuker.com parajumpersstore.online
oktube.ru partner-cdn.men
olvanto.ru pc-services.ru
onlinadverts.com pdamods.ru
online7777.com perfection-pleasure.ru
onlythegames.com petitions.whitehouse.gov
parfusale.se phpdevops.com
partner-trustworthy.men polybuild.ru
pay2me.pl popads.net
playmsn.com prizesbook.online
playtap.us prlog.ru
polimga.pw pro-poly.ru
pons-presse.com pro-tec.kz
potoideas.us prostitutki-odessa.org
promodj.com purchasepillsnorx.com
prosmibank.ru ranking2017.ga
pshare.biz razorweb-a.akamaihd.net
pulse33.ru remont-mobile-phones.ru
quickcashlimited.com
rcb101.ru
realresultslist.com
regionshop.biz
remedyotc.com
replica-watch.ru replica-watch.ru
rock-cafe.info replicaclub.ru
royal-investments.net reversing.cc
runstocks.com ric.info
rutor.group royalcar-ufa.ru
sawin.beth.webd.pl scanner-barak.top
sdrescher.net scanner-walt.top
securesmrt-dt.com seeingmeerkat.com
sethrollins.net seemoreresultshu-a.akamaihd.net
sexkrasivo.net sell-fb-group-here.com
sexobzor.info semalt.com
shopperifymac.com seoheap.com
shtaketniki.kz sex-znakomstva.online
site.ru sexxdate.net
siteaero.com shoesonlinebuy.cn
sitevaluation.com shtora66.ru
skatestick.bid slavic-magic.ru
sklad-24.ru soblaznu.net
smartshoppymac.com splendorsearch-a.akamaihd.net
snip.tw staynplay.net
sockshares.tv strana-solnca.ru
sofit-dmd.ru strigkaomsk.ru
soochi.co strv.se
spyfu.com superinterstitial.com
spymac.net superlist.biz
strana-krasoty.ru swiped.su
stroiminsk.org takeflyte.com
surflinksmedical.com tam-gde-more.ru
the-usa-games.blogspot.com taqywu51.soup.io
thecoolimages.net theheroes.ru
thepantonpractice.co.uk thejournal.ru
thetoiletpaper.com themeforest.net
tivolibasket.it titelhelden.eu
tizanidine6mg.blogspot.com tizanidineinteractions.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidineoral.blogspot.com
tizanidinevscyclobenzaprine.blogspot.com tizanidinepain.blogspot.com
torrents-tracker.com
torrents.cd
toursmaps.com
traffmonster.info
trailer.cinemaflix.website trailer.cinemaflix.website
try-rx.com tuberkuleznik.ru
tt-ipd.info txxx.com
unacittaconte.org ultimatesetnewfreeallsoftupgradesystems.pw
unitexindia.com unitexindia.com
url-extractor.xyz univerfiles.com
vadimkravtcov.ru vandrie-ict.nl
vapsy.com
vchulkax.com
vereo.eu
via-gra.webstarts.com
viagroid.ru
videochat.ph videochat.ph
vilingstore.net videogamesecrets.com
villacoloniale.com videospornogratisx.net
vinsit.ru vip2ch.com
vip-parfumeria.ru virus-schutzmasken.de
volgograd.xrus.org viteonlusarezzo.it
vzlomfb.com vladimir.xrus.org
vzlomtw.com voluumtracker1.com
vzube.com vut.com.ru
waterefficiency.co wcb.su
westen-z.life webshoppermac.com
wma-x.com website-speed-checker.site
womensplay.net weightatraining.com
wygraj-skiny.win whiteproduct.com
xerox-douglas.cf wildcattube.com
xn----itbkqkfiq.xn--p1ai woman-h.ru
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai www.bookmaker-bets.com
xxxnatelefon.ru xclicks.net
za-fun-offer.com xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
za-music.mymobiplanet.com xrus.org
zolotoy-lis.ru xuki.us
zzlgxh.com xvideosporn.biz
yaminecraft.ru
zetmaster.ru
zoolubimets.ru
zzbroya.com.ua

10
dev-tools/test_units/ratelimittest-master.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.85 0; 10.1.1.35 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2717 ### Version: V4.2021.11.2718
### Updated: Mon Nov 15 22:01:37 UTC 2021 ### Updated: Tue Nov 16 10:03:30 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.11.2717 ### Version: V4.2021.11.2718
### Updated: Mon Nov 15 22:01:37 UTC 2021 ### Updated: Tue Nov 16 10:03:30 UTC 2021
### Bad Bot Count: 627 ### Bad Bot Count: 627
################################################### ###################################################
### Version Information ## ### Version Information ##