vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 02:29:58 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.12.2758 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-12-05 10:10:57 +00:00
parent 70ae5a44c9
commit 4bb9656933
47 changed files with 71052 additions and 71042 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2757 #### Version: V4.2021.12.2758
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2757 #### Version: V4.2021.12.2758
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.12.2757 #### Version: V4.2021.12.2758
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8744
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.12.2757 #### Version: V4.2021.12.2758
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8748
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8748
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8748
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8748
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 1; 10.1.0.180 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 0; 10.1.0.180 0;
127.0.0.1 0; 127.0.0.1 0;

8748
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 1; 10.1.0.180 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 0; 10.1.0.180 0;
127.0.0.1 0; 127.0.0.1 0;

8748
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 1; 10.1.0.180 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 0; 10.1.0.180 0;
127.0.0.1 0; 127.0.0.1 0;

8748
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2757 2758

8748
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8748
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
137.135.100.43 0; 52.165.29.153 0;

8748
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 1; 10.1.0.180 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 0; 10.1.0.180 0;
127.0.0.1 0; 127.0.0.1 0;

8748
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 1; 10.1.0.180 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 0; 10.1.0.180 0;
127.0.0.1 0; 127.0.0.1 0;

8748
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 1; 10.1.0.180 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 0; 10.1.0.180 0;
127.0.0.1 0; 127.0.0.1 0;

8748
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
137.135.100.43 0; 52.165.29.153 0;

8748
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
137.135.100.43 0; 52.165.29.153 0;

8748
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 1; 10.1.0.180 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
ALittle Client IDBTE4M
CheeseBot Iskanie
Claritybot LinkextractorPro
Cosmos MeanPath Bot
Crescent Pockey
GT::WWW QueryN Metasearch
MFC_Tear_Sample RankingBot
Picscout SurveyBot
Sogou web spider ZmEu
V-BOT trendiction.de

302
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404checker 360Spider
404enemy ALittle Client
80legs ASPSeek
Aboundex Abonti
Aboundexbot
AhrefsBot
AiHitBot AiHitBot
Alexibot Aipbot
Alligator AllSubmitter
Anarchie Anarchy99
Anarchy Ankit
Anthill Anthill
Apexoo
Asterias
Attach
AwarioRssBot
AwarioSmartBot
BBBike BBBike
BDFetch BDFetch
BLEXBot BackDoorBot
BackStreet
BackWeb BackWeb
Backlink-Ceck Badass
BacklinkCrawler
Battleztar Bazinga
Bigfoot
Bitacle Bitacle
BlackWidow Black Hole
Brandwatch Blackboard
Boardreader
Brandprotect
Buck Buck
Buddy
BuzzSumo BuzzSumo
CATExplorador
CODE87
Calculon
CazoodleBot CazoodleBot
CheeseBot
CherryPicker
ChinaClaw
Chlooe
Cliqzbot Cliqzbot
Cocolyzebot Collector
Cogentbot
Copier Copier
Cosmos
Craftbot Craftbot
CrazyWebCrawler Custo
Crescent DTS Agent
CrunchBot Deusu
DatabaseDriverMysqli Digincore
Dirbuster
Disco
Discobot
Discoverybot
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics
DomCopBot DomCopBot
DomainAppender DomainCrawler
DomainStatsBot DomainSigmaCrawler
Domains Project
EMail Wolf EMail Wolf
Ecxi EasyDL
Express WebPictures EirGrabber
Extractor EroCrawler
Extreme Picture Finder
Ezooms Ezooms
FDM
FHscan FHscan
Foobot Firefox/7.0
FlashGet
Fuzz Fuzz
FyberSpider FyberSpider
GetRight Fyrebot
Gigabot G-i-g-a-b-o-t
Genieo
GermCrawler
GoZilla GoZilla
GrabNet GrapeFX
Grafula
GrapeshotCrawler GrapeshotCrawler
HEADMasterSEO GridBot
HMView HTMLparser
HTTP::Lite
HaosouSpider
Havij Havij
Hloader Hloader
Humanlinks Humanlinks
HybridBot IDBTE4M
IDBot IDBot
IlseBot IRLbot
Image Fetch Iblog
Image Sucker Indy Library
InfoNaviRobot
InfoTekies InfoTekies
InterGET Internet Ninja
InternetSeer Iria
JOC Web Spider IstellaBot
JamesBOT JamesBOT
JennyBot JetCar
Jetty
JikeSpider JikeSpider
Joomla Larbin
Kenjin Spider Leap
Kozmosbot
Lanshanbot
LeechFTP LeechFTP
LeechGet
LexiBot
Lftp Lftp
LibWeb LibWeb
Lightspeedsystems Libwhisker
LieBaoFast
Likse
LinkScan LinkScan
Linkdexbot LinkWalker
LinqiaScrapeBot LinksManager
Lmspider LinqiaRSSBot
Lipperhey Spider
Ltx71 Ltx71
MFC_Tear_Sample
MIIxpc MIIxpc
MSFrontPage Mail.RU_Bot
MTRobot
Magnet
Majestic-SEO Majestic-SEO
MarkMonitor MarkMonitor
Mata Hari
MauiBot
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Metauri MicroMessenger
Minefield Microsoft Data Access
Morfeus Fucking Scanner Moblie Safari
Musobot Mojeek
NICErsPRO Mozlila
Mr.4x3
NPbot
Nameprotect Nameprotect
NearSite Needle
NetSpider NetAnts
NetMechanic
Netcraft Netcraft
Nettrack Nuclei
NextGenSearchBot Offline Explorer
Nibbler
Niki-bot
Ninja
Nutch
Octopus
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openfind Openfind
OrangeBot
OrangeSpider OrangeSpider
OutclicksBot
Page Analyzer Page Analyzer
PageGrabber PageAnalyzer
Pandalytics PageScorer
PeoplePal
Petalbot Petalbot
Picscout Picsearch
PictureFinder Pimonster
PleaseCrawl
ProPowerBot ProPowerBot
ProWebWalker Probethenet
PxBroker Psbot
PyCurl
QueryN Metasearch
Quick-Crawler
RSSingBot
RankActive
RankingBot RankingBot
RankingBot2 RankingBot2
Rankivabot Rankivabot
RankurBot
ReGet
Reaper Reaper
RedesScrapy RebelMouse
RepoMonkey Ripper
SEOkicks RocketCrawler
SEOlyticsCrawler SBIder
SEOkicks-Robot
SEOprofiler SEOprofiler
SalesIntelligent SEOstats
Scrapy SISTRIX
SemanticJuice ScanAlert
SemrushBot Screaming
SeoSiteCheckup ScrepyBot
SeobilityBot Semrush
Shodan
Siphon Siphon
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer SiteSnagger
Sitebeam
Siteimprove Siteimprove
Sitevigil Snapbot
Sociscraper
Sosospider Sosospider
Sottopop SpaceBison
Spanner
Spbot
Spinn3r Spinn3r
SputnikBot
Sqlmap Sqlmap
Sqworm
Steeler Steeler
Stripper
Sucker Sucker
Sucuri
SuperHTTP SuperHTTP
Swiftbot SurveyBot
Suzuran
Szukacz
T8Abot T8Abot
TeleportPro Teleport
Telesoft Telesoft
Telesphoreo
Thumbor Thumbor
TightTwatBot Toweyabot
Toata
Trendiction Trendiction
Trendictionbot Trendictionbot
Turnitin Turnitin
Twice VelenPublicWebCrawler
UnisterBot
Upflow
VB Project
VCI
Virusdie Virusdie
Voil VoidEYE
Voltron
WBSearchBot WBSearchBot
WEBDAV
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWW::Mechanize
WWWOFFLE
WeSEE WeSEE
Web Auto Web Collage
Web Fetch Web Enhancer
Web Fuck
Web Pix Web Pix
Web Sucker WebBandit
WebCopier WebCollage
WebEnhancer WebEnhancer
WebFetch WebFetch
WebFuck
WebLeacher
WebPix WebPix
WebReaper WebReaper
WebStripper WebStripper
WebSucker
WebZIP WebZIP
Webalta Webalta
WebmasterWorldForumBot
Webshag Webshag
Whack
Whatweb Whatweb
Who.is Bot Widow
WiseGuys Robot WinHTTrack
Wonderbot Wonderbot
Wotbox Xaldon WebSpider
Wprecon Zade
Zauba Zauba
Zeus
Zitebot
ZoomBot
ZumBot ZumBot
archive.org_bot ZyBorg
autoemailspider arquivo-web-crawler
backlink-check cah.io.community
check1.exe check1.exe
cognitiveseo cognitiveseo
crawl.sogou.com dataforseo.com
crawler.feedback eCatch
domainsproject.org facebookscraper
evc-batch heritrix
lwp-trivial lwp-trivial
meanpathbot
mediawords
netEstate NE Crawler netEstate NE Crawler
oBot
page scorer page scorer
pcBrowser pcBrowser
plumanalytics plumanalytics
probe-image-size
s1z.ru s1z.ru
scalaj-http
scan.lol
seobility seobility
seoscanners seoscanners
serpstatbot seostar
sexsearcher siteripz
sitechecker.pro
sogouspider sogouspider
sysscan
tAkeOut tAkeOut
voyagerx.com
webpros.com webpros.com
webprosbot
x09Mozilla x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe xpymep1.exe

294
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 404enemy
404checker ASPSeek
80legs Aboundex
Abonti
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AllSubmitter
Alligator
AlphaBot AlphaBot
Ankit Anarchy
Anthill Anarchy99
Attach Asterias
AwarioRssBot
BBBike BBBike
BDCbot BDCbot
BLEXBot BackDoorBot
BacklinkCrawler BatchFTP
Barkrowler BetaBot
Battleztar Bazinga
Bitacle Bitacle
BlackWidow
Blackboard Blackboard
BlowFish Blow
Boardreader Boardreader
Brandprotect Bolt
BuiltWith BunnySlippers
CATExplorador
CODE87
CazoodleBot
Cegbfeieh Cegbfeieh
CheeseBot CherryPicker
Chlooe ChinaClaw
Citoid
Claritybot Claritybot
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Cogentbot
Collector Copier
Copyscape
Crescent Crescent
CrunchBot Curious
Custo
CyotekWebCopy
DBLBot
DIIbot DIIbot
DatabaseDriverMysqli
Deusu Deusu
Digincore Disco
Discobot Discoverybot
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnyzBot
DomainSigmaCrawler DomainStatsBot
Domains Project Download Wonder
Dotbot
Dragonfly Dragonfly
ECCP/1.0 Drip
EMail Wolf EMail Siphon
EasyDL
Ebingbong Ebingbong
EirGrabber Exabot
EroCrawler ExtractorPro
Evil
ExtLinksBot
EyeNetIE
Ezooms
Fimap Fimap
Firefox/7.0 Fuzz
FlashGet
Fyrebot Fyrebot
G-i-g-a-b-o-t G-i-g-a-b-o-t
GermCrawler GalaxyBot
Gigabot Genieo
Go-Ahead-Got-It Grabber
Gotit GrapeshotCrawler
GridBot GridBot
HMView HEADMasterSEO
HTTP::Lite
Harvest
Havij Havij
Hloader
Humanlinks
IlseBot IlseBot
Image Fetch Image Fetch
IndeedBot
InfoNaviRobot
InfoTekies
InterGET InterGET
Internet Ninja IstellaBot
InternetSeer JamesBOT
JOC Web Spider
Jbrofuzz Jbrofuzz
Jetty Jetty
Jyxobot JikeSpider
Keyword Density Kenjin Spider
LNSpiderguy Kozmosbot
LWP::Simple
Lanshanbot Lanshanbot
Leap
LeechGet LeechGet
LexiBot
Lftp Lftp
Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse
LinkScan LinkScan
Linkdexbot
LinkpadBot LinkpadBot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaRSSBot
Lipperhey Lipperhey
Lipperhey Spider Litemage_walker
MFC_Tear_Sample MFC_Tear_Sample
MQQBrowser MIDown tool
MSFrontPage MIIxpc
MSIECrawler MSIECrawler
MTRobot
Mag-Net Mag-Net
Magnet Majestic SEO
Mail.RU_Bot Majestic12
Mb2345Browser MarkMonitor
MarkWatch
Meanpathbot Meanpathbot
MegaIndex.ru Mediatoolkitbot
Metauri
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Microsoft URL Control
Minefield Minefield
Mister PiX
Moblie Safari Moblie Safari
Mozlila Name Intelligence
Mr.4x3
Nameprotect Nameprotect
Navroad
Needle
Net Vampire Net Vampire
Netcraft NetMechanic
Netvibes Nettrack
Nibbler Nimbostratus
Ninja
Nuclei Nuclei
Offline Explorer Nutch
OnCrawl OpenLinkProfiler
OpenVAS Openvas
Openfind OrangeBot
OrangeSpider OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot PHPCrawl
Page Analyzer PageAnalyzer
PageGrabber
PageScorer
PageThing.com PageThing.com
Pandalytics
Panscient Panscient
Papa Foto
Pavuk Pavuk
Pi-Monster Picscout
Pixray PictureFinder
ProPowerBot Pimonster
Probethenet Psbot
Pu_iN
PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler
RSSingBot
RankFlex RankFlex
RankingBot
RankingBot2 RankingBot2
RankurBot Re-re
Recorder ReGet
SEOkicks-Robot RebelMouse
SEOstats RepoMonkey
SalesIntelligent SBIder
SMTBot
Scanbot Scanbot
ScoutJet ScreenerBot
Scrapy
ScrepyBot
SearchmetricsBot SearchmetricsBot
SemanticJuice
Semrush
SemrushBot SemrushBot
SentiBot SentiBot
SeobilityBot SeoSiteCheckup
Seomoz Seomoz
Siphon Siphon
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger
SiteSucker SiteSucker
Siteimprove Siteimprove
SmartDownload SlySearch
Snake
Snapbot
Snoopy
SocialRankIOBot SocialRankIOBot
Sosospider Sociscraper
Spammen
SpankBot SpankBot
Spanner Spanner
Spinn3r Spinn3r
SputnikBot SputnikBot
Sqlmap Steeler
Sqlworm Stripper
SuperBot Sucuri
SurveyBot Suzuran
Telesoft T8Abot
The Intraformant Teleport
TheNomad
Thumbor
Toata
Tracemyfile Tracemyfile
Trendiction
Turingos
Turnitin Turnitin
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Typhoeus Typhoeus
VB Project
VCI VCI
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Virusdie
WBSearchBot Voil
WEBDAV WASALive-Bot
WISENutbot WISENutbot
Web Enhancer WWW-Mechanize
Web Fetch WWWOFFLE
WebAuto Wallpapers/3.0
WallpapersHD
Web Auto
Web Collage
Web Fuck
Web Sauger
Web Sucker
WebCollage WebCollage
WebCopier WebFetch
WebFuck WebGo IS
WebImageCollector
WebLeacher
WebPix WebPix
WebZIP WebSauger
Webalta WebmasterWorldForumBot
Website Quester Website Quester
WebsiteQuester WebsiteQuester
Webster
Whacker Whacker
WiseGuys Robot Whatweb
Wonderbot Wonderbot
Woobot Woobot
Wprecon
Xaldon_WebSpider Xaldon_WebSpider
Xenu YoudaoBot
Zauba Zauba
Zermelo Zitebot
ZmEu ZmEu
ZoominfoBot ZumBot
archive.org_bot adscanner
arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider
cah.io.community
check1.exe
clark-crawler clark-crawler
coccocbot-web coccocbot-web
com.plumanalytics
dataforseo.com
domainsproject.org domainsproject.org
eCatch
evc-batch evc-batch
heritrix heritrix
instabid internetVista monitor
ips-agent isitwp.com
magpie-crawler lwp-trivial
meanpathbot mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot
page scorer page scorer
pcBrowser probe-image-size
ripz
s1z.ru s1z.ru
scalaj-http scan.lol
seobility seobility
seoscanners
seostar seostar
sitechecker.pro
siteripz siteripz
sogouspider sysscan
spyfu tAkeOut
trendiction.com trendiction.de
webmeup-crawler
webpros.com
webprosbot
x09Mozilla x09Mozilla
x22Mozilla
xpymep1.exe
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
advokateg.ru anime.dougasouko.com
bingo8888.com biglistofwebsites.com
diesel-parts28.ru brainboostingsupplements.org
hoverboard360.se errorfixing.space
jrpmakati.com f00kclan.de
rabot.host.sk ge0ip.net
s.lollypopgaming.com go2jump.org
scanner-jess.top konica.kz
sosdepotdebilan.com seoexperimenty.ru
sotechco.co vse-pesni.com

954
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03e.info 0daymusic.org
100searchengines.com 123cha.com
1bet.com 123locker.com
1webmaster.ml 1j7740kd.website
404.mn 2000k.ru
4replicawatch.net 23kw.ru
7xc4n.com 2itech.ru
8gold.com 3dracergames.com
a.frcls.fr 5kstore.com
aa9d046aab36af4ff182f097f840430d51.com 66cpwgln.space
aarbur.com 6hopping.com
abcdefh.xyz 72-news.com
abclauncher.com 883zy.com
abiente.ru 98oi.ru
acads.net 999webdesign.com
acgs.tk aa8780bb28a1de4eb5bff33c28a218a930.com
aa9bd78f328a6a41279d0fad0a88df1901.com
aab94f698f36684c5a852a2ef272e031bb.com
abbanreddy.com
abusora.com
actionnooz.com
adamoads.com adamoads.com
adexprts.com adanih.com
adman.se adconscious.com
ads-seo.men addblueoff.com.ua
ads.tremorhub.com
adsloads.com
advancedmassagebysara.com
advertisingtag.net
afftrack001.com afftrack001.com
agardomains.com aktivator-windows10.blogspot.com
ageofclones.com aktobe.xkaz.org
aghanyna.com
ahrntal.verymes.xyz
airlandsea.info
airmaxshoes-2016.com
albumsuper.info albumsuper.info
alert-fjg.xyz alive-ua.com
alice.it
aliexpresscashback.club
allbizne.co.ua
allblogroll.com
allpdfmags.net allpdfmags.net
alpinism.ru allstatesugarbowl.org
amateurmatch.com alphacarolinas.com
ameblo.top amazon-seo-service.com
anabolics.shop ameblo.jp
anal-acrobats.hol.es angry-fermi-7633.arukascloud.io
angigreene.com anniemation.com
animalia-life.club ap.senai.br
apibetweenlinesn-a.akamaihd.net apiitsthirteende-a.akamaihd.net
apper.de apiqualitinknet-a.akamaihd.net
appfastplay.com arate.ru
appsaurus.com ardimobileinfo.ml
apxeo.info arendatora.ru
arcadepages.com arendovalka.xyz
arquivo.pt arius.tech
arraty.altervista.org artparquet.ru
arvut.org asscenihotosidea.blogspot.co.za
atout-energie-69.com atout-energie-69.com
auto-moto-elektronika.cz ausmepa.org.au
autolombard-krasnodar.ru australia-opening-times.com
autoua.top autobudpostach.club
autowebmarket.com.ua avtolombard-krasnodar.com
aviva-limoux.com avtovolop.ru
avtointeres.ru ayodhya.co
ayakino.net azadnegar.com
azarttoto.com azbukadiets.ru
azazaporn.com
azazu.ru
azlex.uz azlex.uz
babosas.com backuperwebcam.weebly.com
badavit.com.ua bad-stars.net
balkanfarma.org bank.uz
basisches-wasser.net bankofthewext.com
beta.hotkeys.com bannerads.de
bio-market.kz belinka.com.ua
bioinnovate.co beremenyashka.com
bjanshee.ru bestwrinklecreamnow.com
blackhatworth.com bet-prognoz.com
bmsco.co betwinservice.com
bobinoz.com bigames.online
bonkers.name bimmerpost.com
bonvillan.com bioscorp.ru
bradleylive.xyz birzha-truda.eu
brandbucket.com biteg.xyz
brandimensions.com bloggen.be
bugshoot.cn blogsfunky672.weebly.com
buyk.host.sk bloke.com
buyscabiescream.com bmwhighperformers.com
bzero1jewelry.net brandedleadgeneration.com
cakesplus.com.au brendbutik.ru
callawaygolfstore.online buehne-fuer-menschenrechte.de
camdenmemorials.com buketeg.xyz
cand.jp burger-tycoon.com
canrole.com bus-offer.com
carabela.com.do businesn.men
cartierbraceletsreplica.pw buttons-for-your-website.com
cashonads.com buydissertation.net
cbozhe.com buzzzg.men
cctva.tv canacopegdl.com
celejihad.info carloans.com
cenoval.ru carsplate.com
centrdebut.ru cartierreplica.top
ceotrk.com cartierwatch.xyz
chatroulette.life cdnanalytics.xyz
chatroulette.video centraletermice.us
cheap-pills-norx.com cheapestjerseysonwholesale.com
cheap-trusted-backlinks.com chelyabinsk.xrus.org
checkm8.com chinacheapelitejerseys.com
christianlouboutinoutlet.win chocolateslim-en-espana.com
christianlouboutinreplica.win cialischmrx.com
cibpenonptib.flu.cc cialisovercounteratwalmartusa.com
cienum.fr circlesl.com
cinemaflix.website cjmc.info
clean-start.net cleanersoft.com
clothesforcash.com cobaltpro.ru
coldfilm.ru coin-hive.com
collegeessay19.blogspot.ru community.allhiphop.com
companies-catalog.com concealthyself.com
compliance-barak.top cookie-law-enforcement-ff.xyz
compliance-barak.xyz coolgramgoods.com
compliance-jessica.top copypaste.traderzplanet.in
compliance-josh.top crackguru.tk
compliance-mary.top
confib.ifmo.ru
contentlook.co
cookielawblog.wordpress.com
copblock.org
copro.pw
cornerstone-countertops.com
counter.bloke.com
credit-respect.ru credit-respect.ru
cryptoswap.biz custom-product-labels.com
culad.com cypernhuset.se
custom-electric-guitar.com dairyindia.in
cutt.us damasarenai.info
cvety24.by dame-ns.kz
d0t.ru dating-app.ru
ddospanel.com datract.com
decorationspcs.com deanmoore.ie
demenageur.com dear-diary.ru
dengi-pod-zalog-nedvizhimosti.ru deutschland123.de
deplim.com directrev.com
derevesendeco.com discovertreasurenow.com
detalizaciya-tut.biz diusyawiga.tk
dev78.dioniqlabb.se divku.ru
digest-project.ru djonwatch.ru
digital-video-processing.com dlya-android.org
dikqlyremy.info dmmspy.com
dosugrostov.site dojki365.online
drev.biz dokumentalkino.net
druzhbany.ru domainanalyzing.xyz
dstroy.su domainsphoto.com
dtm-spain.com donna7753191.ru
dtnlyss.com dood.live
duplicashapp.com download-wallpaper.net
dustyorate.com dvd-famille.com
dyt.net earthmagic.info
ecommerce-seo.org easync.io
ekosmetyki.net.pl econom.co
electrik-avenue.com educatemetv.com
elektrozigarette2.wordpressy.pl ekspertmed.com
emediate.eu el-nation.com
empis.magix.net elektrozigarettekaufen2.blogse.nl
elektrozigaretten2.yn.lt
enbersoft.com enbersoft.com
englate.com epngo.bz
enhand.se ereko.ru
esnm.ru
euronis-free.com euronis-free.com
ewebarticle.info extlinks.com
extremepornos.net extractorandburner.com
eyes-on-you.ga ezigarettekaufen1.hpage.com
ezigarettekaufen2.mpbloggar.se ezigarettekaufen2.yolasite.com
farmingworm.com facecup.top
favoritemoney.ru famix.xyz
femmesdenudees.com fandlr.com
fermersovet.ru faracontrol.ir
ferrotodo.com fashion-mk.net
fitfloponline.store fatmaelgarny.com
fix-website-errors.com favorcosmetics.com
flashbannernow.com feeriaclub.ru
flipper.top femdom.twiclub.in
floridamobilebillboards.com feorina.ru
foodgid.net fifa-coins.online
forum.tvmir.org financepoints.eu
fr-bearings.ru fishingwholesale.us
free-floating-buttons.com fishtauto.ru
free-share-buttons.com floridahuntingfishingadventures.com
frequiry.com forex-indextop20.ru
frivgame250.com franklinfire.co
futbolkisales.ru free-gluten.ru
galaxy-family.ru freshberry.com.ua
galaxyflowers.ru freshwallpapers.info
gambarkataku.co fsakhalin.ru
gay.adultgalls.com fuckmill.com
gcup.ru funponsel.com
ge0ip.net galaxys6manual.info
gearcraft.us gallerylisting.com
geileweiber.tk gamevalue7.weebly.com
getaclueamerica.com gamingspark.com
glasof.es gateway.zscalerone.net
gok-kasten.net gaytube.com
green-tea.tv gdebestkupit.ru
grizzlysgrill.com ge0ip.org
gsasearchenginerankerdiscount.com geckoandfly.com
guge.io get-free-social-traffic.com
handsandlegs.ru getprismatic.com
hasfun.com getridofstretchmarks.org
haveinc.xyz gg.zzyjxs.com
hermesbirkinhandbagoutlets.com ghernnqr.skyrock.com
hoholikik.club gilsonchiro.xyz
hostcritique.com glavprofit.ru
hoterika.com go2jump.org
houston-vikings.com goldandcard.ru
huhn.altervista.org goo.ne.jp
hyhj.info gq-catalog.gq
hyipmanager.in groupmoney.ru
hystersister.com gsasearchengineranker.xyz
i4track.net gyffu.com
ilbe.club habermetre.com
handicapbathtubarea.com
hdimagegallery.net
hentai-manga.porn
hermesbracelets.xyz
hifidesign.ru
historichometeam.com
hitmuzik.ru
homeandhealth.ru
horoshieokna.com
hostsshop.ru
hotchatdate.com
hotgirlhdwallpaper.com
hotshoppymac.com
howtoclean.club
howtowhitenteethfast.xyz
hscsscotland.com
humanorightswatch.org
igadgetsworld.com
imadedinner.net imadedinner.net
insomniagamingfestival.com imperiafilm.ru
invivo.hu incomekey.net
iqs.biz.ua interferencer.ru
isitwp.com iplogger.org
islamtoday.co.za iqoption.com
istizanidineanopiate.blogspot.com
istizanidinelikexanax.blogspot.com istizanidinelikexanax.blogspot.com
istmira.ru jackwolfskinoutlet.online
ivearchenceinflu.cf jav-idol.com
izoll.ru javcoast.com
jacago.com
janettabridal.com
jav-way.site
jerseywholesalebizchina.com
jerseywholesaleelitestore.com
jimmychoosale.online jimmychoosale.online
johnrobertsoninc.com kabbalah-red-bracelets.com
justbcause.com kaliningrad.zrus.org
kamalsinha.com karusel-market.ru
kambasoft.com kazka.ru
kamorel.com kevblog.top
kino-fun.ru keywordspay.com
kino-rf.ru kino-ecran.ru
kinozapas.org klumba55.ru
kit-opt.ru kochanelli.com
kladrus.ru kolotiloff.ru
klejonka.info konoplisemena.com
kmd-pto.ru kostenlos-sexvideos.com
knowyournextmove.com kostenloser-sex.com
kollekcioner.ru kraljeva-sutjeska.com
kopihijau.info kvartir-remont.biz
kozhniebolezni.com kwzf.net
kretpicf.pw lacasamorett.com
kung-fu-ru.com landinez.co
lafourmiliaire.com laulini.soclog.se
landmania.ru lawyersinfo.org
lapolis.it life-instyle.com
laptop-4-less.com lion.bolegapakistan.com
lazy-z.com liran-locks.com
leadn.pl livingroomdecor.info
lecbter-relationships.ga loadopia.com
ledis.top lomza.info
lflash.ru luckyday.world
liffa.ru luxup.ru
limtu.ifmo.ru magazintiande.ru
linerdrilling.com mamasuper.prom.ua
lion.misba.us managerpak204.weebly.com
liquimondo.com mantramusic.ru
listiseltemournlan.gq marcoislandvacations.net
livingcanarias.com maridan.com.ua
ljusihus.se matpre.top
lovi-moment.com.ua matsdale.com
lsp-awak-perikanan.com mebelcomplekt.ru
luciddiagnostics.in meetlocalchicks.com
lutherstable.org microsearch.ru
m5home.ru mobicover.com.ua
makis.nu mobilemedia.md
malls.com mobot.site
masthopehomes.com mobplayer.ru
maxxtor.eu mockupui.com
memberty.com
mesto-x.com
methodsmarketing.com
mezaruk.info
michaelkorsoutlet.store
millionare.com
mirtorrent.net
mmostrike.ru
mojaocena.com
moje-recenze.cz
monclerjacketsoutlet.win monclerjacketsoutlet.win
moncleroutletonline.xyz moneyzzz.ru
monclervests.xyz montredemarque.nl
monetizer.com-01.site morpicert.pw
moneytop.ru mossmesi.com
monsterdivx.tv mototsikl.org
mountainstream.ms moz.com
moyaterapiya.ru muschisexbilder.com
mp3ringtone.info musirc.com
mrbojikobi4.biz mykings.pw
muizre.ru mylida.org
mulberryoutletonlineeu.com naturalpharm.com.ua
musicas.baixar-musicas-gratis.com nelc.edu.eg
mvpicton.co.uk
mybackgroundlandscape.blogspot.com
mycaf.it
myhealthcare.com
myhitmp3.club
natali-forex.com
neobux-bg.info neobux-bg.info
netvouz.com netvouz.com
news-speaker.com new-post.tk
november-lax.com newfilmsonline.ru
nuit-artisanale.com next-dentists.tk
nw-servis.ru nextconseil.com
oballergiya.ru nextlnk12.com
oecnhs.info nfljerseyscheapbiz.us
offer.wpsecurity.website nmrk.ru
oklogistic.ru northfacestore.online
onlinedomains.ru nudo.ca
ons-add.men o-o-6-o-o.com
onstrapon.purplesphere.in o-o-8-o-o.ru
origin-my.ru oakridgemo.com
painting-planet.com obiavo.su
paparazzistudios.com.au obuv-kupit.ru
ochistka-stokov.ru
oda.as
odywpjtw.bloger.index.hr
offers.bycontext.com
okout.ru
olgacvetmet.com
onblastblog.online
oneclickfiles.com
online-sbank.ru
online.ktc45.ru
onlinefilmz.net
orion-v.com
partner-high.men
partner-host.men partner-host.men
pcboa.se pawli.eu
pdns.download pekori.to
physfunc.ru percin.biz.ly
pics-group.com pertlocogasilk.tk
pictures-and-images.com pestomou.info
pila.pl pflexads.com
pills24h.com phobia.us
pinkduck.ga picurams.pw
piuminiita.com pihl.se
pivka.xyz piter.xrus.org
pizza-imperia.com pochemychka.net
plaff-go.ru pogodnyyeavarii.gq
plastgranar.nu polybuild.ru
pogosh.com
pons-presse.com pons-presse.com
popcash.net popander.mobi
poptool.net popugaychiki.com
popup-jdh.xyz poquoson.org
porno-raskazy.ru pornhubforum.tk
pornokajf.com pornohd1080.online
pornosmola.info pornosexrolik.com
pornozhara.com powenlite24.ru
pornzone.tv preparevideosafesystem4unow.site
pozdrawleniya.ru producm.ru
prizestohandle.club productarium.com
promover.org profitsport.club
proprostatit.com progress-upakovka.ru
prosperent.com psa48.ru
ptr.ruvds.com pussyspace.net
pyrodesigns.com.au qld10000.net
qcstrtvt.bloger.index.hr quebec-bin.com
queerspace.com quick-offer.com
r-control.ru r-e-f-e-r-e-r.com
rankia.com rankexperience.com
ranksonic.info rebelmouse.com
rapidsites.pro redirectingat.com
rating-bestcasino.com refererx.com
razamicroelectronics.com remedyotc.com
redhotfreebies.co.uk renecaovillasale.online
refads.pro rospromtest.ru
referencemoi.com
registrydomainservices.com
remont-komputerov-notebook.ru
remontgruzovik.ru
renhacklids.tk
rentalcarnavi.info
responsinator.com
resultshub-a.akamaihd.net
ring4rhino.com
ringtonepartner.com
ritlweb.com
royal-investments.net royal-investments.net
rus-teh.narod.ru royalcar-ufa.ru
ruscopybook.com ru-dety.ru
salut-camp.ru ru-mediaget.ru
sbdl.no rutor.vip
scanner-julianna.top ryetaw.com
scenicmissouri.us sabizonline.com
sdelatmebel.ru saecsa.co
se.bnt-team.com salmonfishingsacramentoriver.com
segol.tv sampleletters.net
sei80.com sankt-peterburg.nodup.ru
serialsway.ucoz.ru santasgift.ml
sexkontakteao.info sarafangel.ru
sexkrasivo.net saratov.xrus.org
sexpartygirls.net sashagreyblog.ga
sexreliz.com score-ads.men
share-buttons-for-free.com securesmrt-dt.com
shikiso.info securityallianceservices.com
shopsellcardsdumps.com semalt.com
slavia.info seo-prof1.xyz
slonechka.ru sergiorossistore.online
slowmacfaster.trade serviporno.com
servisural.ru
sexs-foto.top
sharebutton.to
shiksabd.com
shymkent.xkaz.org
sierraapps.com
signx.info
sispe.com.br
site5.com
skanninge.se
skladvaz.ru
slkrm.ru
sloopyjoes.com
smichovbike.cz smichovbike.cz
smsactivator.ru snsdeainavi.info
snip.to socialbookmarksubmission.org
snyatie-lomki-v-stacionare.ru socialseet.ru
soft-terminal.ru soda.media
softomix.net softomix.net
sogimlecal.tk solnplast.ru
spb.ru sortthemesitesby.com
specialfinanceoffers.com souvenir.cc
spidtest.org spl63.fr
spidtest.space sports-supplements.us
sps-shop.com spywarebegone.com
statoutlook.info star61.de
staynplay.net stop-zavisimost.com
streetfooduncovered.com stroyplus.ru
summerlinhomes411.info styro.ru
susanholtphotography.com sugarkun.com
swsociety.se sundrugstore.com
tarad.com surffoundation.nl
taxi-v-eisk.ru surgut.zrus.org
td-33.ru svensk-poesi.com
teguh.info svetlotorg.ru
tengohydar.tk taaaak.com
terraclicks.com tackletarts.co
tfxiq.com takeflyte.com
thebluffs.com tattomedia.com
theguardlan.com tedxrj.com
thruport.com templates.franklinfire.co
tizanidineforopiatewithdrawal.blogspot.com templates.radiodigital.co
tizanidineorflexeril.blogspot.com terrafootwear.us
tizanidinepills.blogspot.com thegalerie.eu
tizanidinestreetprice.blogspot.com tiandeural.ru
tkanorganizma.ru timer4web.com
tomatis.gospartner.com tizanidinedosageforsleep.blogspot.com
tongkatmadura.info tizanidineduringpregnancy.blogspot.com
topquality.cf tizanidinerestlesslegsyndrome.blogspot.com
torrentdownloadhub.com tizanidinezanaflex.blogspot.com
totu.us tksn.ru
tour-line.net topappspro.com
tourcroatia.co.uk toproadrunner5.info
tubo360.com torrentgamer.net
tv-spoty.info tradedeals.biz
tvorozhnaja-zapekanka-recept.ru truebeauty.cc
twojebook.pl truemfilelj.gq
typimga.pw tubeoffline.com
ua-company.ru uchetunet.su
ufa.xrus.org uggbootsoutletsale.us
ukr-lit.com ugogo.info
ukkelberg.no
ukrtvir.com.ua ukrtvir.com.ua
unlimitdocs.net ultimateclassicrock.com
urlopener.blogspot.com.au umnovocaminho.com
urlopener.com unrealcommander.biz
uroffer.link uptime-delta.net
usbggettwku.ga urll.eu
usiad.net
uzporno.mobi uzporno.mobi
vacuumscleaner.com vadimkravtcov.ru
valsalud.com vandrie-ict.nl
valueclick.com vapeface.club
vancleefreplica.pw vegan-foods.us
vashsvet.com video-chat.cn
vbtracker.net videochat.ph
vedomstvo.net vipromoffers.com
vellings.info visionwell.com.cn
viddyoze.com vitanail.ru
video-production.com vizitki.net
vip-dom.in vkontarkte.com
virtuagirl.com vodaodessa.com
virtualbb.com voloslove.ru
virus-schutzmasken.de vonradio.com
vseuznaem.com vysigy.su
vzlom-na-zakaz.com warning.or.kr
vzube.com warningzscaler.heraeus.com
walpaperlist.com webs.com
warezkeeper.com wesharepics.site
webads.co.nz whatistizanidinehclusedfor.blogspot.com
webcamdevochka.com where-toget.com
webjam.com whereiskentoday.com
website-analytics.online whipme.yopoint.in
wesharepics.info withstandingheartwarming.com
wikes.20fr.com wleuaprpxuvr.ga
wildnatureimages.com worldmusicfests.com
wjgony.com worst-sites.online
wtsindia.in woweb.com.ua
ww1943.ru www.pinnacle-bets.com
www.arenda-yeisk.ru www.timer4web.com
www.kabbalah-red-bracelets.com x-diesel.biz
xlovecam.com xep.info
xmladserver.com xev.ru
xn----9sbubg3ambdfl1j.xn--p1ai xn--h1aakne2ba.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xnxx699.com
xn--90acenikpebbdd4f6d.xn--p1ai xvideosbay.com
xnxxandxvideos.com y8games-free.com
xxxrus.org yaderenergy.ru
xzlive.com
yaminecraft.ru yaminecraft.ru
yaoguangdj.com yebocasino.com
yatrk.xyz ynymnwbm.bloger.index.hr
yeartwit.com yopoint.in
yoopsie.com youtubedownload.org
yottos.com zacreditom.ru
ytmnd.com zastenchivosti.net
z-master.ru zigarettenonlinekaufen1.bloog.pl
zakazfutbolki.com zrus.org
zarabotok--doma.ru zygophyceous.womanstars.site
zed21.net
zigarettenonlinekaufen2.drupalgardens.com
zixizop.net.ru
znakomstva-moskva77.ru
zoominfo.com
zophim.me

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
08800.top 101billion.com
100searchengines.com 1xbet4you.com
1hwy.com 28n2gl3wfyb0.ru
1qingdao.com 4inn.ru
8558l.jobs.net 8jn.dating-app.ru
91abcw.com addbin.men
991mostfm.co.id agardomains.com
adimmix.com all-number.com
afftrack001.com alldezservice.kz
anabolics.shop allproblog.com
am15.net
amanda-porn.ga
amazingpic.net
androids-store.com
apiholdingmypage-a.akamaihd.net
approved.su
apptester.tk
asiengirls.net asiengirls.net
atmagroup.ru asrvvv-a.akamaihd.net
aviav.eu australia-opening-times.com
azartmix.com aversis.net
backuperwebcam.weebly.com avon-novosib.ru
baladur.ru azazu.ru
bcmp.org babieca.com
bang-hotties.com
battleforupdating.site
bavsac.com
beatifullandscape.co beatifullandscape.co
bensbargains.net bget.ru
bestmarriages.com bharatdefencekavach.com
betterhdporn.com biketank.ga
bidbuy.co.kr bimatoprost-careprost.com.ua
bif-ru.info bio-market.kz
bikini-image.com blogsfunky672.weebly.com
brainxs.us bookhome.info
brainzpod.win bosman.pluto.ro
bristolhostel.com brains2.biz
bscodecs.com butstrap.space
buyessay3.blogspot.ru c-gainsbourg.com
buyparajumpers.online cannibalcheerleader.com
callstevens.com cardul.ru
canoncdriverq3.pen.io cartierlovestore.com
cartierlove2u.com cartierwatch.xyz
cartierring.xyz
cascadelink.org cascadelink.org
cerev.info casinos4dummies.co.uk
chorus.terakeet.com cherrypointplace.ca
cineacademy.ru chiblackhawks-jerseys.com
cjs.com.ru chocolateslim-u-srbiji.com
clicksor.net cityadspix.com
clknsee.com cleanallspyware.com
clothesforcash.com coin-hive.com
coaching-netz.info compliance-alexa.top
coderstate.com
compliance-barak.top
compliance-fred.top
consorzioilmosaico.org
cookie-law-enforcement-ff.xyz cookie-law-enforcement-ff.xyz
czeshop.info copblock.org
dengi-pod-zalog-nedvizhimosti.ru custom-electric-guitar.com
deplim.com ddospanel.com
dikx.gdn dekoration.us
documentsite.net
dogclothing.store
dokfilms.net dokfilms.net
domain-tracker.com domaineaneblanc.com
domoysshop.ru domination.ml
e-biznes.info downloadkakaotalk.com
e-commerce-seo.com dvd-famille.com
earthmagic.info egvar.net
ednorxmed.com election.interferencer.ru
ee77ee.com elektrischezigarette2.onsugar.com
epicbrogaming.com empis.magix.net
everypony.ru englishgamer.com
excitacao.com enskedesquashclub.se
exoneration-project.us ereko.ru
expolicenciaslatam.co essaypro.com
eyes-on-you.ga eurosamodelki.ru
facebook-mobile.xyz expresstoplivo.ru
fastcrawl.com ezigarettenshop1.yolasite.com
faptitans.com
farm26.ru
fellowshipoftheminds.com fellowshipoftheminds.com
ferrotodo.com flash4fun.com
filmgo.ru forex-indextop20.ru
finejewelryshop.ru freemags.cc
forsex.info freshmac.space
fotzen-ficken.com galaxy-family.ru
foxtechfpv.com garciniaxt.us
fuck-paid-share-buttons.xyz gdebestkupit.ru
funponsel.com ge0ip.com
gamerextra.com geilehausfrauen.net
getaclueamerica.com gembird.com
getoutofdebtfree.org getdot.ru
gidonline.one ghazel.ru
given2.com gilsonchiro.xyz
glavprofit.ru girlporn.ru
gne8.com gogps.me
gojiberriess.apishops.ru goldadpremium.com
gorabagrata.ru gsbs.com.ua
graphicwe.org heroz.fr
growboxbl.ru hotsocialz.com
guod.me idegenvezeto.eu
gymi.name ilmen.net
havepussy.com inet-traffic.com
hdimagelib.com iqs.biz.ua
heliko.no isoveti.ru
histats.com ispac.org
homeandhealth.ru jackwolfskinoutlet.online
hoste.octopis.com jav-idol.com
hotel-mkad.ru joessmogtestonly.com
housemilan.ru kidskunst.info
howopen.ru kinobaks.com
hvd-store.com kinocccp.net
hystersister.com kleine-titten.biz
ideawheel.com klumba55.ru
img.wallpaperstock.net kozhniebolezni.com
import-sales.com lanasshop.ru
jmat.cn laserpen.club
jus0wil.pp.ua le-clos-des-alouettes.com
kiskinhouse.com lezbiyanki.net
kit-opt.ru lietaer.com
kovesszucs.atw.hu lineshops.biz
kursy-ege.ru linkarena.com
kurwa.win linkpulse.com
lapolis.it loadingpages.me
laptoper.net lombia.co
ledis.top marcogrup.com
leftofcentrist.com marinetraffic.com
lenvred.org max-p.men
lion.conboy.us microstatic.pl
lion.snapmap.us mjchamonix.org
lmrauction.com mompussy.net
luckyshop.net.ua monclerjacketsoutlet.pw
lunchrock.co monclerjacketsoutlet.win
machicon-ueno.info moneytop.ru
macresource.co.uk moroccosurfadventures.com
magnetpress.sk mp3downloadhq.com
marktforschung-stuttgart.com msk.afora.ru
matchpal-a.akamaihd.net multgo.ru
mazda-roadsters.com musezone.ru
medicines-choice.com musicpro.monster
mitrasound.ru myhydros.org
moinozhki.com nextbackgroundcheck.gq
mrbitsandbytes.com norththeface.store
mulberryoutletonlineeu.com notebook-pro.ru
myhitmp3.club npoet.ru
naturalshair.site nyfinance.ml
next-dentists.tk ochistka-stokov.ru
nisuturnetdgu.tk orgasmus-virtual.com
njkmznnb.ru pacificair.com
nodding-passion.tk parvezmia.xyz
obiavo.ru photochki.com
oddamzadarmo.eu phpdevops.com
orsonet.ru
ownshop.win
oyster-green.com
paccohichetoti.ml
pagesense.com
palvira.com.ua
panamaforbeginners.com
panicatack.com
park.above.com
perfectpracticeweb.com
pic2fly.com
pijoto.net pijoto.net
pinsdaddy.com pipki.r.acdnpro.com
pizdopletka.club pk-pomosch.ru
play.leadzupc.com playtap.us
plugingeorgia.com pojdelo.weebly.com
pp-budpostach.com.ua polimga.pw
pr-ten.de poptool.net
prod2016.com pornmania.pl
productarium.com porno-home365.com
pulseonclick.com porno-play.net
quickloanbank.com pornophoto.xyz
radiogambling.com pornoxxx.com.mx
rankia.com pornozhara.com
razamicroelectronics.com portnoff.od.ua
rcpmda.ikan1080.xyz positive2b.ru
reckonstat.info privatefx.all4invest.info
redirect.trafficreceiver.club pro-tec.kz
remont-mobile-phones.ru proftests.net
renecaovilla.online qualitymarketzone.com
rentaremotecomputer.com raavidesigns.com
rff-cfal.info respublica-otel.ru
santechnik.jimdo.com roma-kukareku.livejournal.com
royal-betting.net
royalads.net
rrutw.com
ruclicks.com
ruex.org.ua
ryetaw.com
sandhillsonline.com
santasgift.ml
satellite.maps.ilovevitaly.com
saugatuck.com saugatuck.com
sawin.beth.webd.pl saveriopiazza.it
scanner-brian.top
scanner-donald.top scanner-donald.top
scanner-jane.top scopich.com
scanner-margo.top sdi-pme.com
scanner-mary.top service.adtech.us
scanner-viktor.top sexystrippe.info
search.1and1.com shtaketniki.kz
seo-traffic-ranking.info sindragosa.comxa.com
seo18.su smart-scripts.com
serpstat.com snip.tw
service-core.ru sobecjvuwa.com.ru
sex-tracker.de softomix.com
shmetall.com.ua soundfrost.org
simple-share-buttons.com spywarespy.com
sinel.info stats-public.grammarly.io
sitiz.club stroimajor.ru
skytraf.xyz sugarkun.com
slowmacfaster.trade telefonsexi.com
softonicads.com thegalerie.eu
speeddream.xyz timdreby.com
sstroy44.ru titangel-vietnam.com
stair.registrydomainservices.com tivolibasket.it
steame.ru tizanidineandhydrocodone.blogspot.com
stockspmb.info tizanidinemusclerelaxant.blogspot.com
surfbuyermac.com top-karkas.ru
swiped.su top10-way.com
szamponrevita.pl topvidos.ru
teslathemes.com torrent-newgames.com
thetardistimes.ovh trafficstars.com
tizanidine4mgtablets.blogspot.com traffixer.com
tizanidinecapsules.blogspot.com tripper.de
toloka.hurtom.com truebeauty.cc
top1-seo-service.com tula.howotorg.ru
topsy.com twiclub.in
torontoplumbinggroup.com ukrtextbook.com
torrentdownloadhub.com unimodemhalfduplefw.pen.io
tracksz.co url-img.link
tri-slona.org vgoloveboli.net
tubeoffline.com viandpet.com
u555u.info
vapeface.club
varikozdok.ru
vbikse.com
vduplo.ru
vepad.com
vesnatehno.com
video-chat.cn
videtubs.pl
viktoria-center.ru
vipromoffers.com vipromoffers.com
w3data.co virtualbb.com
warezaccess.com vseigru.one
website-audit.com.ua vtcdns.com
websiteexplorer.info wallpaperstock.net
wfb.hatedriveapart.com wapsite.me
whiteproduct.com warningzscaler.heraeus.com
webs.com
webstatsdomain.org
weclipart.com
wesharepics.com
wholesalecheapjerseysfree.com wholesalecheapjerseysfree.com
wholesalejerseychinashop.com witherrom55.eklablog.fr
win-spy.com wma-x.com
womama.ru ww1943.ru
world-mmo.com x-rates.ru
wpthemedetector.co.uk x-stars.ru
x-mix.info xcc24.pl
xaylapdiendanang.com xn--80ak6aa92e.com
xep.info xn--h1ahbi.com.ua
xn--80aafb2a.xn--p1ai xxxkaz.org
xn--90acenikpebbdd4f6d.xn--p1ai xxxsiterips.xyz
xn--c1acygb.xn--p1ai youthreaders.com
xxxdatinglocal.us zakonobosago.ru
youdao.com zarenica.net
yourporn.com zeroredirect5.com
yurgorod.ru zigarettenonl.canalblog.com
zarabotat-na-sajte.ru zmoda.hostreo.com
zophim.me znakomstva-piter78.ru
zoolubimets.ru
zscaler.net

10
dev-tools/test_units/ratelimittest-beta.txt
View file

@ -581,4 +581,14 @@
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1> <h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p> <p>This is merely used for testing !!!</p>
</body> </body>
</html><!DOCTYPE html>
<html>
<head>
<title>Welcome to NginX</title>
</head>
<body>
<div align=center>
<h1>Welcome to the NginX Ultimate Bad Bot Blocker</h1>
<p>This is merely used for testing !!!</p>
</body>
</html> </html>

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.162 0; 10.1.0.180 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2757 ### Version: V4.2021.12.2758
### Updated: Sat Dec 4 22:01:10 UTC 2021 ### Updated: Sun Dec 5 10:04:29 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2757 ### Version: V4.2021.12.2758
### Updated: Sat Dec 4 22:01:10 UTC 2021 ### Updated: Sun Dec 5 10:04:29 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##