vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 04:00:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.12.2798 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-12-21 22:07:15 +00:00
parent 89c9e488f9
commit e1efe880ba
46 changed files with 88751 additions and 88751 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2797 #### Version: V4.2021.12.2798
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2797 #### Version: V4.2021.12.2798
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.12.2797 #### Version: V4.2021.12.2798
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10960
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.12.2797 #### Version: V4.2021.12.2798
#### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7086](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10964
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10964
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10964
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10964
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 1; 10.1.1.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 0; 10.1.1.11 0;
127.0.0.1 0; 127.0.0.1 0;

10964
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 1; 10.1.1.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 0; 10.1.1.11 0;
127.0.0.1 0; 127.0.0.1 0;

10964
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 1; 10.1.1.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 0; 10.1.1.11 0;
127.0.0.1 0; 127.0.0.1 0;

10964
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2797 2798

10964
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10964
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.77.157.39 0; 40.77.7.223 0;

10964
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 1; 10.1.1.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 0; 10.1.1.11 0;
127.0.0.1 0; 127.0.0.1 0;

10964
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 1; 10.1.1.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 0; 10.1.1.11 0;
127.0.0.1 0; 127.0.0.1 0;

10964
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 1; 10.1.1.11 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 0; 10.1.1.11 0;
127.0.0.1 0; 127.0.0.1 0;

10964
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.77.157.39 0; 40.77.7.223 0;

10964
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.77.157.39 0; 40.77.7.223 0;

10964
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 1; 10.1.1.11 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Claritybot Black Hole
DIIbot EirGrabber
InterGET Fyrebot
LeechGet Linkbot
Meanpathbot Mr.4x3
RepoMonkey Probethenet
WebStripper Telesoft
Zitebot WWWOFFLE
scan.lol clark-crawler
xpymep1.exe lwp-trivial

288
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404checker 404checker
404enemy
80legs 80legs
Aboundexbot ALittle Client
Acunetix ASPSeek
AfD-Verbotsverfahren AfD-Verbotsverfahren
Aipbot AiHitBot
AllSubmitter
AlphaBot
Anarchie
Anarchy Anarchy
Anarchy99 Anarchy99
Ankit
Anthill
Apexoo Apexoo
Aspiegel AwarioRssBot
Asterias BDCbot
AwarioSmartBot BDFetch
BBBike
BackDoorBot BackDoorBot
BackWeb BackStreet
Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga Bigfoot
Boardreader Blackboard
Brandprotect Blow
Buck BotALot
Buddy Brandwatch
BuiltWith
BunnySlippers BunnySlippers
CATExplorador
CCBot CCBot
CSHttp CSHttp
CherryPicker Cegbfeieh
Chlooe CensysInspect
CheTeam
ChinaClaw
Citoid Citoid
Cloud mapping Claritybot
Collector Cliqzbot
Copyscape Cogentbot
Crescent CopyRightCheck
CrunchBot Crawling at Home Project
Curious Curious
Demon CyotekWebCopy
DBLBot
DataCha0s
DatabaseDriverMysqli
Deusu
DigitalPebble DigitalPebble
DnBCrawler-Analytics Disco
Discoverybot
Dispatch
DnyzBot
DomainAppender DomainAppender
DomainCrawler
Domains Project Domains Project
Download Wonder Dragonfly
Drip Drip
ECCP/1.0 ECCP/1.0
EMail Siphon
Ebingbong Ebingbong
Ecxi Ecxi
Evil
Exabot Exabot
Express WebPictures Express WebPictures
Extractor Extractor
Extreme Picture Finder Extreme Picture Finder
EyeNetIE EyeNetIE
FemtosearchBot FHscan
Fimap Fimap
FlashGet Firefox/7.0
Flunky Flunky
FrontPage FrontPage
Fuzz FyberSpider
Fyrebot
GT::WWW GT::WWW
GetRight Getintent
GetWeb Gigabot
Go!Zilla Go!Zilla
Grafula Grabber
GrapeshotCrawler
HEADMasterSEO
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
HTTrack Harvest
Haansoft Humanlinks
Havij
Heritrix
Hloader
HybridBot
IDBTE4M IDBTE4M
Iblog Image Sucker
Id-search Indy Library
InfoNaviRobot
InterGET InterGET
InternetSeer
JOC Web Spider
JamesBOT
JennyBot JennyBot
JetCar JetCar
JikeSpider Jetty
JustView
Jyxobot
Kinza
LNSpiderguy LNSpiderguy
Lanshanbot Lanshanbot
Larbin
Leap Leap
LexiBot
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkScan
LinkWalker
Linkdexbot Linkdexbot
LinkpadBot LinkextractorPro
LinqiaMetadataDownloaderBot Lipperhey
LinqiaScrapeBot Lipperhey Spider
Litemage_walker
Lmspider Lmspider
MQQBrowser MQQBrowser
MSFrontPage MSFrontPage
MTRobot
Mag-Net
Magnet Magnet
Majestic-SEO Majestic12
MarkWatch Masscan
MegaIndex.ru Mediatoolkitbot
Metauri Metauri
Mister PiX MicroMessenger
Minefield
Moblie Safari
Mojolicious
Mozlila
Musobot Musobot
NPbot Nameprotect
Name Intelligence Navroad
NearSite Needle
NetAnts
NetLyzer
NetZIP NetZIP
Netcraft Nettrack
Netvibes
Nibbler Nibbler
Niki-bot Ninja
NimbleCrawler Not
Nuclei
Nutch Nutch
Offline Navigator
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS
OrangeSpider
OutclicksBot OutclicksBot
PECL::HTTP PECL::HTTP
POE-Component-Client-HTTP
PageScorer PageScorer
PeoplePal PageThing.com
Pandalytics
Panscient
Piepmatz Piepmatz
Pimonster Pimonster
Pixray Pixray
Pockey QueryN Metasearch
ProPowerBot Quick-Crawler
ProWebWalker RankActive
Probethenet RankActiveLinkBot
Psbot
Pu_iN
Pump
PyCurl
RankFlex RankFlex
Rankivabot RankingBot
RankurBot RebelMouse
ReGet RedesScrapy
RealDownload RepoMonkey
Recorder Ripper
SBIder SEOstats
Scanbot SISTRIX
Screaming ScanAlert
ScreenerBot ScoutJet
Scrapy
Searchestate
SearchmetricsBot SearchmetricsBot
SemanticJuice
Semrush
SentiBot SentiBot
SeobilityBot SeobilityBot
Seomoz Seomoz
SiteExplorer Site Sucker
SiteSnagger
Sitevigil
Snapbot Snapbot
Snoopy Snoopy
SocialRankIOBot
Sociscraper Sociscraper
Sosospider Sogou web spider
SputnikBot SputnikBot
Sqlmap
Sqworm Sqworm
Steeler Steeler
Sucker
Sucuri Sucuri
SuperBot SuperHTTP
Surfbot Surfbot
SurveyBot SurveyBot
T0PHackTeam Suzuran
T8Abot T8Abot
Telesoft Telesoft
TheNomad Telesphorep
Thumbor The Intraformant
TightTwatBot Trendiction
Tracemyfile
Trendictionbot Trendictionbot
Turingos
Turnitin
TurnitinBot TurnitinBot
Twice
Typhoeus
URLy Warning
URLy.Warning
UnisterBot UnisterBot
Upflow
V-BOT
VCI VCI
Vacuum VidibleScraper
VelenPublicWebCrawler Virusdie
Voltron Voltron
WEBDAV WASALive-Bot
WISENutbot WWW::Mechanize
WPScan
WWWOFFLE
Wallpapers
Wallpapers/3.0
Web Pix
Web Sauger
WebBandit
WebCopier
Web Fuck Web Fuck
Web Pix Web Pix
WebAuto
WebEnhancer
WebImageCollector
WebLeacher
WebReaper WebReaper
WebSauger
WebStripper
WebSucker WebSucker
WebZIP Webalta
Webshag Website Quester
Whack WebsiteExtractor
Who.is Bot Whacker
Widow Widow
WinHTTrack WinHTTrack
WiseGuys Robot
Wotbox Wotbox
Wprecon
Xaldon WebSpider Xaldon WebSpider
Zauba YoudaoBot
Zade
Zermelo Zermelo
Zeus
Zitebot Zitebot
ZumBot
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt arquivo.pt
autoemailspider backlink-check
check1.exe
clark-crawler clark-crawler
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler4j crawler4j
demandbase-bot
domainsproject.org domainsproject.org
eCatch eCatch
facebookscraper
gopher gopher
heritrix heritrix
instabid
ips-agent ips-agent
isitwp.com
lwp-request
lwp-trivial
magpie-crawler magpie-crawler
meanpathbot
mediawords mediawords
muhstik-scan oBot
netEstate NE Crawler
page scorer page scorer
polaris version plumanalytics
probe-image-size ripz
s1z.ru scalaj-http
satoristudio.net
seobility seobility
seoscanners
seostar seostar
serpstatbot sysscan
sitechecker.pro trendiction.com
spyfu
tAkeOut
trendiction.de trendiction.de
voyagerx.com ubermetrics-technologies.com
webgains-bot
webmeup-crawler
webpros.com
x09Mozilla
x22Mozilla

288
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
80legs 360Spider
AIBOT 404enemy
Aboundexbot Aboundex
Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
Alexibot AiHitBot
Aipbot
AllSubmitter
Alligator Alligator
AlphaBot
Anarchie Anarchie
Ankit Anarchy99
Anthill
Apexoo Apexoo
Asterias Attach
AwarioRssBot BBBike
BDCbot BackStreet
BDFetch Badass
BackDoorBot Barkrowler
Backlink-Ceck Black Hole
BacklinkCrawler Blow
Battleztar Bazinga Boardreader
BetaBot
Bigfoot
BotALot BotALot
Brandwatch Buck
Buddy
BuiltBotTough
BuiltWith BuiltWith
CATExplorador Bullseye
BunnySlippers
CCBot CCBot
CODE87
Calculon Calculon
CazoodleBot CazoodleBot
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
CherryPicker CheTeam
Chlooe Citoid
Cloud mapping Claritybot
Cogentbot Copier
Collector Crawling at Home Project
Copyscape
Cosmos
Craftbot
CrazyWebCrawler CrazyWebCrawler
Crescent Crescent
CrunchBot Custo
Curious CyotekWebCopy
DBLBot
DIIbot DIIbot
DSearch
DTS Agent DTS Agent
DatabaseDriverMysqli
Demon Demon
Deusu
DigitalPebble DigitalPebble
Discobot Discobot
Dragonfly Dispatch
Drip DittoSpyder
DnBCrawler-Analytics
DomCopBot
Download Wonder
ECCP/1.0
EMail Siphon
EMail Wolf
EasyDL
Ecxi Ecxi
ExtLinksBot EroCrawler
Extractor Exabot
ExtractorPro Express WebPictures
Ezooms
FDM FDM
FHscan FHscan
FemtosearchBot FemtosearchBot
Fimap Fimap
Firefox/7.0 Firefox/7.0
FrontPage FlashGet
FyberSpider Foobot
GalaxyBot Freeuploader
Getintent Getintent
Go!Zilla Gigabot
Gotit Go-Ahead-Got-It
GrabNet
Grabber Grabber
GrapeFX Grafula
GrapeshotCrawler GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HTMLparser HTMLparser
HTTrack
Haansoft Haansoft
Harvest Harvest
Heritrix IRLbot
Humanlinks
IlseBot IlseBot
Indy Library
InfoNaviRobot
InfoTekies
Intelliseek Intelliseek
InternetSeer InterGET
Iria Iria
JamesBOT JOC Web Spider
JetCar
Jetty Jetty
JikeSpider Joomla
Jyxobot
Kenjin Spider Kenjin Spider
Kozmosbot
LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Leap LeechGet
LexiBot
LibWeb LibWeb
LieBaoFast Libwhisker
Lightspeedsystems Lightspeedsystems
LinkScan
Linkdexbot Linkdexbot
LinksManager LinksManager
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Litemage_walker Lipperhey Spider
Lmspider Lmspider
MIDown tool Ltx71
MFC_Tear_Sample
MIIxpc MIIxpc
MJ12bot MQQBrowser
Mail.RU_Bot MTRobot
Majestic SEO Majestic SEO
Majestic-SEO
MarkMonitor MarkMonitor
MarkWatch
Mass Downloader
MauiBot MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
MegaIndex.ru MegaIndex.ru
MicroMessenger Mister PiX
Microsoft Data Access Moblie Safari
Microsoft URL Control Morfeus Fucking Scanner
Minefield Msrabot
Mojeek NICErsPRO
Mojolicious
Mozlila
NPbot
Nameprotect Nameprotect
NearSite NearSite
Nessus Net Vampire
NetLyzer NetZIP
NetMechanic
Netcraft Netcraft
Niki-bot Nettrack
NextGenSearchBot
Nikto
Nimbostratus
Ninja
Not Not
Nutch
Octopus
Offline Navigator Offline Navigator
OnCrawl OutclicksBot
OpenLinkProfiler OutfoxBot
Openfind PageAnalyzer
OrangeSpider PageScorer
PECL::HTTP
POE-Component-Client-HTTP
Pandalytics Pandalytics
Papa Foto Papa Foto
PeoplePal
Pi-Monster Pi-Monster
Picscout PictureFinder
Piepmatz Pixray
ProPowerBot ProPowerBot
Probethenet Probethenet
Psbot PxBroker
Pump PyCurl
RankActive Quick-Crawler
RankActiveLinkBot RSSingBot
RankFlex
RankingBot2
Rankivabot Rankivabot
Re-re
RealDownload
RebelMouse
Recorder
RedesScrapy
RocketCrawler RocketCrawler
Rogerbot SEOkicks
SBIder
SEOkicks-Robot SEOkicks-Robot
ScanAlert SMTBot
Scanbot
ScoutJet ScoutJet
ScreenerBot
SearchmetricsBot SearchmetricsBot
SemanticJuice
Semrush Semrush
SemrushBot SemrushBot
SeoSiteCheckup
Siphon Siphon
Siteimprove Site Sucker
SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger
Sitevigil
SlySearch
Snake
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper
Sogou web spider Sogou web spider
Sottopop
SpankBot SpankBot
SputnikBot Spbot
Sqlmap Spinn3r
Sqworm Sqworm
Steeler
Stripper Stripper
Sucker
SuperBot SuperBot
SuperHTTP
SurveyBot SurveyBot
Szukacz
Teleport Teleport
Telesoft
The Intraformant
TheNomad TheNomad
Toata Thumbor
True_Robot TightTwatBot
TurnitinBot Titan
URLy Warning Tracemyfile
Upflow Trendictionbot
V-BOT
VCI
Vacuum Vacuum
VeriCiteCrawler
Virusdie Virusdie
VoidEYE WWW::Mechanize
WASALive-Bot
WBSearchBot
WWW-Collector-E
WWW-Mechanize
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
Web Enhancer Web Enhancer
Web Fetch Web Fuck
Web Pix
Web Sucker Web Sucker
WebBandit WebAuto
WebCollage WebCollage
WebFetch
WebLeacher WebLeacher
WebStripper WebStripper
WebWhacker Website Quester
WebmasterWorldForumBot
WebsiteExtractor WebsiteExtractor
WebsiteQuester
Whack Whack
Wonderbot Who.is Bot
Widow
Woobot
Wotbox
Wprecon Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade
Zauba Zauba
Zitebot Zeus
ZoomBot
ZoominfoBot
ZumBot ZumBot
arquivo.pt check1.exe
autoemailspider clark-crawler
crawl.sogou.com coccocbot-web
cognitiveseo
crawler.feedback crawler.feedback
gopher domainsproject.org
instabid facebookscraper
heritrix
isitwp.com isitwp.com
lwp-request lwp-request
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
pcBrowser
plumanalytics plumanalytics
ripz
s1z.ru s1z.ru
seocompany.store scalaj-http
seoscanners seoscanners
seostar seostar
sysscan serpstatbot
trendiction.com sexsearcher
siteripz
sogouspider
sp_auditbot
trendiction.de trendiction.de
ubermetrics-technologies.com
voyagerx.com
webgains-bot webgains-bot
webmeup-crawler
x09Mozilla x09Mozilla
x22Mozilla
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
arendas.net 0lovespells0.blogspot.com
bikini-image.com bolegapakistan.com
brainsandeggs.blogspot.com eldiariodeguadalajara.com
capture-room.com levaquin750mg.blogspot.com
kinotorka.ru o-o-8-o-o.com
msk.afora.ru rating-bestcasino.com
plastweb.ru scanner-elena.top
razamicroelectronics.com serialsway.ucoz.ru
wrona.it si-unique.com
zigarettenonlinekaufen2.bloog.pl stretchmate.net

930
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 0c47f8422d3f.com
1-free-share-buttons.com 1314dh.com
123movies.love 1kinobig.ru
178evakuator178.ru 24-ak.ru
1pamm.ru 7ooo.ru
202ch.com 98oi.ru
2ads.co.uk aad1f4acb0a373420d9b0c4202d38d94fa.com
2nt.ru absugars.com
3w24.com adanih.com
4ureyesonly.com adbetclickin.pink
5tey7463.weebly.com adinterax.com
7a2rnuey1tw9ar.ru adman.gr
7makemoneyonline.com adultfullhd.com
91abcw.com advancedcleaner.com
aa9bd78f328a6a41279d0fad0a88df1901.com advokateg.ru
aadroid.net ageofclones.com
aasoldes.fr agroeconom.kz
abiente.ru akama.com
absolute-s.ru alert.scansafe.net
acmebtn.ml alive-ua.com
adclickservice.com all4invest.ru
adclickthru.net alloysteel.ru
adservme.com alot.com
adtology.com amateurmatch.com
adult3dgames.com amehdaily.com
adultmeetonline.info amigobulls.com
advertex.info amoi.tn
advertisingtag.net android-vsem.org
adviceforum.info anniemation.com
affrh2015.com apessay.com
afslankpillen2017nl.eu apiholdingmypage-a.akamaihd.net
aksonural.ru apiitsthirteende-a.akamaihd.net
alf-img.com aquarium-pro.ru
alienwheel.es arendakvartir.kz
allbizne.co.ua
allforminecraft.ru
allstatesugarbowl.org
aloofly.com
alphacarolinas.com
amanda-porn.ga
ameblo.top
ampower.me
analytics-ads.xyz
ananumous.ru
angry-fermi-7633.arukascloud.io
anlimebel.kiev.ua
apidiamondatanet-a.akamaihd.net
app-ready.xyz
appsaurus.com
apxeo.info
arabgirls.adultgalls.com
arius.tech arius.tech
arquapetrarca.info arpelsreplica.xyz
artparquet.ru asiengirls.net
asacopaco.tk
ascat.porn
asia-forum.ru
asmxsatadriverin.aircus.com
astrochicks.com
atdedinotuho.tk
audiobangout.com
auto-moto-elektronika.cz
autodo.info autodo.info
autonew.biz automobile-spec.com
avtochehli.by awaybird.ru
azazaporn.com azbukafree.com
backiee.com b-styles.xyz
bagcionderlab.com badavit.com.ua
balans.shahterworld.org baixar-musicas-gratis.com
balitouroffice.com balitouroffice.com
balla-boo.se balkanfarma.org
balois.worldbreak.com bannertesting.com
bannerswap.com battle.net
bear.gotcher.us bbsoldes.fr
bestofferswalkmydogouteveryday.gq bcmp.org
bestwebsiteawards.com becuo.com
bezcmexa.ru beetpics.pw
bigcareer.info bensbargains.net
bigpictures.club best-games.today
biketank.ga bestcurs.org
birzha-truda.eu bestimagecoollibrary.com
bitcoin-ua.top bestmobilityscooterstoday.com
bizfly.info bestsexyblog.com
bjorkbacken.nu beta.hotkeys.com
blavia.00author.com blackbabesporn.com
blobar.org blog.f00kclan.de
blog.koorg.ru bloggerads.net
blpmovies.com bluerobot.info
bonkers.name break-the-chains.com
bonky.biz buqayy0.livejournal.com
bpro1.top burger-tycoon.com
bracketsmackdown.com buy-cheap-pills-order-online.com
brainxs.us buy-loft.ru
brainzpod.win buy-meds24.com
brandimensions.com buyessaynow.biz
bristolhotel.com.ua buyfriend.ru
browserprotecter.com
bum.com.ru
bumble.cheapwebsitehoster.com
businesn.men
buynorxx.com buynorxx.com
buyscabiescream.com camsex.xxx-cam.webcam
bvlgaribracelet.xyz canacopegdl.com
c1.onedmp.com carloans.com
cakemediahq.com.au carson.getenjoyment.net
candypeople.se casinopinup-wins.com
capture-room.com celebsopics.com
carsplate.com chatroulette.life
caulderwoodkennels.com cheapjerseysbizwholesale.us
cejewelry.xyz chinajerseyswholesalecoupons.com
ceotrk.com cialischmrx.com
cete.ru clickpapa.com
cfacarrosserie74.com clickso.com
cheapestjerseys-wholesale.com clksite.com
cheapmarmot.online coaching-netz.info
chocolateslim-original.com collegeessay19.blogspot.ru
chocoslim.pro com-secure.download
cicaki.net compiko.info
cienum.fr compliance-checker.info
clean-virus-mac.com
clknsee.com
cocolyze.com
coin-hive.com
collegerentals.com
commoncrawl.org
compliance-elena.top
compliance-jane.top
compliance-jessica.top compliance-jessica.top
corta.co conocer-sanabria.com
crafthubs.com copblock.org
crazyprotocol.com counciltally.com
credit-respect.ru counter.bloke.com
crest-poloski.ru counterbot.com
crynet.cc craftburg.ru
craftinsta.ru
crazyboost.pro
custodieva.ru custodieva.ru
custom-electric-guitar.com d-black.bz
decorazilla.com dailyrank.net
dairyindia.in
daneshetabiat.com
de.zapmeta.com
deanmoore.ie
dear-diary.ru
decoratinghomes.ga
dekorkeramik.ru
derevesendeco.com derevesendeco.com
dieswaene.com detoxmed24.ru
dieta-personalna.pl
diffbot.com
dimaka.info dimaka.info
djihispano.com dimfour.com
djonwatch.ru dimkino.ru
docs4all.com dipstar.org
domcran.net discovertreasure-a.akamaihd.net
dorratex.tn djstools.com
dosugrostov.site dkvorota.ru
download-of-the-warez.blogspot.com docsportal.net
downloadeer.net doctissimo.fr
downloadmefiranaratb1972.xpg.com.br dolohen.com
doxyporno.com domaincdn.xyz
druzhininevgeniy63.blogspot.com domik-derevne.ru
eduserver.net drev.biz
ekspertmed.com drugspowerstore.com
electro-prom.com dverimegapolis.ru
elmacho.xyz dyshagi.ru
eluxer.net e-poker-2005.com
elvenar.com eachdayisagift.review
embedle.com educatemetv.com
eredijovon.com educontest.net
eropho.net elektroniksigaraankara.info
esfchat.tk elektronischezigarettekaufen1.beeplog.com
eshop4u.jp elektrozigarette2.webs.com
eugenevaultstorage.com elektrozigaretten2.yn.lt
euromasterclass.ru elitesportsadvisor.com
elstal.com.pl
empirepoker.com
eng-lyrics.com
enthuse.computernetworksonline.com
erectile.bid
escort-russian.com
escortslet.net
estibot.com
exlarseva.webblog.es
exoclick.com exoclick.com
extener.org
extremal-blog.com extremal-blog.com
ezigarettenkaufen2.dreamwidth.org failingmarriege.blogspot.com
factorynightclub.com fashionavenuegame.com
famix.xyz femmesdenudees.com
fashion.stellaconstance.co ferretsoft.com
femdom.twiclub.in fetishinside.com
filerockstar298.weebly.com
filesmonster.porn filesmonster.porn
filmidivx.com filmania-x.ru
firstsiteguide.com fishtauto.ru
flagcounter.me fitnesspiks.com
flauntyoursite.com formularz-konkurs.tk
floridamhca.org foto-weinberger.at
forexgb.ru franch.info
foxinsocks.ru free-deals.faith
foxweber.com free-fbook-traffic.com
free-share-buttons.com free-share-buttons.com
freesitetest.com free-social-buttons.com
freeuploader.ml freefoto.ca
friendflnder.com freevpn.space
front.ru freza-sverlo.ru
fuck-paid-share-buttons.xyz fullfileaccess.com
fxtips.ru funponsel.com
g.starmoe.xyz fusoradio.info
gagrasector.ru garciniaxt.us
gameplexcity.com geileweiber.tk
ge0ip.com get-free-social-traffic.com
ge0ip.org getpopunder.com
geckoandfly.com glavtral.ru
genericviagrasildenafiled.net globalsurfari.com
genetworx.com go2mike.ru
ghernnqr.skyrock.com gogps.me
gigapeta.com gojiberriess.apishops.ru
girlporn.ru google-liar.ru
glcomputers.ru googlsucks.com
global-ics.co.za goroda-vsego-mira.ru
gomusix.com gosmeb.ru
gorodservis.ru groupmoney.ru
gotomontenegro.net gsasearchenginerankerhelp.com
gov.yanao.ru gta-club.ru
growmyfunds.ca guidefs.ru
guod.me guildebzh.info
gungamesz.com
guruofcasino.com
gz2.bbsoldes.fr
handicapbathtubarea.com handicapbathtubarea.com
hannasolution.ru hanink.biz.ly
haveinc.xyz hasshe.com
hawaiisurf.com hazardky.net
hem.passagen.se hdimagegallery.net
herehloadibs.cf hermesbirkinhandbagoutlets.com
heygidday.biz hidefiles.org
hkdiiohi.skyrock.com hledejvshopech.cz
hkladys.com hotenergy.ru
homeandhealth.ru hotshoppymac.com
hotblognetwork.com hptwaakw.blog.fc2.com
hotkeys.com humanelydrew.com
houtings.xyz husky-shop.cz
howopen.ru hyipmanager.in
i-service.kz i-midias.net.br
i4track.net ilmen.net
idc.com.ua ilovevitaly.xyz
idegenvezeto.eu indianmedicaltourismshop.com
ilovevitaly.ru infodocsportal.com
imediadesk.com innodgfdriverhm.aircus.com
imgata.com instabid.tech
incanto.in.ua invest-pamm.ru
incolors.club isoveti.ru
indetiske.ya.ru ivearchenceinflu.cf
internetartfair.com
iplogger.org
irkutsk.zrus.org
it-max.com.ua
iwantedmoney.com
jackpotchances.com
janavibekken.no janavibekken.no
javspace.net janerikholst.se
jikoman.info japfm.com
jav-idol.com
jerseyswholesalechinalimited.com
jobmarket.com.ua jobmarket.com.ua
johnrobertsoninc.com jolic2.com
junglenet-a.akamaihd.net jongose.ninja
kalandranis.gr kadashihotel.com
kareliatobacco.ru kam-dom.ru
keenoutlet.online kanimage.com
keywordblocks.com karting196.ru
kazrent.com
kerei.ru
keywesthideaways.co
keywordsdoctor.com keywordsdoctor.com
kicknights.gq kino-key.info
kidskunst.info kinoflux.net
kinobaks.com kiskinhouse.com
kinocccp.net knowyournextmove.com
kinofak.net kongoultry.net
komputers-best.ru koronirealestate.gr
konpax.com ks1234.com
kung-fu-ru.com kurwa.win
kustanay.kz
kutikomi.net kutikomi.net
lalalove.ru kvartira-sutochno.com
lanadelreyfans.us labafydjxa.su
lanasshop.ru labvis.host.sk
laparfumotec.com ladsblue.com
larose.jb2c.me larger.io
lashstudia.ru laulini.soclog.se
lawyersinfo.org law-check-seven.xyz
life.biz.ua laxob.com
limtu.ifmo.ru lernur.net
link.ac lexixxx.com
linkrr.com lidoradio.com
lizace.com lifeinsurancekb.com
loadingpages.me lion.bolegapakistan.com
locimge.pw lion.misba.us
lolnex.us lion.playtap.us
lonfon.xyz liran-locks.com
longadventure.com livefixer.com
lyrics.home-task.com livingroomdecor.info
m-google.xyz lombia.co
m.b00kmarks.com low-format.ru
mackeeper-center.club lunamedia.co
mafcards.ru lutherstable.org
magazintiande.ru luxmagazine.cf
manualterap.roleforum.ru macdamaged.space
maximpartnerspr.com magento-crew.net
mebeldekor.com.ua magicplayer-s.acestream.net
medi-fitt.hu mainhp.net
marblestyle.ru
markjaybeefractal.com
medmajor.ru medmajor.ru
medtherapy.ru meget.co.za
metabar.ru menetie.ru
mhtr.be mexicotravelnet.com
michaelkorssaleoutletonline.net michaelkorsoutletstore.net
migente.com mikrobiologies.ru
mindeyegames.com mirzonru.net
missis.top mmstat.com
missvietnam.org mompussy.net
mnogabukaff.net moncleronline.xyz
mobplayer.ru moneyteam24.com
mobstarr.com
monarchfind-a.akamaihd.net
moneyzzz.ru moneyzzz.ru
morlat.altervista.org mosrif.ru
mossmesi.com mrwhite.biz
movies-in-theaters.net mural.co
moyaterapiya.ru
mp3films.ru
mrbitsandbytes.com
musirc.com
mustwineblog.com mustwineblog.com
muznachas-service.ru muycerdas.xxx
mybackgroundlandscape.blogspot.com muz-tracker.net
myfreemp3.eu
mykings.pw
mypets.by mypets.by
myseoconsultant.com myyour.eu
nacap.ru
naperehresti.info naperehresti.info
narkologiya-penza.ru naphukete.ru
narosty.com narco24.me
needtosellmyhousefast.com narutonaruto.ru
networkcheck.xyz negral.pluto.ro
nippon-bearings.ru netfacet.net
nosecret.com.ua nevansk.ru
novosti-hi-tech.ru nhl17coinsforps3.gratisblog.biz
nowtorrents.com nicovideo.jp
obiavo.by niroo.info
obiavo.su njpalletremoval.com
obesidadealgarve.com
obuv-kupit.ru obuv-kupit.ru
ok.ru onclkads.com
okel.co online-hit.info
okmusic.jp orsonet.ru
onclickpredictiv.com ozoz.it
onlainbesplatno.ru page2rss.com
onlinefilmz.net painting-planet.com
onlineserialy.ru
openfrost.net
oranga.host.sk
ostrovtaxi.ru
ovirus.ru
ozas.net
paleohub.info paleohub.info
palvira.com.ua papasdelivery.ru
partnerads.men par-fallen.ga
paydayloanslocal.com perfection-pleasure.ru
pensplan.com perm-profnastil.ru
perso.wanadoo.es perso.wanadoo.es
peugeot-club.org piccshare.com
phpdevops.com picmoonco.pw
pics-group.com picsfair.com
picturesify.com picturesboss.com
pillscheap24h.com pills24h.com
pivka.xyz pinstake.com
pixshark.com pixelrz.com
plaff-go.ru pizda.lol
ploenjitmedia.azurewebsites.net pk-services.ru
pokemon-go-play.online pkr1hand.com
polytopesexempt.com pl.id-forex.com
powc.r.ca.d.sendibm2.com playlott.com
preconnubial.usuby.site pobeiranie.pl
pregnant.guru pooleroadmedicalcentre.co.uk
princevc.com porn-w.org
printingpeach.com porndl.org
producm.ru pornhub-ru.com
proprostatit.com pornofiljmi.com
psa48.ru pornowarp.info
pwwysydh.com pr0fit-b0x.com
quizzitch.net prlog.ru
ragecash.com proekt-gaz.ru
rankingchart.de professionaldieselcare.com
realitykings.com profitsport.club
projefrio.com.br
prombudpostach.com.ua
pronekut.com
prostitutki-kharkova.org
pssucai.info
push-ad.com
ranksonic.org
rasteniya-vs-zombi.ru
realresultslist.com realresultslist.com
redbottomheels.xyz reckonstat.info
refudiatethissarah.info remmling.de
registry-clean-up.net remont-fridge-tv.ru
reklama1.ru rentalcarnavi.info
relayblog.com restaurantlescampi.com
rennlist.com rewardit.com
replica-watch.ru rossanasaavedra.net
restorator-msk.ru
retailwith.com
ric.info
risparmiocasa.bz.it
roofers.org.uk
ros-ctm.ru
rostov.xrus.org rostov.xrus.org
rus-pornuha.com rus-teh.narod.ru
rutor.group rybalka-opt.ru
safe-app.net sa-live.com
satellite.maps.ilovevitaly.com sa-rewards.co.za
satoristudio.net sajatvelemeny.com
savetubevideo.com savingsslider-a.akamaihd.net
sax-sex.com sax-sex.com
scanner-barak.top scanner-margo.top
scanner-brian.top scanner-walter.top
screpy.com searchimage.co
sdi-pme.com seminarykansas904.ml
seadragonherbery.com seo-2-0.com
seansonline24.pl seo-platform.com
search.pch.com sevendays.com.ua
searchengineranker.email sex-dating.co
searchimpression.com sexpornotales.com
seksvideoonlain.com sexystrippe.info
seo-prof1.xyz shiksabd.com
sexstream.pl shohanb.com
sexvideo-sex.com shoppingmiracles.co.uk
sexvporno.ru sibtest.ru
sexy-screen-savers.com sigmund-freud.co.uk
sexyebonyteen.com siha.de
sharebutton.net simplepooltips.com
shitmovs.com sitechecker.pro
shivafurnishings.com siteheart.net
shymkent.xkaz.org sklad-24.ru
sierraapps.com skylta.com
sk.golden-praga.ru skyway24.ru
slowmacfaster.trade slavic-magic.ru
small-games.biz slkrm.ru
sockshare.net smartshoppymac.com
socialsignals24.com
socialtrade.biz
soft-terminal.ru soft-terminal.ru
sotechco.co solmarket.by
staff.prairiesouth.ca sonnikforme.ru
staticfs.host spb-plitka.ru
steamoff.net spb.afora.ru
stickers-market.ru sport-video-obzor.ru
stpicks.com sribno.net
stroilka.info standardchartered-forex.com
stroymonolit.su start.myplaycity.com
studworks.org staynplay.net
subj.ukr-lit.com streha-metalko.si
superkanpo.com stroimajor.ru
svensk-poesi.com suture.co
svnuppsalaorebro.se
svolze.com svolze.com
swingerseiten.com swagbucks.com
symbaloo.com tastyfoodideas.com
tagil.zrus.org
tam-gde-more.ru
tarad.com
tattooreligion.ru
tecspb.ru tecspb.ru
thebestphotos.eu thebitcoincode.com
thebluffs.com thecoral.com.br
thefotosgratis.eu thedownloadfromwarez.blogspot.com
thegioixekhach.com thenews-today.info
timer4web.com thetoiletpaper.com
titslove.yopoint.in thexart.club
tmtrck.com timecrimea.ru
tootoo.to tizanidine4mgprice.blogspot.com
tour-line.net tizanidineformigraines.blogspot.com
toy-shop.top tizanidinesideeffectsweightloss.blogspot.com
traffmonster.info tizanidinewithdrawal.blogspot.com
trahvid.com toon-families.com
tuberkulezanet.ru torrentgamer.net
tuckpointingmasonrysystems.com torrents-tracker.com
tvory.predmety.in.ua tracfone.com
txxx.com track112.site
trafficjunky.net
trafficmonetize.org
trafficmp.com
trafficnetzwerk.de
tramadolandtizanidine.blogspot.com
turist-strani.ru
tx41tclega.ru
uasb.ru uasb.ru
uginekologa.com ukrobstep.com
underthesite.com unionmarkt.de
unitexindia.com unitygame3d.com
uptime-delta.net uptime-gamma.net
usbggettwku.ga urlopener.blogspot.com.au
validccseller.com usiad.net
valoresito.com usswrite.com
vandrie-ict.nl uzungil.com
varikozdok.ru valsalud.com
vasileostrovsky-rayon.ru valueclick.com
velobikestock.com veerotech.com
verymes.xyz video-camer.com
viberdownload10.com videonsk.com
videochat.tv.br villacoloniale.com
vielporno.net vinsit.ru
vk-mus.ru virtualbb.com
vltai.com volgograd.xrus.org
vrnelectro.ru voloomoney.com
wallpapers-best.com vozbujdenie.com
watchmygf.net vut.com.ru
watracker.net wallpapersist.com
webalan.ru weblibrary.win
weblo.com weblo.com
webmasterhome.cn weboptimizes.com
webnode.me webpromotion.ae
websitevaluebot.com
webstatsdomain.org
wedding0venues.tk wedding0venues.tk
wemarketing.se whengirlsgowild.com
wesharepics.site winner7777.net
wholesalejerseys-cheapest.com womensplay.net
whos.amung.us woodyguthrie.se
witherrom55.eklablog.fr worldoffiles.ru
xerox-douglas.cf wrona.it
xlolitka.com www.888.com
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai x-diesel.com
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xbaboon.com
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xchat26.myfreecams.com
xn----9sbebi2bvzr7h.xn--p1ai xgames-04.com
xmnb.net
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--d1acah0c.xn--p1ai xn--80aodinpgi.xn--p1ai
xn--d1aifoe0a9a.top xtrafficplus.com
xsfetish.org xuki.us
xsion.net xzlive.com
xxxsiterips.xyz yellowstonevisitortours.com
yellowproxy.net yes-com.com
zaimhelp.ru yginekologa.com
zarepta.com ynymnwbm.bloger.index.hr
zdesformula.ru youandcredit.ru
zigarettenonlinekaufen.tumblr.com youporn-ru.com
zigarettenonlinekaufen2.bloog.pl youtubedownload.org
zrizvtrnpale.tk ypmuseum.ru
zagadki.in.ua
zeroredirect7.com
zirondelli.it
zocaparj.kz
zoogdiesney.com
zskdla.site
zx6.ru
zytpirwai.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123locker.com 007angels.com
1qingdao.com 1008.su
123cha.com
2itech.ru
4senses.co 4senses.co
83net.jp a.frcls.fr
888.com aac500b7a15b2646968f6bd8c6305869d7.com
991mostfm.co.id activepr.ru
aab94f698f36684c5a852a2ef272e031bb.com acunetix-referrer.com
abusora.com adconscious.com
accessoires-mode.in adloads.com
activecampaign.dreamhosters.com ads.tremorhub.com
adamoads.com akita.kz
ads.gold all4wap.ru
adultnet.in allboard.xobor.de
aimayangzhi.com amospalla.es
alert-hgd.xyz angkortours.vn
alfapro.ru apartment.ru
allproblog.com appsaurus.com
apiitsthirteende-a.akamaihd.net arabsexxxtube.com
apimossnetinfo-a.akamaihd.net asdfz.pro
arendas.net asseenontvonline.ru
artdestshop.eu atmovs.com
asmxsatadriverin.aircus.com atout-energie-69.com
atovh.local-finders.com audiobangout.com
azinoofficial777.ru audiofree.ru
backgroundpictures.net autoseo-traffic.com
baersaratov.ru aviav.co
battle.net aviav.ru
bayareaaeroclub.org avramstroy.ru
becuo.com azazaporn.com
bestcurs.org bestmobilityscooterstoday.com
bif-ru.info bestmouthwash.club
biketank.ga bonux.nextview.ru
bioscorp.ru bookmark4you.com
bjgugu.net.ua brainboostingsupplements.org
blackwitchcraft.ru breastaugmentation.co.za
bloggers.nl browsepulse-a.akamaihd.net
blogos.kz buron.pw
book-bets.com buy-cheap-online.info
bosefux.esy.es buyscabiescream.com
buytizanidineonline.blogspot.com carapics.com
californianews.cf cartierlove2u.xyz
cand.jp cejewelry.xyz
canvas.gloverid.site cete.ru
cartierreplica.pw changde.58.com
caulderwoodkennels.com chlooe.com
cfacarrosserie74.com clickbanksites.info
chinese-amezon.com compliance-julia.top
cialgenisrx.com compucelunlock.net
cigarpass.com contextualyield.com
citysecurity.nu counterbot.com
compliance-viktor.top creams.makeforum.eu
costablue.xyz crest3d.ru
covetnica.com crynet.cc
custom-product-labels.com crystalslot.com
cutt.us customboxes4less.com
dailystrength.org danashop.ru
dangphoto.trade daymusam.com
delayreferat.ru detskie-zabavi.ru
dimaka.info deutschland123.de
dinosaurus.site
djekxa.ru djekxa.ru
djonwatch.ru dojki365.online
dktr.ru domaincheck.io
dojki-hd.com donvito.unas.cz
e2click.com dostavka-v-ukrainu.ru
easytuningshop.ru download-of-the-warez.blogspot.com
empirepoker.com doyouknowtheword-flummox.ml
enbersoft.com dscaas.website
etenininrade.ga earthmagic.info
everytalk.tv elektrozigarette1.dreamwidth.org
exlarseva.webblog.es empis.magix.net
exonline.info eshop.md
eyemagination.com evening-dating-club.info
ezigarettenshop2.postbit.com face.hostingx.eu
f5mtrack.com fealq.com
farmingworm.com fetishinside.com
fashion-mk.net fighrofacciufreesig.ga
figuringmoneyout.com firstdrugmall.ru
forexgb.ru forodvd.com
forum69.info forpackningsutveckling.se
free411games.com forumprofi.de
galeria-zdjec.com fotoxxxru.com
gamesprite.me free-floating-buttons.com
gamewrath.com freefoto.ca
gateway.zscalerone.net freejabs.com
ge0ip.com fusoradio.info
geckoandfly.com gdcentre.ru
glall.ru gearcraft.us
glasof.es gepezz.info
golaya.pw gg-arena.ru
gorodservis.ru goroda-vsego-mira.ru
gsou.cf gosarhivrt.ru
gstatey.net gosreg.amchs.ru
gwebtools.com graphid.com
hacktougroup.ru grupografico-pilar.com.ar
handsandlegs.ru habermetre.com
hamilton.ca
hannasolution.ru hannasolution.ru
heygidday.biz healing-dysplasia.ru
hotelcrocenzi.sm hitmuzik.ru
housekuba.org homemature.net
houston-vikings.com hothot.ru
hut1.ru howtoclean.club
ideawheel.com htmlcorner.com
ihtec2019.org hybrid.ru
imagerydatabase.com imk.com.ua
incep.imagine-ex.co in-tandem.co
indetiske.ya.ru informatiecentro.be
instabid.tech ipornox.xxx
kihi.gdn isitpaleo.info
kino-filmi.com islamtoday.co.za
kinohit1.ru istizanidineacontrolledsubstance.blogspot.com
kinozapas.com jamiembrown.com
klin3952.ru javspace.net
kmd-pto.ru je7.us
komputernaya-pomosh-moscow.ru jerseywholesalebizchina.com
lafourmiliaire.com jofucipiku.tk
lampokrat.ws johannesburgsingles.co.za
le-clos-des-alouettes.com jumptap.com
lemon-ade.site kenaba.su
lennatin.info khadastoafarde.tk
life-instyle.com kw21.org
lightinghomes.net kwzf.net
limtu.ifmo.ru lancheck.net
lion.misba.us leboard.ru
lol-smurfs.com lifebyleese.com
lollypopgaming.com lincolntheatre.com
lotto6888.com linkbux.ru
ltvperf.com lmrauction.com
m292.info lockerz.com
magazin-pics.ru lombia.com
makis.nu m4ever.net
mamasuper.prom.ua manifestyourmillion.com
manimpotence.com maranbrinfo.com.br
mcnamaratech.com marktforschung-stuttgart.com
medicovi.com mebel-vstroika.ru
megahdporno.net megahdporno.net
menetie.ru menetie.ru
menhealed.net mini.7zap.com
miniads.ca misterjtbarbers.com
mistr-x.org
monclerboots.xyz
moneytop.ru
morefastermac.trade
mountainstream.ms mountainstream.ms
music7s.me moz.com
muztops.ru mp3ringtone.info
mycouponizemac.com mwtpludn.review
networkad.net mydownlodablefiles.com
njpalletremoval.com mygameplus.com
nzfilecloud.weebly.com nagdak.ru
obesidadealgarve.com nakozhe.com
officedocuments.net narco24.me
online-sbank.ru netcheckcdn.xyz
onlineslotmaschine.com november-lax.com
onlyforemont.ru online247.ml
openlibrary.org onlyporno.ru
optom-deshevo.ru ons-add.men
oynat.info open-odyssey.org
pacificair.com oyster-green.com
pandroid.co painting-planet.com
pdamods.ru panouri-solare-acoperis.com
petedrummond.com patterntrader-en.com
picquery.com pdn-4.com
picturesfrom.com phormchina.com
pinup-casino1.ru picsfair.com
pixgood.com pinstake.com
polytopesexempt.com pivka.xyz
pornoxxx.com.mx pl.aasoldes.fr
positive2b.ru pornstartits.xblog.in
privatamateure.com prchecker.info
proftests.net prefersurvey.net
progonrumarket.ru prizeestates.cricket
proxyradar.com prostitutki-astana.org
pussyspace.net pwwysydh.com
putitin.me qwarckoine.com
ragecash.com registry-cleaner.net
rankexperience.com relatodelpresente.com.ar
razvratnoe.org resultshub-a.akamaihd.net
real-time-analytics.com riders.ro
redhotfreebies.co.uk ring4rhino.com
redirectme.net robertefuller.com
rennlist.com roleforum.ru
resellerclub.com ruex.org.ua
reversing.cc
rewardit.com
rockprogblog.com
ruclicks.com
ruspdd.com ruspdd.com
scanner-donald.top schlampen-treffen.com
scanner-elena.top searchengineranker.email
scottbywater.com seasaltwithfood.com
sdsjweb.com sex-watch.com
seeresultshub-a.akamaihd.net sexgalleries.top
seo-smm.kz sexyteens.hol.es
seocheckupx.com shtora66.ru
servisural.ru shymkent.xkaz.org
sexkontakteao.info sildenafil-tadalafil.info
sexstream.pl sim-service.net
simon3.ru siteaero.com
site-speed-check.site smailik.org
sky-mine.ru spamnuker.com
sobecjvuwa.com.ru stats-public.grammarly.io
spacash.com steamoff.net
stardevine.com strfls.com
streetfooduncovered.com
stretchingabuckblog.com
stretchmate.net
studiofmp.com
subj.ukr-lit.com subj.ukr-lit.com
superoboi.com.ua superstarfloraluk.com
thegameriders.com syndicate.fun
tizanidinepills.blogspot.com tacbelarus.ru
tmm-kurs.ru taqywu51.soup.io
toondinsey.com thenews-today.info
topbestgames.com thiegs.reco.ws
topcar-krasnodar.ru tizanidine6mg.blogspot.com
topclickguru.com tizanidineandgabapentin.blogspot.com
traffictrade.life tizanidinehcl4mgtablet.blogspot.com
trkdf.com tmtrck.com
turist-strani.ru tongkatmadura.info
twelvevisionspartyofcolorado.com toonfamilies.net
tytoona.com tubeline.biz
udayavani.com tula.mdverey.ru
uac.net.au
ugguk.online
ukrtextbook.com
unimodemhalfduplefw.pen.io unimodemhalfduplefw.pen.io
valkiria-tk.ru uptime-delta.net
valsalud.com vashsvet.com
velen.io viagengrarx.com
vipms.ru video-camer.com
visa-pasport.ru vkontaktemusic.ru
volume-pills.biz vksaver-all.ru
w-journal.ru vladimir.zrus.org
watchmygf.net vote-up.ru
wdrake.com wallpapersist.com
we-are-gamers.com walpaperlist.com
womensplay.net webenlace.com.ar
word-vorlagen.net webmasters.stackexchange.com
www.bookmaker-bets.com webs.com
x-diesel.biz where-toget.com
x-musics.com wikes.20fr.com
xfire.com wjgony.com
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai wwwadultcheck.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai x-porno.video
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai x-stars.ru
xn--e1aggki3c.xn--80adxhks xn----7sbifcamovvfggw9d.xn--p1ai
xnxx699.com xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
yatrk.xyz xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
yoursite.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
youtuhe.com xn--d1abj0abs9d.in.ua
zaapplesales.blogspot.com xportvusbdriver8i.snack.ws
zeroredirect9.com xvideosbay.com
zscaler.net xxxsiterips.xyz
youradexchange.com
zaim-pod-zalog-krasnodar.ru
zarepta.com
zhongwenlink.com
zverokruh-shop.cz

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.39 0; 10.1.1.11 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2797 ### Version: V4.2021.12.2798
### Updated: Tue Dec 21 10:03:32 UTC 2021 ### Updated: Tue Dec 21 22:01:20 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2797 ### Version: V4.2021.12.2798
### Updated: Tue Dec 21 10:03:32 UTC 2021 ### Updated: Tue Dec 21 22:01:20 UTC 2021
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##