vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2637 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-07 22:07:32 +00:00
parent 512adef51f
commit fa956da8ad
46 changed files with 74811 additions and 74811 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2636 #### Version: V4.2021.10.2637
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2636 #### Version: V4.2021.10.2637
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2636 #### Version: V4.2021.10.2637
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9218
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2636 #### Version: V4.2021.10.2637
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9222
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9222
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9222
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9222
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.54 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.54 0;
127.0.0.1 0; 127.0.0.1 0;

9222
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.54 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.54 0;
127.0.0.1 0; 127.0.0.1 0;

9222
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.54 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.54 0;
127.0.0.1 0; 127.0.0.1 0;

9222
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2636 2637

9222
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9222
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.170.139.251 0; 20.69.93.88 0;

9222
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.54 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.54 0;
127.0.0.1 0; 127.0.0.1 0;

9222
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.54 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.54 0;
127.0.0.1 0; 127.0.0.1 0;

9222
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.54 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.54 0;
127.0.0.1 0; 127.0.0.1 0;

9222
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.170.139.251 0; 20.69.93.88 0;

9222
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.170.139.251 0; 20.69.93.88 0;

9222
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 1; 10.1.1.54 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Blow AwarioSmartBot
Bolt Crawling at Home Project
CherryPicker DBLBot
ChinaClaw Evil
CrazyWebCrawler LinksManager
Metauri Microsoft Data Access
Toweyabot Panscient
WWW::Mechanize PyCurl
WebCopier crawl.sogou.com
instabid lwp-request

284
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
80legs 80legs
AIBOT AIBOT
ALittle Client ALittle Client
ASPSeek Aboundex
Abonti Acunetix
AhrefsBot AhrefsBot
Alligator Alligator
Anarchy Ankit
Apexoo Asterias
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BBBike
BDCbot BDCbot
BackDoorBot BackStreet
Backlink-Ceck
BacklinkCrawler
Badass Badass
Bandit BatchFTP
Battleztar Bazinga Black Hole
Blackboard
Blow Blow
Bolt
BotALot BotALot
Brandprotect Brandprotect
Buck Buck
Buddy
Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CATExplorador
CSHttp
Calculon Calculon
CazoodleBot Cegbfeieh
CensysInspect
CheTeam
CheeseBot CheeseBot
Chlooe
Claritybot Claritybot
Cliqzbot Cliqzbot
Cocolyzebot Cocolyzebot
Collector Cogentbot
Copier Copyscape
CopyRightCheck
Craftbot Craftbot
CrunchBot Crawling at Home Project
Curious CrazyWebCrawler
DBLBot Custo
DIIbot
DSearch
DTS Agent
DatabaseDriverMysqli DatabaseDriverMysqli
Digincore Demon
Disco Deusu
Discobot
Discoverybot Discoverybot
DittoSpyder
DnyzBot DnyzBot
DomCopBot DomainStatsBot
Domains Project Domains Project
Dotbot Dotbot
Download Wonder EMail Siphon
Dragonfly
EMail Wolf
EasyDL
Ebingbong Ebingbong
EroCrawler EroCrawler
Express WebPictures
ExtLinksBot ExtLinksBot
Extreme Picture Finder Extractor
ExtractorPro
EyeNetIE EyeNetIE
Ezooms Ezooms
FDM
FHscan
FemtosearchBot FemtosearchBot
Flunky Fimap
Foobot Firefox/7.0
Freeuploader Freeuploader
FrontPage
Fuzz Fuzz
G-i-g-a-b-o-t Genieo
GalaxyBot
GermCrawler GermCrawler
GetRight GetRight
GetWeb GetWeb
Getintent Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GrabNet
Gotit Grafula
GrapeFX GrapeshotCrawler
HEADMasterSEO
HMView HMView
HTMLparser Havij
Haansoft Hloader
HaosouSpider Humanlinks
HybridBot
Iblog
Id-search Id-search
IndeedBot Image Sucker
Intelliseek InfoTekies
Internet Ninja
InternetSeer InternetSeer
JamesBOT IstellaBot
JetCar JOC Web Spider
JikeSpider
Joomla
Jorgee Jorgee
JustView JustView
Kozmosbot Kozmosbot
LNSpiderguy
Lanshanbot Lanshanbot
Larbin
Leap Leap
LeechGet LeechFTP
LibWeb LibWeb
Libwhisker
Lightspeedsystems
Likse Likse
LinkWalker
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey
Lipperhey Spider Ltx71
Litemage_walker MFC_Tear_Sample
Lmspider MIDown tool
MSFrontPage
MSIECrawler MSIECrawler
Magnet Magnet
Majestic12 Majestic SEO
MarkMonitor
MarkWatch MarkWatch
Mass Downloader
Masscan Masscan
MauiBot
Mb2345Browser
MeanPath Bot MeanPath Bot
Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Metauri
Mister PiX Mister PiX
Mojeek Mojeek
Name Intelligence Mozlila
NearSite Msrabot
Nessus Musobot
Navroad
NetLyzer NetLyzer
NetMechanic NetZIP
Netcraft
Nettrack
Netvibes Netvibes
NextGenSearchBot Nutch
Nibbler Offline Explorer
Nikto
NimbleCrawler
Nimbostratus
Nmap
Offline Navigator
OnCrawl
OpenVAS
Openfind
Openvas Openvas
OrangeBot OutclicksBot
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PECL::HTTP
Page Analyzer Page Analyzer
PageAnalyzer PageScorer
Papa Foto
PeoplePal PeoplePal
Petalbot Pi-Monster
Picscout
Picsearch
PictureFinder PictureFinder
Piepmatz Piepmatz
ProPowerBot Pimonster
Pump Pixray
QueryN Metasearch PleaseCrawl
RankActiveLinkBot Pockey
RankFlex ProWebWalker
Pu_iN
PxBroker
RankingBot RankingBot
RankingBot2 Reaper
RedesScrapy RocketCrawler
RepoMonkey SBIder
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOprofiler
SISTRIX SISTRIX
ScoutJet ScoutJet
ScreenerBot
ScrepyBot ScrepyBot
Semrush SearchmetricsBot
SemrushBot
SentiBot SentiBot
SeoSiteCheckup
SeobilityBot
Site Sucker Site Sucker
SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker
Sitebeam
Sitevigil Sitevigil
SlySearch
Snake Snake
Snapbot Snapbot
SpankBot Sociscraper
Sogou web spider
SpaceBison
Spinn3r Spinn3r
SputnikBot
Sqlworm
Sqworm Sqworm
Steeler
Stripper Stripper
Sucuri Sucker
SuperBot Surfbot
SuperHTTP Swiftbot
SurveyBot T0PHackTeam
Suzuran TeleportPro
Telesphoreo Telesphoreo
Telesphorep
The Intraformant The Intraformant
TheNomad
Thumbor
TightTwatBot TightTwatBot
Titan
Toweyabot
Tracemyfile Tracemyfile
Trendictionbot Trendiction
True_Robot
TurnitinBot TurnitinBot
Twice Typhoeus
URLy Warning Vagabondo
Upflow Virusdie
V-BOT
VCI
VeriCiteCrawler
VidibleScraper
Voltron Voltron
WISENutbot
WWW-Mechanize WWW-Mechanize
WWWOFFLE Wallpapers
WallpapersHD Wallpapers/3.0
Web Auto WeSEE
Web Collage
Web Enhancer Web Enhancer
Web Sauger
Web Sucker
WebAuto WebAuto
WebBandit
WebCollage
WebCopier WebCopier
WebEnhancer WebFetch
WebGo IS
WebImageCollector
WebPix WebPix
WebWhacker
WebsiteQuester WebsiteQuester
Webster
Whacker Whacker
Widow Whatweb
WinHTTrack WiseGuys Robot
Xenu Woobot
Wprecon
Zeus
Zitebot
ZmEu ZmEu
ZoomBot ZoomBot
archive.org_bot ZoominfoBot
ZumBot
ZyBorg
arquivo-web-crawler arquivo-web-crawler
autoemailspider cah.io.community
check1.exe coccocbot-web
clark-crawler
cognitiveseo cognitiveseo
com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler.feedback
dataforseo.com eCatch
gopher facebookscraper
internetVista monitor
ips-agent ips-agent
isitwp.com
lwp-request lwp-request
magpie-crawler magpie-crawler
netEstate NE Crawler muhstik-scan
oBot
pcBrowser pcBrowser
plumanalytics
polaris version polaris version
probe-image-size ripz
s1z.ru scalaj-http
satoristudio.net scan.lol
seoscanners sitechecker.pro
serpstatbot
sogouspider sogouspider
trendiction.com spyfu
trendiction.de trendiction.de
zauba.io x09Mozilla

286
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker 404checker
80legs 80legs
ADmantX
ALittle Client
ASPSeek ASPSeek
Aboundex Acunetix
AfD-Verbotsverfahren
Aipbot
Alligator
AlphaBot AlphaBot
Anarchie Anarchie
Anarchy
Anarchy99 Anarchy99
Apexoo AwarioRssBot
AwarioSmartBot
BBBike
BDCbot
BDFetch BDFetch
BLEXBot
BackDoorBot BackDoorBot
Barkrowler BetaBot
Battleztar Bazinga
Bigfoot Bigfoot
BlackWidow
Blackboard Blackboard
Buddy BlowFish
BuiltBotTough Bolt
Brandwatch
Buck
BuiltWith
Bullseye Bullseye
BunnySlippers BunnySlippers
CATExplorador CATExplorador
CSHttp
Calculon Calculon
CazoodleBot CensysInspect
Cegbfeieh CheeseBot
CheTeam Chlooe
CherryPicker Claritybot
ChinaClaw Cliqzbot
Collector Cocolyzebot
Copier
Cosmos
Crescent
CrunchBot CrunchBot
Custo
CyotekWebCopy CyotekWebCopy
DIIbot DSearch
DataCha0s
DatabaseDriverMysqli
Demon Demon
Devil Deusu
Digincore
DigitalPebble DigitalPebble
Dirbuster
Disco Disco
Discobot Discobot
Discoverybot Dispatch
DomCopBot DomCopBot
DomainAppender
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Dragonfly Download Wonder
EMail Siphon
EasyDL
Ebingbong
Ecxi Ecxi
EirGrabber EirGrabber
EroCrawler EroCrawler
Exabot
ExtLinksBot ExtLinksBot
Extractor Extractor
ExtractorPro ExtractorPro
EyeNetIE
Ezooms
FDM FDM
FlashGet FHscan
FemtosearchBot
Foobot
FrontPage FrontPage
FyberSpider
G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo
GermCrawler GermCrawler
GetRight GetRight
GetWeb GoZilla
Go-Ahead-Got-It Gotit
GrabNet GrabNet
Grabber Grafula
GrapeFX
GridBot GridBot
HMView HEADMasterSEO
HTTP::Lite HTTP::Lite
HTTrack HaosouSpider
Harvest Havij
IRLbot
Iblog Iblog
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker
InfoNaviRobot InfoNaviRobot
Internet Ninja InfoTekies
InternetSeer Intelliseek
Iria
Iskanie Iskanie
IstellaBot IstellaBot
JOC Web Spider
JamesBOT JamesBOT
Jbrofuzz JennyBot
JetCar
Jetty Jetty
Joomla
JustView JustView
Keyword Density Jyxobot
Kenjin Spider
Kinza Kinza
Kozmosbot
LNSpiderguy
Leap Leap
LeechGet Libwhisker
LexiBot
Lightspeedsystems Lightspeedsystems
LinkScan LinkScan
Linkbot Linkdexbot
LinksManager LinkextractorPro
LinqiaRSSBot LinkpadBot
Litemage_walker LinqiaScrapeBot
Lmspider Lipperhey
Ltx71 MFC_Tear_Sample
MTRobot MIDown tool
Mail.RU_Bot MJ12bot
MSFrontPage
MSIECrawler
Magnet
Majestic SEO
Majestic-SEO Majestic-SEO
Mass Downloader Mass Downloader
Mediatoolkitbot Masscan
MauiBot
Mb2345Browser
MegaIndex.ru
Metauri Metauri
Microsoft URL Control Moblie Safari
Mojolicious NICErsPRO
Morfeus Fucking Scanner NPbot
Mozlila
Musobot
Nameprotect
Navroad Navroad
Nessus Needle
NetAnts NetAnts
NetLyzer
NetMechanic
NetSpider NetSpider
NetZIP NetZIP
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler Nibbler
Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Nmap Nmap
Nuclei
Nutch Nutch
OpenLinkProfiler Offline Explorer
OrangeBot Openfind
OrangeSpider
OutfoxBot OutfoxBot
POE-Component-Client-HTTP PECL::HTTP
PHPCrawl
Page Analyzer Page Analyzer
Pandalytics Panscient
Papa Foto
Pavuk Pavuk
PeoplePal Petalbot
Pi-Monster Pi-Monster
Picsearch
Pimonster Pimonster
Pixray ProPowerBot
PleaseCrawl
ProWebWalker ProWebWalker
Psbot
Pump Pump
Recorder RSSingBot
RankActive
RankActiveLinkBot
Rankivabot
RealDownload
Reaper
RepoMonkey
Ripper
RocketCrawler RocketCrawler
Rogerbot SBIder
SEOprofiler SEOkicks
SISTRIX SEOstats
SMTBot
SalesIntelligent
Scanbot
Scrapy Scrapy
ScreenerBot Screaming
ScrepyBot
SearchmetricsBot SearchmetricsBot
SemrushBot
SentiBot SentiBot
Seomoz Siphon
Shodan SiteExplorer
Site Sucker SiteSnagger
SiteCheckerBotCrawler Snake
SiteSucker Snapbot
Sitebeam
Siteimprove
Sitevigil
SmartDownload
SpaceBison SpaceBison
Spanner Spammen
Spbot SpankBot
Sqlmap Steeler
Stripper Sucker
SuperHTTP SuperBot
SurveyBot Surfbot
Suzuran
T0PHackTeam T0PHackTeam
T8Abot T8Abot
Teleport
Telesoft Telesoft
Telesphoreo
Telesphorep Telesphorep
TheNomad
Toata Toata
Toweyabot
Trendictionbot Trendictionbot
True_Robot True_Robot
TurnitinBot
Typhoeus Typhoeus
URLy.Warning
UnisterBot
VB Project VB Project
Vacuum VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
VoidEYE Virusdie
WBSearchBot Voil
WEBDAV WASALive-Bot
WISENutbot WISENutbot
WWW-Collector-E WPScan
WWW::Mechanize
WWWOFFLE WWWOFFLE
WallpapersHD Wallpapers
WeSEE WeSEE
Web Collage
Web Fetch Web Fetch
Web Sucker
WebBandit
WebCollage
WebEnhancer
WebFetch
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebSauger WebSauger
WebWhacker
Website Quester
WebsiteQuester
Whack Whack
Whacker Whacker
Whatweb Widow
Who.is Bot Wprecon
WiseGuys Robot Zitebot
Wonderbot
Woobot
Wotbox
Xaldon_WebSpider
Zauba
Zermelo
ZoomBot ZoomBot
archive.org_bot ZoominfoBot
clark-crawler ZumBot
ZyBorg
arquivo-web-crawler
arquivo.pt
autoemailspider
backlink-check
cah.io.community
check1.exe
cognitiveseo cognitiveseo
crawler.feedback crawler.feedback
crawler4j dataforseo.com
demandbase-bot demandbase-bot
facebookscraper domainsproject.org
internetVista monitor internetVista monitor
isitwp.com
lwp-request lwp-request
lwp-trivial
meanpathbot
mediawords mediawords
netEstate NE Crawler muhstik-scan
page scorer
pcBrowser pcBrowser
plumanalytics
polaris version
ripz ripz
satoristudio.net scan.lol
seoscanners seoscanners
seostar
sexsearcher
sitechecker.pro sitechecker.pro
spyfu sp_auditbot
tAkeOut tAkeOut
trendiction.com trendiction.com
webmeup-crawler trendiction.de
ubermetrics-technologies.com
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
acads.net buttons-for-website.com
azartmix.com ccbill.com
cete.ru europages.com.ru
friendflnder.com gifspics.com
m5home.ru ranking2017.ga
make-money-online.com responsinator.com
millionare.com sfj-ror.no
searchlock.com tonerbox.kz
videositename.com vip-parfumeria.ru
youtuhe.com zolotoy-lis.ru

930
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
14b.info 1-99seo.com
1hwy.com 13tabs.com
1pamm.ru 1j7740kd.website
300richmond.co.nz 1kdailyprofit.me
3dracergames.com 23kw.ru
404.mn 2pxg8bcf.top
4senses.co 3w24.com
4webmasters.org 4inn.ru
51.la 4kplayer.pl
7search.com 4pp13.com
aa8780bb28a1de4eb5bff33c28a218a930.com 76brighton.co.uk
aa9bd78f328a6a41279d0fad0a88df1901.com 7makemoneyonline.com
actionnooz.com
acunetix-referrer.com
adamoads.com adamoads.com
adclickservice.com adclickservice.com
adidas.frwebs.fr adconscious.com
adloads.com adexprts.com
advancedmassagebysara.com adserver-e7.com
advertisingtag.net adsfresh.men
alarmobninsk.ru adultfriendfinder.com
alienwheels.de adultwebhosting.info
alive-ua.com adventureparkcostarica.com
alldezservice.kz adverster.com
allfinweb.com aero2.ru
allwomen.info ahrntal.verymes.xyz
amanda-porn.ga airmaxshoes-2016.com
ameblo.jp aktivator-windows10.blogspot.com
anal-acrobats.hol.es alert-fjg.xyz
android-systems.ru alkoravto.ru
anlimebel.kiev.ua all4invest.ru
apccargo.com alloysteel.ru
apiappenableinfo-a.akamaihd.net alltheviews.com
apidigidocketnet-a.akamaihd.net alpharma.net
approved.su alyeskaresort.com
artdestshop.eu am15.net
autochoixspinelli.com anapa-inns.ru
avkzarabotok.info android4fun.org
apple.com-webbrowsing-security.science
aramaicmedia.org
asociatia-tipografilor-transilvania.ro
atout-energie-69.com
atovh.local-finders.com
australia-opening-times.com
autoloans.com
autoplate.info
autoseo-traffic.com
avon-ukraine.com
avtoarenda.by
b2b-lounge.com b2b-lounge.com
backgroundpictures.net baltgem.com
baikaleminer.com bank.uz
balkanfarma.ru bannertesting.com
bamo.xsl.pt bazaronline24.ru
bannerads.de beetpics.pw
bdsmgalls.net berrymall.ru
best-deals-products.com bestdooz.com
best-way.men besthoro.ru
bestdraws.com bestofupload.info
bestofferswalkmydogouteveryday.gq bestpornuha.com
bet365.com bezsporno.ru
betteroffers.review bfz.biz
beztuberkuleza.ru big-cash.net
bidr.trellian.com blackplanet.com
bikini-image.com blogrankers.com
billiard-classic.com.ua blogsfunky672.weebly.com
bimatoprost-careprost.com blpmovies.com
bjorkbacken.nu bluerobot.info
blogig.org bmsco.co
bloglag.com bmw-ark.ru
bracketsmackdown.com board.f00d.de
brainboosting.club bochemit.com.ua
brandedleadgeneration.com bombla.org
brendbutik.ru bonky.biz
brimstonehillfortress.org book-bets.com
briomotor.co boole.onlinewebshop.net
browserprotecter.com bouda.kvalitne.cz
brus-vsem.ru brainsandeggs.blogspot.com
btnativenav.com brandimensions.com
buchananshardware.com brg8.com
bukleteg.xyz brk-rti.ru
buqayy0.livejournal.com brokergid.ru
buy-cheap-online.info bryansk.zrus.org
buyessay3.blogspot.ru budpost.com.ua
buyparajumpers.online buyclomidonlaine.com
cah.io.community buyk.host.sk
callejondelpozo.es calcularpagerank.com
calstaterealty.us calstaterealty.us
cand.jp capacitacionyaprendizaje.com
capture-room.com carabela.com.do
cartujano-pre.de cardul.ru
cayado.snn.gr cartierjusteunclou.xyz
celejihad.info cartierreplica.top
cenoval.ru catalogs-parts.com
chatroulette.life cbcseward.com
chcu.net cellfun.mobi
checkm8.com charmstroy.info
christianlouboutinshoes.xyz cheapbelstaff.online
cialisovercounteratwalmartusa.com cheapmoncler.win
ciproandtizanidine.blogspot.com cheapmoncler.xyz
clickbank.net checkhit.com
clickso.com ciceros.co
clubfashionista.com ciekawinki.pl
cmd.kz cinemaflix.website
cocyq.inwtrade.com clicksor.net
coderstate.com com-supportcenter.website
codq.info commoncrawl.org
compliance-andrew.xyz community.allhiphop.com
compliance-jane.top compliance-checker-7.info
compliance-mark.top compliance-ivan.top
construmac.com.mx compliance-julianna.top
conocer-sanabria.com
cookielawblog.wordpress.com cookielawblog.wordpress.com
cool-wedding.net cottageofgrace.com
countbertwistdisp26.soup.io creatives.adbetclickin.pink
courtshipgift.com cscwtalkto.site
csgo4.win cvety24.by
cubs-jerseys.us decorazilla.com
culmimg.pw defenderxtactical.com
culturevie.info dellalimov.com
custom-product-labels.com demon-tweeks.com
customboxes4less.com detective01.ru
cypernhuset.se detki-opt.ru
danceuniverse.ru detsky-nabytek.info
dawlenie.com dev-seo.blog
deplim.com dfwu1019.info
design-lands.ru dickssportinggoods.com
dikx.gdn divatraffic.com
dimaka.info dogbreedspicture.net
discountbarbour.online doktoronline.no
disruptingdinnerparties.com domain-tracker.com
dogsrun.net domain2008.com
dojki365.online domcran.net
dokfilms.net dominterior.org
dokumentalkino.net
domainsphoto.com
dorratex.tn dorratex.tn
downloads-whatsapp.com download-walpaperhd.blogspot.com
drhomes.biz downloadme.life
druzhbany.ru drugstoreforyou.com
dustyorate.com e-buyeasy.com
easyshoppermac.com e-c.al
edudocs.net e-collantes.com
elektronischezi.livejournal.com easycommerce.cf
elexies.info ebooknovel.club
elitedollars.com ekaterinburg.xrus.org
energy-ua.com elektrischezigarette1.blog.pl
elektrischezigarette2.devhub.com
elektroniksigaraankara.info
elektrozigarettekaufen2.blogse.nl
elektrozigaretten1.postbit.com
elkacentr.ru
energydiet-info.ru
enge-fotzen.info
enpolis.ru
erectile.bid erectile.bid
erotik-kostenlos.net erot.co
essay-edu.biz erotische-geschichten-xxl.com
essayassist.com estibot.com
evening-dating-club.info europages.com.ru
everypony.ru eurosamodelki.ru
executehosting.com
expolicenciaslatam.co
expresstoplivo.ru expresstoplivo.ru
eyessurgery.ru extlabs.io
ez8motelseaworldsandiego.com
ezigarettekaufen1.hpage.com
ezigaretteshop.webs.com
facialporntube.com
fakehandbags.xyz
familienzahnaerzte.com familienzahnaerzte.com
favorcosmetics.com fertilitetsradgivningen.se
favornews.com fickenprivat.info
financeloan.us filesdatabase.net
fivedwld.cf filmania-x.ru
flashahead.info filmetricsasia.com
flauntyoursite.com filmidivx.com
foodgid.net finteks.ru
forum69.info fiuxy.com
foto-weinberger.at flirt4free.com
fotopop.club floridahuntingfishingadventures.com
fortevidyoze.net
forum.tvmir.org
forzeronly.com
frameimage.org frameimage.org
franklinfire.co frighteningremain.cf
free-fbook-traffic.com front.ru
free-stock-illustration.com fym.com.ru
free-today.com gamedayassist.com
fuck-paid-share-buttons.xyz gayxperience.com
fullfileaccess.com geile-lelly.eu
funtoonez.com
fxgallery.com
g7m.pl
gamebackyard.com
games.kolossale.ru
gamevalue7.weebly.com
gaytube.com
genericlowlatencyasiodriverhq.aircus.com
getprismatic.com
getyourimage.club getyourimage.club
gfaq.ru gktt.ru
gifspics.com glavtral.ru
gigapeta.com glogow.pl
girlsatgames.ru go2album.com
golmau.host.sk gok-kasten.net
goo.ne.jp goldadpremium.com
greatgrace.ru goodnightjournal.com
growmyfunds.ca gosarhivrt.ru
gsasearchenginerankerhelp.com grand-chlen.ru
gtopstats.com greendream.com.ua
hasshe.com greenzaim.ru
healbio.ru guardlink.com
hebr.myddns-flir.com guigyverpo.cf
hikesearch.net guildebzh.info
homemade.gq guruofcasino.com
hotblog.top harmonyglen.us
hotshoppymac.com hatedriveapart.com
houseofrose.com haveinc.xyz
hoverboard360.es hazardky.net
howlongdoestizanidinestayinyoursystem.blogspot.com hdseriale.pl
ibmdatamanagement.co helicalpile.us
ilovevitaly.com highstairs-a.akamaihd.net
imgarit.pw himazin.info
homeart.space
host-tracker.com
hotblognetwork.com
hoverboard360.nl
howopen.ru
humanorightswatch.org
hut1.ru
ibb.com.ua
icaseclub.ru
iflycapetown.co.za
images.gyffu.com
img.wallpaperstock.net
infobabki.ru
informatiecentro.be
insomniagamingfestival.com insomniagamingfestival.com
iomoio.net insta-add.pro
irkutsk.online-podarki.com invest-pamm.ru
ispac.org investyb.com
itrevolution.cf ipchicken.com
jabimgo.pw isistaylorporn.info
jav-fetish.site jav-fetish.site
jaxcube.info jofucipiku.tk
jimmychoosale.online junglenet-a.akamaihd.net
job.icivil.ir
juliaworld.net
kalandranis.gr kalandranis.gr
kam-dom.ru kamagragelusa.net
keywordbasket.com kambasoft.com
keywords-monitoring-success.com karachev-city.ru
kellyonline.xyz
kenaba.su
kevblog.top
keywesthideaways.co
keywordspay.com
kfon.eu kfon.eu
kinky-fetishes.com kidskunst.info
kinoflux.net kino2018.cc
kinopolet.net kinoplen.ru
kinozapas.org kinotorka.ru
klikbonus.com kinozapas.com
konoplisemena.com kiprinform.com
konteiner24.com kit-opt.ru
koptims.tiu.ru kovrenok.ru
kosmetyki.tm.pl kraljeva-sutjeska.com
larger.io krasivye-devushki.net
layola.biz.tc krassh.ru
leboard.ru kumuk.info
lecbter-relationships.ga law-check-nine.xyz
lida-ru.com lawyersinfo.org
lincolntheatre.com lescinq.com
linerdrilling.com life-instyle.com
linkredirect.biz lignofix.ua
locksmith.jp limtu.ifmo.ru
lombardfinder.ru liverpool.gsofootball.com
lombia.com liveshoppersmac.com
losangeles-ads.com livingcanarias.com
mackeeper-center.club lmrauction.com
macotool.com long-beach-air-conditioning.com
magento-crew.net longlifelomilomi.info
mall.uk
manifestation.betteroffers.review
marktforschung-stuttgart.com
masserect.com masserect.com
matpre.top matpre.top
max-eclat.men maxxtor.eu
mcnamaratech.com mcnamaratech.com
medispainstitute.com.au medispainstitute.com.au
mega-bony-2017.pl meget.co.za
megagrabber.ru merryhouse.co.uk
memberty.com mhtr.be
mensandals.xyz mielec.pl
mesto-x.com minyetki.ru
metaxalonevstizanidine.blogspot.com mitrasound.ru
mex-annushka.ru
mezaruk.info
michaelkorssaleoutletonline.net
mikrobiologies.ru
mil-stak.com
miniature.io
mixtapetorrent.com
mmofreegames.online mmofreegames.online
mmstat.com mobile.ok.ru
mockupui.com mobilemedia.md
modnie-futbolki.net mobioffertrck.com
moesexy.com moi-glazki.ru
mojpregled.com mojaocena.com
monclerboots.xyz movies-in-theaters.net
monclercheap.xyz mrinsidesales.com
monsterdivx.com muz-shoes.ru
more-letom.ru muz-tracker.net
morefastermac.trade my-bc.ru
mossmesi.com myindospace.com
mouselink.co mymobilemoneypages.com
moz.com myphotopipe.com
mp3ritm.top naphukete.ru
mrmoneymustache.com nationalbreakdown.com
mundoaberrante.com naturalbreakthroughsresearch.com
music.utrolive.ru
musicktab.com
musicvidz.ru
muznachas-service.ru
my-aladin.com
myblogregistercm.tk
myperiod.club
mystats.xyz
naj-filmy24.pl
najaden.no
narkologiya-penza.ru
naturalshair.site
nbsproject.ru
neobux-bg.info
netanalytics.xyz
netcheckcdn.xyz
netoil.no
new-apps.ru new-apps.ru
newstaffadsshop.club nfvsz.com
nightvision746.weebly.com njpalletremoval.com
nootrino.com
nordstar.pro nordstar.pro
novatech.vn nowtorrents.com
npoet.ru oballergiya.ru
nrjmobile.fr obiavo.com
oakridgemo.com odesproperty.com
ochistka-stokov.ru old-rock.com
oecnhs.info onblastblog.online
offergroup.info onlinemeetingnow.com
okout.ru onlyporno.ru
oksrv.com onlythegames.com
onlywoman.org onlywoman.org
ontargetseo.us onstrapon.purplesphere.in
ooo-olni.ru onthemarch.co
ordernorxx.com openlibrary.org
oren-cats.ru outshop.ru
pandarastore.top oyster-green.com
paperwritingservice17.blogspot.ru papasdelivery.ru
partner-pop.men partner-host.men
pc-services.ru patterntrader-en.com
pearlisland.ru pcboa.se
photosaga.info pdn-4.com
picsforkeywordsuggestion.com pelfind.me
pizda.lol petedrummond.com
pkr1hand.com pixell.club
pod-muzyku.club pizdopletka.club
pokemon-go-play.online play-mp3.com
pokemongooo.ml playlott.com
pontiacsolstice.info playmsn.com
popup.matchmaker.com pokerniydom.ru
pornhive.org poligon.com
porno-transsexuals.ru pony-business.com
pornodojd.ru popmarker.com
pornolook.net pornolook.net
pornozhara.com pornosee.info
pourvous.info potoideas.us
predominant-invent.tk prizesbook.online
privetsochi.ru pro-tec.kz
professionaldieselcare.com prostitutki-tyumeni.org
proposal-engine.com pskcijdc.bloger.index.hr
prosmibank.ru ptr.ruvds.com
prostitutki-kharkova.org pufip.com
pussyfleet.com py100.ru
puteshestvennik.com quality-traffic.com
ranksonic.org quickbuck.com
redhotfreebies.co.uk r-control.ru
reelheroes.net rankia.com
registratciya-v-moskve.ru ranksignals.com
relatodelpresente.com.ar redirect.trafficreceiver.club
remmling.de reeyanaturopathy.com
remont-ustanovka-tehniki.ru rep-am.com
renecaovilla.online
retailwith.com
rfd-split.hr
riralmolamsaca.tk
rocis.site rocis.site
roznica.com.ua royalads.net
rrutw.com rubanners.com
sabizonline.com ruclicks.com
sad-torg.com.ua rustic-quiver.win
sammlungfotos.online sadaholding.com
sashagreyblog.ga safe-app.net
sayyoethe.blogspot.co.za salut-camp.ru
sbornik-zakonov.ru sammyweaver.com
score-ads.men sankt-peterburg.nodup.ru
search.1and1.com santechnik.jimdo.com
searchlock.com savefrom.com
semalt.com scanner-alexa.top
seo-2-0.com scanner-andrew.top
scanner-brian.top
scanner-george.top
sdrescher.net
searchaddis.com
searchimage.co
searchkut.com
searchtooknow-a.akamaihd.net
seccioncontrabajo.com
seksotur.ru
sel-hoz.com
sell-fb-group-here.com
seminarygeorgia59.ga
seo-platform.com
seo-prof1.xyz
seofirmreviewsus.info seofirmreviewsus.info
seonetwizard.com seomarketings.online
serptehnika.ru sex.hotblog.top
sexcamamateurchat.com sexobzor.info
sexfreepornoxxx.com sexsaoy.com
sexpornotales.com sextracker.com
sexuria.net sexuria.net
shell-pmr.ru shivafurnishings.com
shodanhq.com signal03.ru
shopsellcardsdumps.com simplepooltips.com
silvercash.com sinel.info
site-speed-check.site sitebeam.net
sitechecker.pro
siteonomy.com siteonomy.com
skachat-besplatno-obrazcy.ru sitevaluation.com
skutecznetabletkinaporostwlosow.pl sixcooler.de
skypasss.com sklad-24.ru
slavia.info slutloadlive.com
sloopyjoes.com smartshoppymac.com
smartpet.ru snow.nvr163.com
smichovbike.cz social-buttons.com
snip.tw social-s-iii.xyz
snjatie-geroinovoy-lomki.ru
social-button.xyz
social-buttons.xyz
social-fun.ru
socialbutton.xyz
socialtrade.biz
softonicads.com softonicads.com
solution4u.com softtor.com
sonata-arctica.wz.cz softxaker.ru
soserfis.com soviet-portal.do.am
spb-plitka.ru space-worry.ml
spb.afora.ru space2019.top
ssconstruction.co spravka-medosmotr.ru
steamoff.net spywarenuker.com
stroiminsk.org stard.shop
styro.ru stathat.com
supermama.top stretchmate.net
supervesti.ru stroy-portal22.ru
survival.betteroffers.review studworks.org
swinger-mobil.net stuff-about-money.com
swsociety.se sugarlyflex.pw
tacbelarus.ru supers.com.ua
taraz.xkaz.org svnuppsalaorebro.se
tdsing.ru t-bygg.com
tedxrj.com taihouse.ru
tehngr.ru tampabaywatch.org
telegramdownload10.com thedownloadfromwarez.blogspot.com
template-kid.com thegalerie.eu
tfxiq.com thegolfclub.info
tgtclick.com thelottosecrets.com
thebluffs.com ticketsys.inetwd.com
thefarmergame.com timeallnews.ru
thin.me.pn titslove.yopoint.in
tizanidine4mgstreetvalue.blogspot.com tivolibasket.it
tizanidineinteractions.blogspot.com tizanidine6mg.blogspot.com
toonfamilies.net tizanidineandhydrocodone.blogspot.com
topanasex.com tizanidinedosage.blogspot.com
topdownloads.ru tizanidinesleepaid.blogspot.com
toy-shop.top tizanidinewithdrawal.blogspot.com
traffic2money.com tjkckpytpnje.com
traffixer.com tongkatmadura.info
triplepanda.xyz tooplay.com
trustaffs.com toproadrunner5.info
turbodsp.com trafficjunky.com
turizm.bz trafficmonetize.org
turvgori.ru trafficmp.com
tvorozhnaja-zapekanka-recept.ru trahic.ru
ua-company.ru ttrraacckkrr.com
tula.howotorg.ru
typer.one
uasb.ru uasb.ru
undergroundcityphoto.com ucsol.ru
univerfiles.com uptime.com
uptime-alpha.net url-extractor.xyz
uptime-as.net usdx.us
urdoot.win utrolive.ru
url2image.com v24s.net
urzedowski.eu vduplo.ru
usbggettwku.ga vellings.info
uvozdeckych.info veopornogratis.xxx
vadimkravtcov.ru vezuviy.su
valsalud.com
vapsy.com
vereo.eu
video-chat.love video-chat.love
videochat.tv.br vip-dom.in
vitalads.net vip2ch.com
vkonche.com voltrknc1.com
vozbujdenie.com vote-up.ru
vrotike.ru vpnmouse.com
vtc.pw vtc.pw
w3javascript.com walpaperlist.com
wallpapersdesk.info
wallpaperstock.net
wanker.us wanker.us
waysbetter.cn webadvance.club
wdfdocando.com
webix.biz
webmasterhome.cn
website-speed-up.top website-speed-up.top
websites-reviews.com websiteexplorer.info
websitevaluebot.com wellcome2slovenia.ru
weburlopener.com wetgames.ru
whatistizanidinehclusedfor.blogspot.com whatistizanidine2mg.blogspot.com
wholesalejerseychinaoutlet.com whereverdesperate.gq
wnhjavlhezp.gq whipme.yopoint.in
whos.amung.us
withstandingheartwarming.com
workle.website
worldhistory.biz worldhistory.biz
wovis.site www.wohnkabinen-shop.de
writersgroup580.web.fc2.com xaijo.com
ws.ampower.me xchangetrak.com
www.ehscloud.cn xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai
www.event-tracking.com xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
www.jbetting.com xn--80ab4aa2g.xn--p1ai
xcc24.pl xn--80aeahghtf8ac5i.xn--p1ai
xn----7sbho2agebbhlivy.xn--p1ai xn--b1ag5cfn.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai xn--d1acah0c.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai xn--e1aggki3c.xn--80adxhks
xn--b1adccaf1bzj.xn--p1ai xolodremont.ru
xn--c1acygb.xn--p1ai xtrafficplus.com
xn--d1abj0abs9d.in.ua xxxhdvideo.site
xn--lifehacer-1rb.com xxxnatelefon.ru
xxxtube69.com xzlive.com
yebocasino.co.za yaoguangdj.com
yellowstonesafaritours.com yebocasino.com
yoluxuryevents.com yellowads.men
yopoint.in ynymnwbm.bloger.index.hr
youporn-forum.ga yogamatsexpert.com
youtubologia.it yougetsignal.com
youtuhe.com youradulthosting.com
zahvat.ru yourdesires.ru
zarepta.com yubikk.info
zeroredirect5.com za-fun-offer.com
zeroredirect9.com zakonobosago.ru
zoompegs.com zaloadi.ru
zscalerone.net zatjmuzu.info
zvuker.net zebramart.ru

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1xbet4you.com 03e.info
20pascals.nl 0daymusic.org
1webmaster.ml
214jaluwobapef.cf
2itech.ru
51.la 51.la
51unlim.ru 883zy.com
5forex.ru aa625d84f1587749c1ab011d6f269f7d64.com
a96527gi.beget.tech abctoppictures.net
aa81bf391151884adfa3dd677e41f94be1.com actionnooz.com
adconscious.com admanaerofoil.com
addray.pro admeasures.com
adidas.frwebs.fr adprotect.net
adsland.men ads.gold
advokat-grodno.by adzpower.com
aktobe.xkaz.org ahmedabadwebs.com
allbizne.co.ua alert-jdh.xyz
allkrim.com all-streaming-media.com
alltheviews.com
altamayoztourism.com altamayoztourism.com
an-donut.com anal-acrobats.hol.es
animal-drawings.com anmysite.com
aquarium-pro.ru appfixing.space
arcadeplayhouse.com aversis.net
atofilms.com avitocars.ru
aviav.co b00kmarks.com
avon-severozapad.ru
azazaporn.com
babieca.com
babyfactory.fr babyfactory.fr
bagcionderlab.com backlink4u.net
battlecarnival.su backuperwebcam.weebly.com
beget.tech bear.gotcher.us
best-seo-solution.com berrymall.ru
bet-prognoz.com bestchoice.cf
bigcareer.info
bioca.org bioca.org
blogporn.in bizzliving.com
bracketsmackdown.com blackplanet.com
braintobe.win blogrankers.com
bukleteg.xyz blogsfunky672.weebly.com
burn-fat.ga bluesman.nu
calcularpagerank.com bongiornos.info
cartierreplica.xyz brus.city
cdncash.org bumskontakte.org
cdnnetwok.xyz buypuppies.ca
cellfun.mobi cardsdumps.com
chelyabinsk.xrus.org casablancamanor.co.za
cherubinimobili.it chcu.net
chocolateslim-u-srbiji.com chimiver.info
chuckguilford.com
cialisndbrx.com
cialisovercounteratwalmartusa.com cialisovercounteratwalmartusa.com
cinemaenergy-hd.ru
civilwartheater.com
coaching-netz.info coaching-netz.info
codysbbq.com com-secure.download
conocer-sanabria.com com-supportcenter.website
cookielawblog.wordpress.com compliance-julianna.top
coolyarddecorations.com connectingsingles.com
coslab.club constantaservice.net
credit-online.ws contentsexpress.com
crystalslot.com crazyprotocol.com
cubs-jerseys.us credit-respect.ru
customcollegeessays.net cutt.us
cyclobenzaprinevstizanidine.blogspot.com danschawbel.com
daretodonate.co davebestdeals.com
defenderxtactical.com dekoration.us
devochki.top dekorkeramik.ru
dimfour.com designdevise.com
domik-derevne.ru discovertreasurenow.com
domoysshop.ru distver.ru
elitepcgames.com divku.ru
emailaccountlogin.co dosug-lux.ru
emergencyneeds.org doxyporno.com
erotische-geschichten-xxl.com dtnlyss.com
escortplius.com e2click.com
esnm.ru e90post.com
executehosting.com earncash.com.ua
extads.net editors.choice6912650.hulfingtonpost.com
ezigarettekaufen2.yolasite.com educontest.net
f-online.de egvar.net
fenoyl.batcave.net eluxer.net
findpik.com eropornosex.ru
fix-website-errors.com erotikstories.ru
fm-upgrade.ru erotiktreff24.info
frameimage.org eu-cookie-law.info
franklinfire.co exchanges-bet.com
free-fb-traffic.com exonline.info
freefoto.ca fellowshipoftheminds.com
freewlan.info filmania-x.ru
fungamelands.com fingerprintjs.com
fxgallery.com free-laptop-reward.com
galaxy-family.ru free-traffic.xyz
galeon.com gamebackyard.com
game-mmorpg.net gaytube.com
game300.ru geileweiber.tk
gateway.zscalertwo.net genetworx.com
genericlowlatencyasiodriverhq.aircus.com get-your-social-buttons.info
girlsfuckdick.com globalsurfari.com
glopages.ru gsasearchengineranker.pw
googlsucks.com hatedriveapart.com
gotorussia.com hccoder.info
greatzip.com hoverboard360.se
hamacapty.com icaseclub.ru
hamilton.ca igtools.club
hard-porn.mobi images-graphics-pics.com
himazin.info imfamous.info
hitsbox.info imperia31.ru
hothot.ru incomekey.net
hotloans.ru intim-uslugi.info
https-legalrc.biz iqs.biz.ua
imgata.com iron-age.info
indetiske.ya.ru islandminingsupply.wordpress.com
insider.pro jackwolfskinoutlet.online
investsuccess.org jbl-charge.info
invitefashion.com journeydownthescale.info
ipornox.xxx kinozapas.com
isotoner.com kostenlos-sexvideos.com
itronics.ca labplus.ru
izhevsk.xrus.org laptoper.net
izismile.com larutti.ru
jav-way.site lflash.ru
javcoast.com littlesunraiser.com
kareliatobacco.ru lookingglassemb.com
kashubadesign.ru m.b00kmarks.com
katjimej.blog.fc2.com marinetraffic.com
kaz.kz marketingvici.com
keywesthideaways.co marktforschung-stuttgart.com
keywordspay.com medi-fitt.hu
kmd-pto.ru medizinreisen.de
koptims.tiu.ru menhealed.net
kutikomi.net menstennisforums.com
landinez.co mirzonru.net
lankarns.com mjchamonix.org
laptop-4-less.com modnie-futbolki.net
linkrr.com mompussy.net
lovelycraftyhome.com monthlywinners.com
lutherstable.org mp3downloadhq.com
m4ever.net
maju.bluesalt.co
maladot.com
managerpak204.weebly.com
medanestesia.ru
medicovi.com
meds-online24.com
mncrftpcs.com
moviezbonkerssk.cf
mrsdalloways.com mrsdalloways.com
mykings.pw myseoconsultant.com
mymobilemoneypages.com nationalbreakdown.com
nakrutka.cc newyorkhotelsmotels.info
natali-forex.com nibbler.silktide.com
netvouz.com no-rx.info
news-readers.ru nodup.ru
newtechspb.ru novodigs.com
normalegal.ru o-o-11-o-o.com
norththeface.store omgtnc.com
notebook-pro.ru onescreen.cc
nrv.co.za oneshotdate.com
nudejapan.net ordernorxx.com
obiavo.net ororodnik.goodbb.ru
olympescort.com paleohub.info
parajumpersstore.online panchro.co.uk
perfectplanned.com partnersafe.men
pawli.eu
pestomou.info pestomou.info
petsblogroll.com petrovka-online.com
phobia.us picquery.com
photokitchendesign.com placid-rounded-coast.glitch.me
photosaga.info pooleroadmedicalcentre.co.uk
pillscheap24h.com popup.matchmaker.com
pizda.lol poquoson.org
pizdeishn.com porno-raskazy.ru
pizdopletka.club pornofeuer.com
pizza-imperia.com pornosexrolik.com
pl-top.pl portnoff.od.ua
plastweb.ru portside.xyz
playlott.com potoideas.us
porno-dojki.net prchecker.info
pricheskaonline.ru priceg.com
pricheski-video.com privhosting.com
privatbank46.ru prizesbook.online
psoriasis-file.trade prombudpostach.com.ua
rapidgator-porn.ga purplesphere.in
realitykings.com randki-sex.com
recinziireale.com ranksonic.info
relayblog.com raschtextil.com.ua
runofilms.ru rating-casino2021.ru
rusprostitute.com rixpix.ru
rybalka-opt.ru robertefuller.com
sbricur.com russia-today-video.ru
schlampen-treffen.com s-forum.biz
scmor.ilxc.cc satellite.maps.ilovevitaly.com
searchaddis.com saveriopiazza.it
seorankinglinks.xyz savetubevideo.com
shariki-zuma-lines.ru scanner-barak.top
shopcheermakeup.info scanner-marwin.top
skylta.com search.alot.com
social-fun.ru seo-2-0.com
soda.media seoholding.com
solution4u.com sexpartygirls.net
spacash.com shillyourcoins.com
staff.prairiesouth.ca shlyahten.ru
stillmiracle.com shopvilleroyboch.com.ua
stroilka.info sientalyric.co
strongsignal-a.akamaihd.net signoredom.com
studworks.org sivs.ru
supermesta.ru skytraf.xyz
svarbit.com smart-balancewheel.com
tampabaywatch.org smsactivator.ru
taqywu51.soup.io smstraf.ru
td-l-market.ru snsdeainavi.info
tecnoteakviareggio.it soc-proof.su
thegioixekhach.com softomix.com
tizanidine4mgstreetvalue.blogspot.com soochi.co
tizanidineonline.blogspot.com stal-rulon.ru
toondinsey.com steebook.com
top-l2.com stickers-market.ru
stockquotes.wooeb.com
strana-solnca.ru
teenforporn.com
template-kid.com
thfox.com
thin.me.pn
tizanidine4mgstreetprice.blogspot.com
tizanidinehcl2mg.blogspot.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidinewithdrawal.blogspot.com
tomck.com
tonivedu.it
topappspro.com topappspro.com
topmira.com tracksurf.daooda.com
topsiteminecraft.com tula.howotorg.ru
track2.shop turkeyreport.tk
unionmarkt.de tvory.predmety.in.ua
unmaroll.ya.ru udayavani.com
v24s.net vasileostrovsky-rayon.ru
vacances-voyages.info vchulkax.com
valsalud.com videokrik.net
videosbox.ru vipromoffers.com
viktoria-center.ru vkmusics.ru
wallpaperstock.net vtc.pw
wardreapptokone.tk w-journal.ru
watracker.net wallpapers-best.com
website-stealer.nufaq.com webix.biz
wedding0venues.tk websites-reviews.com
wildcattube.com webstatsdomain.org
win-spy.com westermarkanjou.se
wiosenny-bon-1500.pl windowssearch-exp.com
withstandingheartwarming.com wineration.com
wnoz.de ww1943.ru
womensterritory.ru x-musics.com
www.souvenirua.com x-true.info
wwwadultcheck.com xev.ru
wzgyyq.com xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xchangetrak.com xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xep.info xnxxandxvideos.com
xn----7sbifcamovvfggw9d.xn--p1ai xxxsiterips.xyz
xn----btbdvdh4aafrfciljm6k.xn--p1ai youdesigner.kz
xxxhdvideo.site yourtemplatefinder.com
xxxtubesafari.com youtube-downloader.savetubevideo.com
yhit.press zapatosenventa.info
yopoint.in zeroredirect12.com
youdao.com zoolubimets.ru
yunque.pluto.ro zreloeporno.tv
zverokruh-shop.cz zscalertwo.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.195 0; 10.1.1.54 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2636 ### Version: V4.2021.10.2637
### Updated: Thu Oct 7 10:03:18 UTC 2021 ### Updated: Thu Oct 7 22:01:34 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2636 ### Version: V4.2021.10.2637
### Updated: Thu Oct 7 10:03:18 UTC 2021 ### Updated: Thu Oct 7 22:01:34 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##