vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 19:50:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.12.3504 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-12-02 22:07:03 +00:00
parent fc2397c911
commit b100fc8203
46 changed files with 90065 additions and 90065 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3503 #### Version: V4.2022.12.3504
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3503 #### Version: V4.2022.12.3504
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.12.3503 #### Version: V4.2022.12.3504
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11122
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.12.3503 #### Version: V4.2022.12.3504
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11126
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11126
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11126
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11126
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.17 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.17 0;
127.0.0.1 0; 127.0.0.1 0;

11126
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.17 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.17 0;
127.0.0.1 0; 127.0.0.1 0;

11126
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.17 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.17 0;
127.0.0.1 0; 127.0.0.1 0;

11126
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3503 3504

11126
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11126
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.14.185.164 0; 172.177.210.40 0;

11126
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.17 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.17 0;
127.0.0.1 0; 127.0.0.1 0;

11126
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.17 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.17 0;
127.0.0.1 0; 127.0.0.1 0;

11126
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.17 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.17 0;
127.0.0.1 0; 127.0.0.1 0;

11126
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.14.185.164 0; 172.177.210.40 0;

11126
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.14.185.164 0; 172.177.210.40 0;

11126
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 1; 10.1.0.17 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
DomainSigmaCrawler CSHttp
HTTP::Lite Dotbot
Image Fetch Fyrebot
LinqiaScrapeBot Go-Ahead-Got-It
Mass Downloader Magnet
Picsearch Not
SEOkicks-Robot OpenLinkProfiler
V-BOT Scanbot
heritrix Sitebeam
scalaj-http Telesphoreo

322
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404enemy 404checker
ADmantX ADmantX
ALittle Client
ASPSeek ASPSeek
Abonti Aboundexbot
Aboundex
Acunetix Acunetix
AfD-Verbotsverfahren
AllSubmitter AllSubmitter
Alligator
AlphaBot
Anarchy
Ankit Ankit
Anthill
Apexoo
Aspiegel Aspiegel
Asterias Atomseobot
AwarioRssBot Attach
AwarioSmartBot AwarioSmartBot
BBBike BBBike
Backlink-Ceck BLEXBot
BacklinkCrawler
Badass Badass
Bandit Battleztar Bazinga
Barkrowler
BetaBot
Bigfoot
Bitacle Bitacle
Black Hole BlackWidow
Blackboard
BlowFish
Boardreader Boardreader
Bolt
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
Bullseye
BunnySlippers BunnySlippers
CATExplorador
CCBot CCBot
CODE87 CSHttp
Calculon
CazoodleBot CazoodleBot
Cegbfeieh CheTeam
CensysInspect CheeseBot
CherryPicker CherryPicker
Citoid Cloud mapping
Cogentbot Collector
CopyRightCheck CopyRightCheck
Copyscape
Cosmos
Crawling at Home Project
CrazyWebCrawler
Crescent Crescent
CrunchBot
Curious
Custo Custo
CyotekWebCopy DTS Agent
DataCha0s Demon
Deusu Discoverybot
DigitalPebble Dispatch
Disco DnBCrawler-Analytics
Discobot Download Wonder
DomainStatsBot ECCP/1.0
Dotbot
Dragonfly
EMail Siphon EMail Siphon
EasyDL Ebingbong
Ecxi
EroCrawler EroCrawler
Evil Evil
Exabot
ExtLinksBot
Extractor Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
FDM FDM
FHscan
FemtosearchBot FemtosearchBot
FlashGet
Flunky Flunky
FrontPage FrontPage
Fuzz FyberSpider
G-i-g-a-b-o-t
Genieo
GermCrawler GermCrawler
Getintent GetRight
Go!Zilla Gigabot
GoZilla GoZilla
GrabNet Gotit
HMView GrapeFX
HTMLparser GrapeshotCrawler
Havij HTTP::Lite
HonoluluBot Haansoft
HybridBot HybridBot
IDBot
Iblog Iblog
Image Sucker Id-search
IndeedBot IlseBot
InfoNaviRobot InfoNaviRobot
InfoTekies InfoTekies
InternetSeer InterGET
Internet Ninja
Iskanie Iskanie
JamesBOT
Jbrofuzz
Jetty Jetty
Joomla Jorgee
JustView JustView
Keybot Translation-Search-Machine Jyxobot
Kenjin Spider
Keyword Density
Kinza Kinza
Kozmosbot Lanshanbot
LNSpiderguy Leap
Larbin LinkWalker
LeechGet
LibWeb
Libwhisker
LinkpadBot
LinksManager LinksManager
LinqiaRSSBot LinqiaRSSBot
Lipperhey Litemage_walker
Lmspider Lmspider
MJ12bot Majestic SEO
MSFrontPage
MSIECrawler
MTRobot
Mail.RU_Bot
Majestic12
Mass Downloader Mass Downloader
Mata Hari Mata Hari
MauiBot Mb2345Browser
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru Metauri
MicroMessenger
Microsoft URL Control Microsoft URL Control
Minefield
Mojeek
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Mozlila
Mr.4x3 Mr.4x3
NICErsPRO
NPbot
Navroad
NearSite NearSite
Nibbler Needle
Nimbostratus Nessus
Net Vampire
NetSpider
Netcraft
Nettrack
NextGenSearchBot
Niki-bot
Ninja
Nmap Nmap
Not Nutch
Nuclei Octopus
Offline Explorer
OnCrawl OnCrawl
Openvas OpenLinkProfiler
OrangeBot Openfind
OrangeSpider
OutfoxBot
PECL::HTTP
PageAnalyzer PageAnalyzer
PageGrabber
PageScorer PageScorer
PageThing.com Panscient
Pandalytics Pavuk
Pi-Monster PeoplePal
Picsearch Picsearch
Piepmatz Piepmatz
PleaseCrawl Pimonster
ProWebWalker Pixray
Probethenet Pu_iN
PxBroker PxBroker
Quick-Crawler
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankurBot RankingBot
RankingBot2
Rankivabot
ReGet
RealDownload RealDownload
Reaper Recorder
Ripper Ripper
RocketCrawler
Rogerbot Rogerbot
SBIder SBIder
SEOkicks-Robot SEOkicks
SEOprofiler SEOlyticsCrawler
SISTRIX ScrepyBot
ScanAlert Semrush
Scanbot
ScoutJet
Scrapy
Screaming
SearchmetricsBot
SemanticJuice
SemrushBot SemrushBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot Seomoz
Site Sucker Siphon
SiteCheckerBotCrawler SiteExplorer
SiteSnagger SiteSnagger
SiteSucker
Sitebeam
Siteimprove Siteimprove
Sitevigil
SlySearch
Snake Snake
Spbot Snapbot
Spinn3r SocialRankIOBot
Sqlworm Sogou web spider
Stripper SpankBot
Sucuri Spanner
SurveyBot
Suzuran Suzuran
T0PHackTeam T0PHackTeam
TeleportPro T8Abot
Telesphoreo Teleport
Telesphorep Telesphorep
Thumbor The Intraformant
Toata TightTwatBot
Toweyabot Trendictionbot
Tracemyfile True_Robot
TurnitinBot Turingos
Twice TwengaBot
Typhoeus
URLy.Warning
UnisterBot
Upflow Upflow
V-BOT
VB Project
Vacuum Vacuum
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
Virusdie VoidEYE
Voltron Voltron
WWW-Collector-E WASALive-Bot
Web Collage WEBDAV
Web Fetch WWW::Mechanize
Web Pix Wallpapers/3.0
WebAuto WeSEE
WebReaper Web Auto
WebStripper Web Enhancer
WebWhacker Web Sucker
WebCollage
WebCopier
WebEnhancer
WebFetch
WebFuck
WebImageCollector
WebLeacher
WebPix
WebSauger
Webalta Webalta
Webshag WebmasterWorldForumBot
Whack Whack
Whacker
Whatweb Whatweb
Widow WiseGuys Robot
WinHTTrack Wonderbot
Wotbox Woobot
Wprecon
Xaldon WebSpider
Zermelo Zermelo
backlink-check Zitebot
check1.exe ZmEu
clark-crawler ZoomBot
coccocbot ZoominfoBot
crawl.sogou.com archive.org_bot
crawler.feedback crawler.feedback
demandbase-bot crawler4j
domainsproject.org dataforseo.com
gopher
heritrix
instabid instabid
ips-agent ips-agent
isitwp.com linkdexbot
iubenda-radar netEstate NE Crawler
lwp-trivial
mediawords
muhstik-scan
oBot oBot
page scorer
plumanalytics
polaris version polaris version
ripz ripz
s1z.ru
scalaj-http scalaj-http
seobility seobility
seocompany.store
seostar
serpstatbot serpstatbot
sexsearcher siteripz
sitechecker.pro sp_auditbot
sogouspider spyfu
trendiction.de tAkeOut
ubermetrics-technologies.com
webgains-bot webgains-bot
webmeup-crawler xpymep1.exe
webprosbot zauba.io
x22Mozilla
zgrab

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider AIBOT
ADmantX
ALittle Client ALittle Client
Abonti Abonti
Aboundex Aipbot
Acunetix
AhrefsBot
AiHitBot
Alexibot Alexibot
AllSubmitter
Anarchy Anarchy
Anthill Anthill
Apexoo
Asterias Asterias
Atomseobot Atomseobot
Attach Barkrowler
AwarioRssBot
BBBike
BetaBot BetaBot
Bigfoot Bigfoot
Bitacle
Black Hole
BlackWidow BlackWidow
Blackboard Blow
BlowFish Boardreader
Bolt Brandwatch
Brandprotect Buddy
Buck Bullseye
BuiltBotTough BuzzSumo
CATExplorador
CODE87 CODE87
Calculon CensysInspect
Cegbfeieh ChinaClaw
CheTeam Cocolyzebot
Chlooe Copyscape
Citoid Craftbot
Cloud mapping
Cogentbot
CopyRightCheck
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler CyotekWebCopy
CrunchBot DSearch
DTS Agent DatabaseDriverMysqli
Demon
DigitalPebble DigitalPebble
Dirbuster Dirbuster
Disco DnBCrawler-Analytics
Dispatch DomCopBot
Dotbot DomainAppender
Dragonfly DomainCrawler
DomainSigmaCrawler
Drip Drip
ECCP/1.0 ECCP/1.0
EasyDL EasyDL
Ebingbong
EroCrawler EroCrawler
Evil
Express WebPictures
ExtLinksBot ExtLinksBot
Extractor Extractor
EyeNetIE
FHscan FHscan
FlashGet Firefox/7.0
Flunky Flunky
Fyrebot Foobot
G-i-g-a-b-o-t Freeuploader
FrontPage
Fuzz
GT::WWW
GalaxyBot GalaxyBot
Genieo GermCrawler
GetWeb GetRight
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit Gotit
GrabNet Grafula
Grabber GrapeFX
HEADMasterSEO GrapeshotCrawler
HTMLparser
HTTrack
Heritrix Heritrix
Hloader Hloader
Humanlinks
HybridBot HybridBot
IDBot IDBTE4M
IRLbot Image Fetch
IndeedBot Image Sucker
InfoTekies Indy Library
InterGET Intelliseek
Iria Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JennyBot Joomla
Jetty Jorgee
JustView
Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kinza
Kozmosbot
LNSpiderguy
Leap
LeechFTP
LeechGet LeechGet
LibWeb Lftp
Linkbot Lightspeedsystems
LinkScan
LinkWalker
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71 MFC_Tear_Sample
MSFrontPage MIDown tool
Mail.RU_Bot MQQBrowser
Magnet
Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkWatch
Mass Downloader Mass Downloader
Mata Hari Mata Hari
MauiBot MauiBot
Meanpathbot MeanPath Bot
MegaIndex.ru MegaIndex.ru
MicroMessenger
Mister PiX Mister PiX
Mojeek Moblie Safari
MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mr.4x3 Msrabot
Name Intelligence NICErsPRO
Nameprotect NPbot
NetAnts Nessus
Net Vampire
NetLyzer
NetZIP NetZIP
Netvibes Niki-bot
Nibbler Not
Nmap Octopus
Nuclei OrangeSpider
Nutch PECL::HTTP
Offline Navigator PageAnalyzer
OpenLinkProfiler Petalbot
Openfind
Openvas
OrangeBot
OutclicksBot
POE-Component-Client-HTTP
Page Analyzer
Pandalytics
Papa Foto
Pi-Monster Pi-Monster
Picsearch Pimonster
Pixray Pixray
PleaseCrawl PleaseCrawl
Probethenet ProPowerBot
Psbot Pump
RSSingBot PxBroker
PyCurl
QueryN Metasearch
Quick-Crawler
RankActive RankActive
RankFlex RankFlex
RankingBot
RankingBot2
Re-re
ReGet ReGet
RealDownload RealDownload
RedesScrapy
RepoMonkey RepoMonkey
Ripper RocketCrawler
Rogerbot
SBIder
SEOkicks SEOkicks
SEOlyticsCrawler SEOlyticsCrawler
SEOstats SEOprofiler
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScanAlert Scanbot
ScoutJet ScoutJet
ScreenerBot
ScrepyBot ScrepyBot
SearchmetricsBot
Semrush
SentiBot SentiBot
SeobilityBot SeobilityBot
Shodan SiteCheckerBotCrawler
Siphon SiteSnagger
SiteExplorer Sitevigil
SiteLockSpider
Sitebeam
SlySearch
SmartDownload
Snoopy
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider Sogou web spider
Sosospider
SpaceBison SpaceBison
Spanner Spammen
Spinn3r
SputnikBot SputnikBot
Steeler Sqlmap
Stripper Sqlworm
Sucker Sucker
Surfbot Sucuri
Suzuran SuperBot
T8Abot SuperHTTP
TeleportPro SurveyBot
Swiftbot
Telesoft Telesoft
Telesphoreo
Telesphorep Telesphorep
TightTwatBot
Toata
Tracemyfile Tracemyfile
Trendiction Trendiction
Trendictionbot True_Robot
Turnitin
TurnitinBot
TwengaBot TwengaBot
Twice Twice
UnisterBot URLy.Warning
Upflow Upflow
VB Project
VeriCiteCrawler VeriCiteCrawler
VidibleScraper WBSearchBot
VoidEYE WWW-Mechanize
WASALive-Bot WWW::Mechanize
WEBDAV Wallpapers
WISENutbot
WWW-Collector-E
WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Enhancer Web Enhancer
WebBandit Web Fuck
WebEnhancer Web Pix
Web Sauger
Web Sucker
WebAuto
WebCollage
WebFetch
WebFuck WebFuck
WebLeacher WebLeacher
WebReaper
WebSucker
WebWhacker
Webalta
WebmasterWorldForumBot
Webshag
WebsiteExtractor WebsiteExtractor
WebsiteQuester WebsiteQuester
Whack Whack
Whacker Who.is Bot
Woobot WiseGuys Robot
Wonderbot
Xaldon WebSpider Xaldon WebSpider
Xenu
YoudaoBot
Zade Zade
Zeus Zermelo
ZoomBot Zitebot
ZoominfoBot ZoominfoBot
arquivo-web-crawler ZumBot
adscanner
archive.org_bot
check1.exe
clark-crawler clark-crawler
crawl.sogou.com coccocbot
crawler4j cognitiveseo
dataforseo.com dataforseo.com
evc-batch domainsproject.org
eCatch
facebookscraper facebookscraper
gopher
heritrix heritrix
instabid ips-agent
isitwp.com isitwp.com
linkdexbot lwp-request
meanpathbot
netEstate NE Crawler
page scorer page scorer
pcBrowser probe-image-size
polaris version ripz
scalaj-http satoristudio.net
scan.lol
seocompany.store seocompany.store
serpstatbot seostar
siteripz sexsearcher
sitechecker.pro
sp_auditbot
spyfu spyfu
sysscan sysscan
trendiction.com webgains-bot
trendiction.de
ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
webprosbot
xpymep1.exe xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
2x2fan.ru amoi.tn
animal-drawings.com apiallgeniusinfo-a.akamaihd.net
ilikevitaly.com ezofest.sk
lionking-1994.blogspot.com hatedriveapart.com
lostaruban.com pdn-4.com
misandesign.se printdirectforless.com
torrent-newgames.com rankingchart.de
wrona.it servethis.com
xn--d1acah0c.xn--p1ai theautoprofit.ml
zawyna.ua zigarettenonlinekaufen2.bloog.pl

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
100dollars-seo.com 1001watch.com.ua
12u.info 20pascals.nl
2015god.org 2kata.ru
2nt.ru 4inn.ru
3weekdiet.com 51unlim.ru
45en.ru 76brighton.co.uk
7yue.org adamoads.com
991mostfm.co.id adman.se
aadroid.net adobereader-free.ru
aasoldes.fr ads-cool.pro
absolutelyanalog.com advokateg.xyz
actulite.com
addblueoff.com.ua
adexprts.com
adultwebhosting.info
adviceforum.info
agapovdg.ru agapovdg.ru
aghanyna.com
aimayangzhi.com
aktivator-windows10.blogspot.com
alert-hgd.xyz
alevinefesleri.com alevinefesleri.com
alfa-medosmotr.ru alienwheels.de
alkoravto.ru all4invest.info
all4wap.ru allfinweb.com
allcredits.su amt-k.ru
alloysteel.ru analnoeporno.tv
aludecor.info angel1777.kz
amanda-porn.ga animebox.com.ua
amoi.tn anonymizeme.pro
ampower.me apartmentratings.com
anaksma.info approvedlocal.co.za
anaseracresar.tk arabseyes.com
animal-drawings.com arendatora.ru
animal36.com arquapetrarca.info
apifasterlightin-a.akamaihd.net ausmepa.org.au
apxeo.info autobrennero.it
arenda-yeisk.ru autoloans.com
artavenuegardenstudios.com avitocars.ru
artpicso.com b-styles.xyz
arvut.org babyguns.ru
atovh.local-finders.com bad-stars.net
auto-zapchasti.org
auto.rusvile.lt
autoua.top
avtocredit-legko.ru
backlink4u.net
badavit.com.ua
bala.getenjoyment.net
balkanfarma.ru
bamo.xsl.pt
bankofthewext.com
bard-real.com.ua bard-real.com.ua
bazaronline24.ru belstaffstore.online
best-coupon-offer.com
bestmobilityscooterstoday.com bestmobilityscooterstoday.com
bestpriceninja.com
bestprofits-there.com
besttorrentknifta.weebly.com besttorrentknifta.weebly.com
bfz.biz bharatdefencekavach.com
bio-optomarket.ru bidr.trellian.com
bizcheapjerseyswholesalechina.com biglistofwebsites.com
bjgugu.net.ua bkgr.se
bjorkbacken.nu bmwhighperformers.com
blog.f00kclan.de bodybuilding-shop.biz
bloggedporn.com brewdom.ru
bloggen.be bristolhostel.com
bolitgorlo.net brothers-smaller.ru
book-bets.com bscodecs.com
bosman.pluto.ro buigas.00it.com
brillianty.info buron.pw
bristolhotel.com.ua buscarfoto.com
brk-rti.ru buyclomidonlaine.com
buydissertation.net bvlgariwallet.xyz
buypanicdisorderpill.com cactussoft.biz
bvlgariring.xyz callawaygolfstore.online
bwlx.prepedu.cn candlewooddentalcentre.com.au
calstaterealty.us carmuffler.net
canrioloadm.gq castingbank.ru
carstrends2015.com ccjp.eu
cartierreplica.win
cbcseward.com
cdnanalytics.xyz cdnanalytics.xyz
centrdebut.ru cdnnetwok.xyz
cheap-trusted-backlinks.com cgi2.nintendo.co.jp
checktext.org chatroulette.world
chlooe.com cheatcode-lita12.rhcloud.com
christianlouboutinshoes.xyz chocolateslim-en-espana.com
cialgenisrx.com cialisndbrx.com
cineacademy.ru cicaki.net
citetick.com ciekawinki.pl
cmd.kz cindymatches.com
coin-hive.com cjs.com.ru
collegerentals.com clicksor.net
communisave.co.za clips.ua.ac.be
compliance-alexa.top
compliance-barak.xyz compliance-barak.xyz
cookie-law-enforcement-hh.xyz compliance-jessica.top
counter.bloke.com compliance-walt.top
covid-schutzmasken.de conocer-sanabria.com
cracksplanet.com cornerstone-countertops.com
cranly.net covadhosting.biz
creatives.adbetclickin.pink cpajunkies.com
credit-respect.ru cupday.com
cryptoswap.biz darkbooks.org
cyclobenzaprinevstizanidine.blogspot.com daymusam.com
daretodonate.co ddlmega.net
data-ox.com ddospanel.com
data.vtc.pw defenderxtactical.com
ddpills.com delitime.info
dear-diary.ru demenageur.com
decenttools.com demon-tweeks.com
deda-moroza-zakaz.ru dentfidemountpreach.tk
deplim.com deplim.com
detmebel.su detsky-nabytek.info
diegolopezcastan.com dimkino.ru
dieta-personalna.pl
digest-project.ru
dimfour.com
divku.ru
dkvorota.ru dkvorota.ru
doctormakarova.ru dmmspy.com
documentbase.net documentbase.net
domainroam.win dojki-hd.com
dood.live domain-submit.info
dorratex.tn domain2008.com
dosugrostov.site dostavka-v-ukrainu.ru
downloaddy.net downloaddy.net
e-buyeasy.com doxysexy.com
eandsgallery.com dvervmoskvu.ru
eas-seo.com eduserver.net
easync.io elektrischezi.canalblog.com
ebooknovel.club elektronischezigarettekaufen1.beeplog.com
ednorxmed.com elektrozigarettekaufen2.blogse.nl
education-cz.ru elektrozigaretten1.tumblr.com
ekosmetyki.net.pl elektrozigaretten1.webs.com
electro-prom.com elitesportsadvisor.com
eleimgo.pw empathica.com
elektrischezigarette1.blog.pl englishdictionaryfree.com
elektrischezigarette2.onsugar.com erotik0049.com
elektrischezigarettekaufen2.cowblog.fr esfchat.tk
elektronischezi.livejournal.com essay-writing.work
elidelcream.weebly.com estibot.com
elmifarhangi.com ewebarticle.info
enge-fotzen.info exchangeit.gq
enginebay.ru excitacao.com
euronis-free.com exlarseva.webblog.es
evogarage.com eyemagination.com
extremepornos.net ez8motelseaworldsandiego.com
eyessurgery.ru ezofest.sk
ezigaretteshop2.mywapblog.com f5mtrack.com
fanoboi.com faithe.top
filerockstar298.weebly.com fanrto.com
filesdatabase.net faracontrol.ir
filmgo.ru fashion-mk.net
finejewelryshop.ru fashionavenuegame.com
firstsiteguide.com fast-wordpress-start.com
fivedwld.cf favorcosmetics.com
flauntyoursite.com fbdownloader.com
florida-tourism.net fickblock18.com
fodelsedagspresenter.nu fidelityfunding.com
filesclub.net
fillmewithhappiness.com
filmi-onlain.info
filmidivx.com
financepoints.eu
financetip.eu
findclan.org
flowersbazar.com
forex.osobye.ru forex.osobye.ru
forex21.ru forex21.ru
forum.doctissimo.fr fortwosmartcar.pw
fotosfotos.eu fotopop.club
franch.info fr-bearings.ru
free-share-buttons.top frameimage.org
freeuploader.com franecki.net
front.to frbizlist.com
free-floating-buttons.com
freesitetest.com
freetangodownload.com
freshmac.space
fuckbuddybestgilf.info
fugarif.ga fugarif.ga
gamedayassist.com funnypica.com
gamesprite.me fyl.com.ru
gameonasia.com
gamingspark.com
gardene.ru gardene.ru
geckoandfly.com gay-site.store
gemgrab-a.akamaihd.net gaytube.com
get-free-social-traffic.com gayxperience.com
gg.zzyjxs.com gazoblok.net.ua
gk-atlant.info gazporno.com
glass-msk.ru gdzkurokam.ru
glcomputers.ru gembird.com
ghostvisitor.com
glastecfilms.com.my
golden-catalog.pro golden-catalog.pro
gsbs.com.ua goodly.pro
gyffu.com gorabagrata.ru
hackers-crackers.tk gotomontenegro.net
hausfrauensex18.com gq-catalog.gq
havepussy.com greenidesign.co
hawaiielectriclight.com gribokstop.com
hdpixion.com gsasearchengineranker.site
helicalpile.us guitar-master.org
heroz.fr handicapvantoday.com
histock.info hccoder.info
hobild.net hebr.myddns-flir.com
home.myplaycity.com help.tpu.ru
homedo.fabpage.com hetmanship.xyz
house.sieraddns.com hidemyass.com
hoverboard360.es hmmm.cz
howmuchdoestizanidinecost.blogspot.com holidaypics.org
htmlcorner.com homeandhealth.ru
hunterboots.online homeart.space
ideoworld.org howtoclean.club
ihc2015.info husky-shop.cz
iinstalll-fii1leis.jus0wil.pp.ua hyipmanager.in
ilbe.club idegenvezeto.eu
imgarit.pw imagerydatabase.com
incolors.club images-free.net
indetiske.ya.ru imagez.co
infobanks.ru imagui.eu
infodocsportal.com img.wallpaperstock.net
inmate-locator.us imgarcade.com
instabid.tech importchinacoach-teach.com
int.search.mywebsearch.com indiasourcemart.in
infazavr.ru
insomniagamingfestival.com
intelhdgraphicsgtdrive6w.metroblog.com
internetproviderstucson.com internetproviderstucson.com
iradiology.ru iplogger.org
ipostroika.ru
istizanidineanopiate.blogspot.com istizanidineanopiate.blogspot.com
itservicesthatworkforyou.com japan-bearings.ru
izhevsk.zrus.org javcoast.com
jennyfire.ru javxxx18.com
jeremyeaton.co
jerseywholesaleelitestore.com jerseywholesaleelitestore.com
johannesburgsingles.co.za jestr.org
johnnyhaley.top johnnyhaley.top
joy-penguin.com joy-penguin.com
jumptap.com jpcycles.com
jyvopys.com jrcigars.com
kabbalah-red-bracelets.com jwcialislrt.com
kalb.ru kam-dom.ru
kazka.ru kamorel.com
kerei.ru kchaxton.com
khafre.us keywordblocks.com
kinoflux.net kfon.eu
klitimg.pw kinky-fetishes.com
knigonosha.net kinogolos.ru
kochanelli.com kinotorka.ru
komputernaya-pomosh-moscow.ru kinozapas.org
koptims.tiu.ru kiskinhouse.com
kovesszucs.atw.hu kletkimehan.ru
kovrenok.ru koversite.info
kraljeva-sutjeska.com
krasnodar.ru krasnodar.ru
ktotut.net kupiproday.com.ua
labafydjxa.su larchik.net
larutti.ru law-check-two.xyz
le-clos-des-alouettes.com lescinq.com
legionalpha.com letto.by
lesbian.xyz
libertybilisim.com libertybilisim.com
likrot.com liffa.ru
linkwithin.com lignofix.ua
liky.co.ua
limtu.ifmo.ru
lion.bolegapakistan.com
lion.playtap.us
listiseltemournlan.gq listiseltemournlan.gq
locatellicorretor.com.br lizace.com
locationdesaison.com localmatchbook.com
lookingglassemb.com lombardfinder.ru
louboutinshoes.xyz louisvuittonoutletstore.net
lovelycraftyhome.com luchshie-topcasino.ru
lubetube.com luckyday.world
lutherstable.org lumb.co
luxup.ru
mac-shield.com mac-shield.com
magnetic-bracelets.ru macfix.life
madot.onlinewebshop.net
magda-gadalka.ru
mahnwachen-helfen.info
mailemedicinals.com
makis.nu
malls.com malls.com
marcogrup.com managerpak204.weebly.com
marktforschung-stuttgart.com maridan.com.ua
mebel-vstroika.ru marketland.ml
medmajor.ru matb3aa.com
meduza-consult.ru mature.free-websites.com
meendo-free-traffic.ga max-eclat.men
mega-bony2017.pl me-ke.com
members.ghanaweb.com meds-online24.com
menetie.ru megakino.net
metascephe.com mirmedinfo.ru
mezaruk.info mirzonru.net
micasainvest.com mlf.hordo.win
mikozstop.com
mil-stak.com
millionare.com
minyetki.ru
mir-betting.ru
mitrasound.ru
mlvc4zzw.space
mmgq.ru mmgq.ru
mobile.ok.ru mnimmigrantrights.net
mobstarr.com mobot.site
monclerjacketsoutlet.pw moinozhki.com
moncleronline.xyz moje-recenze.cz
moyaterapiya.ru monclerboots.xyz
musicas.baixar-musicas-gratis.com moncleroutletonline.pw
musicpro.monster moscow-clining.ru
mydownlodablefiles.com moyakuhnia.ru
myftpupload.com mttwtrack.com
na15.ru musicspire.online
nacap.ru musicstock.me
namecrumilchlet.tk muz-tracker.net
naturalpharm.com.ua my-bc.ru
news-readers.ru myanyone.net
nextconseil.com mygameplus.ru
nfljerseys.online mylesosibirsk.ru
nhl09.ru mylovelibrabry.com
noscrapleftbehind.co myonigroup.com
notaria-desalas.com naughtyconnect.com
noumeda.com negral.pluto.ro
neodownload.webcam
netanalytics.xyz
nevansk.ru
nfljerseysforsalewholesaler.com
nikkiewart.ru
nippon-bearings.ru
nootrino.com
nosecret.com.ua
nottyu.xyz
november-lax.com
nrv.co.za
nudejapan.net
nuup.info
nyfinance.ml
o-o-6-o-o.ru
oakleyglassesonline.us
obiavo.in obiavo.in
ofanda.com okuos.com
offergroup.info online-hit.info
offtopic.biz ooo-gotovie.ru
onemactrckr.com ordernorxx.com
onlyporno.ru orion-code-access.net
papasdelivery.ru pacificair.com
pcads.ru paintingplanet.ru
pdn-4.com partner-stop.men
phrcialiled.com partners-ship.pro
picsfair.com paydayonlinecom.com
pictures-and-images.com pdfprof.com
pijoto.net penisvergrotendepillennl.ovh
pinwallpaper.top percin.biz.ly
pio.polytopesexempt.com piccdata.com
pix24x7.com picturesfrom.com
pizda.lol picturesify.com
picurams.pw
pintattoos.com
pk-services.ru pk-services.ru
plastweb.ru playfortuna-play.ru
play.leadzupc.com podshipniki-ntn.ru
pochemychka.net poisk-zakona.ru
poem-paying.gq popunder.ru
pontiacsolstice.info porndl.org
popcash.net porngalleries.top
poptool.net pornhubforum.tk
pornoforadult.com porno-chaman.info
potoideas.us pornogig.com
praisong.net powc.r.ca.d.sendibm2.com
princeadvantagesales.com preg.marketingvici.com
privatefx.all4invest.info privacyassistant.net
proekt-gaz.ru privatbank46.ru
professionalwritingservices15.blogspot.ru pro-okis.ru
projefrio.com.br proftests.net
proposal-engine.com progonrumarket.ru
prostitutki-yaroslavlya.org promalp-universal.ru
psa48.ru prostitutki-tyumeni.org
pufip.com pssucai.info
pushdata.sendpulse.com puzo2arbuza.ru
q-moto.ru rapevideosmovies.com
qpypcx.com registrationdomainsite.com
questionmarque.ch remont-comp-pomosh.ru
rank-checker.online remontbiz.ru
rankia.com remorcicomerciale.ro
rankingchart.de research.ifmo.ru
razvratnoe.org ritlweb.com
recordpage-a.akamaihd.net rocis.site
reelheroes.net rospromtest.ru
refererx.com
regionshop.biz
rentehno.ru
resellerclub.com
robotixix.com
rogervivierforsale.com
rossanasaavedra.net
royalcar-ufa.ru royalcar-ufa.ru
roznica.com.ua safe-app.net
rus-pornuha.com sankt-peterburg.nodup.ru
samara.rosfirm.ru scanmarine.info
samoiedo.it scrapinghub.com
santasgift.ml searchmywindow-a.akamaihd.net
satoristudio.net secretscook.ru
scanhub.ru sei80.com
scanner-julianna.top seinterface.com
scanner-walt.top semxiu.com
scenarii-1-sentyabrya.uroki.org.ua seo18.su
searchinterneat-a.akamaihd.net seobility.net
seasonvar.ru
semaltmedia.com
seo-smm.kz
seocheckupx.com seocheckupx.com
serpstat.com seofied.com
sevendays.com.ua service.adtech.fr
sevenstars7.com serviporno.com
sex-dating.co sex-watch.com
sex-sex-sex5.com sexkrasivo.net
sexreliz.com sextracker.de
sexvporno.ru shivafurnishings.com
shemale-sex.net shoppingmiracles.co.uk
shop-electron.ru shtaketniki.ru
shopsellcardsdumps.com sibecoprom.ru
sisiynas.ru sibtest.ru
siteripz.net sigmund-freud.co.uk
slowmacfaster.trade silvercash.com
smokewithrabbits.com silvermature.net
snegozaderzhatel.ru simplepooltips.com
snip.to sinestesia.host.sk
snow.nvr163.com skutecznetabletkinaporostwlosow.pl
soblaznu.net skytraf.xyz
socas.pluto.ro snip.tw
socialmediasuggest.com social-search.me
soft1.ru soft1.ru
softomix.net softxaker.ru
soietvousmaime.fr sohoindia.net
spanking.to solnplast.ru
specialfinanceoffers.com spb.afora.ru
spitfiremusic.com sport7777.net
spy-sts.com srdrvp.com
srecorder.com stanthonyscatholicchurch.org
starwars.wikia.com stopnarco.ru
staticfs.host sukarame.net
stop-zavisimost.com superlist.biz
stroyplus.ru svarkagid.com
sugarkun.com sygraem.com
supermesta.ru tacbibirfa.tk
superstats.com takethatad.com
surffoundation.nl taqywu51.soup.io
swinginwithme.ru tecspb.ru
symbaloo.com teenfuck.tv
syvertsen-da.no terraclicks.com
taboola.com tetracsaudi.com
telefonsexkostenlos.tk thecoral.com.br
telegraf.by thetattoohut.com
teletype.in timer4web.com
tengohydar.tk titan-cloud.life
teslathemes.com tizanidine4mgstreetvalue.blogspot.com
testingads.pro tizanidinehcl2mg.blogspot.com
theallgirlarcade.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
thebestphotos.eu tizanidinepain.blogspot.com
thetardistimes.ovh tizanidinevscyclobenzaprine.blogspot.com
thexart.club
timetorelax.biz
tizanidineoral.blogspot.com
tmearegion26.com tmearegion26.com
tongkatmadura.info tnaionline.org
top10-online-games.com tobeyouday.win
top10-way.com tomatis.gospartner.com
topclickguru.com topbestgames.com
topshef.ru topcar-krasnodar.ru
torrent-newgames.com track.deriv.com
tourcroatia.co.uk trafficfactory.biz
tourismvictoria.com trafficnetzwerk.de
trackmedia101.com trion.od.ua
trafficshaper.com trubywriting.com
trk-4.net trustaffs.com
trumpetedextremes.com
tt-ipd.info
tubeoffline.com
tuberkulezanet.ru tuberkulezanet.ru
tupper-shop.ru turbo-suslik.org
turizm.bz
twiclub.in twiclub.in
tx41tclega.ru tytoona.com
txxx.com u-cheats.ru
ultramart.biz ultimateclassicrock.com
uptimebot.net unmaroll.ya.ru
valsalud.com urlopener.blogspot.com.au
varikozdok.ru
vegan-foods.us vegan-foods.us
vepad.com vegascosmetics.ru
vezuviy.su videosbox.ru
vgoloveboli.net vip.51.la
via-energy-order.com vip2ch.com
via-gra.webstarts.com vkonche.com
videooko.weebly.com
viteonlusarezzo.it
viveresaniesnelli.it
vladimir.xrus.org
voluumtracker1.com voluumtracker1.com
vote-up.ru vriel.batcave.net
vremya.eu vroze.com
vuclip.com vykup-avto-krasnodar.ru
vut.com.ru vzube.com
vzglyadriv.kg w3javascript.com
warningwar.ru warningzscaler.heraeus.com
watracker.net watchinf.com
webextract.profound.net wcb.su
webinstantservice.com web-analytics.date
weboptimizes.com website-speed-checker.site
welck.octopis.com website-speed-up.site
wheelchairliftsarea.com wechatdownload10.com
whipme.yopoint.in wedding-salon.net
wholesalejerseychinaoutlet.com westvilletowingservices.co.za
wildworld.site wholesalecheapjerseysfree.com
win-spy.com wholesalejerseychinashop.com
wineitudes.wordpress.com wordpresscore.com
womensterritory.ru worldhistory.biz
woweb.com.ua wosik-dach.service-for-web.de
wpthemedetector.co.uk wpthemedetector.co.uk
wrona.it wrz0iuebwhp5fg.freeddns.com
wtsindia.in www.ehscloud.cn
wurr.voila.net wyniki-lista.pl
x-musics.com x-lime.net
x-true.info xaylapdiendanang.com
xblog.in xev.ru
xlolitka.com xfluro.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn----9sbebi2bvzr7h.xn--p1ai
xn--80aaagvmjabrs1aoc9luc.xn--p1ai xn----ctbigni3aj4h.xn--p1ai
xn--80aeahghtf8ac5i.xn--p1ai xn----dtbndd4ae7eub.top
xn--e1aggki3c.xn--80adxhks xn----itbeirbjbi7bc6bh2d.xn--p1ai
xxxrus.org yellowstonesafaritours.com
xxxtube69.com
yadro.ru
yogamatsexpert.com yogamatsexpert.com
youdesigner.kz yopoint.in
youporn-forum.ga youjizz.com
yourdesires.ru yourmovies.pl
zaapplesales.blogspot.com youtubologia.it
zaloadi.ru zajm-zalog-krasnodar.ru
zatjmuzu.info zeleznobeton.ru
zeikopay.com zeroredirect11.com
znakomstva-piter78.ru zhorapankratov7.blogspot.com
znaniyapolza.ru zigarettenonlinekaufen2.bloog.pl
zog.link znakom.sibtest.ru
zone-kev717.info zoogdisany.com
zoominfo.com zootoplist.com
zootravel.com
zskdla.site

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 000free.us
108shot.com 01apple.com
1xbet4you.com 0lovespells0.blogspot.com
2000k.ru 3dracergames.com
a.pr-cy.ru 3th.co.in
acmebtn.ml 4rent.ru
actices.com 4ureyesonly.com
addtoadd.men aa81bf391151884adfa3dd677e41f94be1.com
adman.gr addbin.men
adservme.com adnotbad.com
advokat-grodno.by affiliate-fr.com
anaksma.info agysya.ru
animalrank.com albamargroup.com
apishops.ru alldezservice.kz
apparel-offer.com android-style.com
arcadeplayhouse.com apimossnetinfo-a.akamaihd.net
architecturebest.com app5.letmacworkfaster.world
arendas.net ayeartoforget.com
arewater.com balitouroffice.com
arkartex.ru beatifullandscape.co
aruplighting.com betonka.pro
ascat.porn betwinservice.com
asrv-a.akamoihd.net bio-optomarket.ru
azbukafree.com bioca.org
bablonow.ru bizfly.info
backgroundpictures.net bjorkbacken.nu
battlecarnival.su blackbabesporn.com
beauty-clinic.ru brandbucket.com
bedcapdealers.com bugshoot.cn
best-seo-solution.com bus-offer.com
bestbuy.ca buydissertation.net
bestimagecoollibrary.com candy-glam-hp.com
betshuckclean.com cardsharp1.ru
bimatoprost-careprost.com chatroulette.online
bitcoinpile.com cheapmoncler.win
blogs.rediff.com chinacheapelitejerseys.com
boo-arts.com cjs.com.ru
break-the-chains.com clayvasedesigns.tk
brianhenry.co click2pawn.com
bristolhotel.com clicksor.com
bukleteg.xyz com-onlinesupport.site
businescoop.men cometorussia.net
business-suggested.tk companies-catalog.com
buy-forum.ru cracksplanet.com
buyessay3.blogspot.ru craftburg.ru
cakesplus.com.au cranly.net
canrole.com customboxes4less.com
capture-room.com dailystrength.org
carfax.com.ua dealitright.click
ccjp.eu dekoration.us
cenoval.ru delfin-aqua.com.ua
cheapcigarettesc.info dellalimov.com
cheapmarmot.online directrev.com
chuckguilford.com dojki365.online
cinemaflix.website domain2008.com
clickso.com doxyporno.com
cookielawblog.wordpress.com drugspowerstore.com
coolgramgoods.com dvd-famille.com
coolingoods.com dvervmoskvu.ru
cowlmash.com dyshagi.ru
customsua.com.ua elektroniksigaraankara.info
daymusam.com elkacentr.ru
digital-sale.su enews.tech
dildofotzen.net epngo.bz
djihispano.com eralph.tk
docsportal.net esnm.ru
doctormakarova.ru etenininrade.ga
domination.ml exchangeit.gq
dostavimvdom.ru expertnaya-ocenka.ru
downloadmefiranaratb1972.xpg.com.br exportshop.us
e-commerce-seo1.com extads.net
ebooknovel.club extlabs.io
eksprognoz.ru f012.de
elitedollars.com face.hostingx.eu
eloxal.ru fashionavenuegame.com
enge-fotzen.info felizporno.com
ero-advertising.com ferieboligkbh.dk
expolicenciaslatam.co formulaf1results.blogspot.com
falopicm.pw free-share-buttons.blogspot.com
fapgon.com free-social-buttons.xyz
fast-wordpress-start.com ftns.ru
filkhbr.com
filmi-onlain.info
foto-basa.com
fungirlsgames.net
fusoradio.info
fxtips.ru
gambarkataku.co
gamevalue7.weebly.com gamevalue7.weebly.com
gateway.zscalerone.net gaz-voshod.ru
gaytube.com goodnightjournal.com
geileweiber.tk gtopstats.com
glall.ru hatdc.org
global-ics.co.za hcate.com
gombita.info healbio.ru
good-mummy.ru hermesbelts.xyz
goodhousekeeping.com hmywwogw.bloger.index.hr
gsasearchengineranker.site hotchatdate.com
gungamesz.com houseofgaga.ru
hawaiisurf.com humanelydrew.com
highstairs-a.akamaihd.net iboss.com
holidaypics.org igrovyeavtomaty777.ru
hoste.octopis.com ilbe.club
hotshoppymac.com immigrational.info
humanorightswatch.org importchinacoach-teach.com
humbmosquina.tk impresagaia.it
hydropump.su infazavr.ru
igadgetsworld.com infodocsportal.com
imadedinner.net intim-uslugi.info
imfamous.info investsuccess.org
imgata.com ip-guide.com
interferencer.ru isotoner.com
internetproviderstucson.com itch.io
iqs.biz.ua
ivoiretechnocom.ci
javlibrary.cc javlibrary.cc
jerseychinabizwholesale.us justprofit.xyz
jetsli.de karpun-iris.ru
jo24news.com kerwinandcariza.com
kinobaks.com kino-rf.ru
kinobest.pl knogg.net
kol-energo.ru konteiner24.com
kosmetyki.tm.pl koral.se
landscaping.center kountrylife.com
leboard.ru laapp.com
linkdebrideur.xyz laparfumotec.com
loftdigital.eu laptop-4-less.com
lol-smurfs.com leadn.pl
louboutinreplica.xyz lecbter-relationships.ga
lulea-auktionsverk.se lightinghomes.net
likesdesign.com
listiseltemournlan.gq
lordzfilmz.me
lrac.info
lunchrock.co lunchrock.co
m5home.ru
magda-gadalka.ru magda-gadalka.ru
mastershef.club magnetpress.sk
mebel-alait.ru mainhp.net
meendo-free-traffic.ga mamasuper.prom.ua
microstatic.pl mapquestz.us
mixtapetorrent.com marsgatan.com
mmofreegames.online mediawhirl.net
mnogolok.info medicineseasybuy.com
montredemarque.nl mhi-systems.ru
mostantikor.ru mhtr.be
mrpornogratis.xxx minneapoliscopiers.com
msk-diplomat.com mlf.hordo.win
multgo.ru moncleroutletonline.pw
musicktab.com moneyteam24.com
myseoconsultant.com monsterdivx.com
noumeda.com morf.snn.gr
novgorod.xrus.org mydoctorok.ru
myprintscreen.com
nalogovyy-kodeks.ru
nelc.edu.eg
nonews.co
normalegal.ru
npoet.ru
nuker.com
nyfinance.ml nyfinance.ml
o-o-11-o-o.com of-ireland.info
onetravelguides.com oivcvx.website
onko-24.com ok.ru
onlineporno.site okeinfo.online
onlythegames.com onlythegames.com
oren-cats.ru ooo-gotovie.ru
orenburg-gsm.ru openfrost.net
ororodnik.goodbb.ru ordernorxx.com
outshop.ru orion-code-access.net
pacificair.com paceform.com
pagesense.com pattersonsweb.com
parlament.biz photochki.com
patterntrader-en.com pl.aasoldes.fr
penisvergrotendepillennl.ovh popads.net
picphotos.net pornohd1080.online
picsforkeywordsuggestion.com pornosok.ru
pizdeishn.com praisong.net
poisk-zakona.ru
pornogad.com
pornoklad.net
pornoxxx.com.mx
poshiv-chehol.ru
prodess.ru prodess.ru
promgirldresses.xyz qpypcx.com
prostitutki-kharkova.org quelle.ru
ranksignals.com rating-bestcasino.com
rapidsites.pro ric.info
registratciya-v-moskve.ru rockingclicks.com
remont-komputerov-notebook.ru rollercoin.com
rossmark.ru roma-kukareku.livejournal.com
sa-rewards.co.za ru-dety.ru
sbf441.com s-iwantyou.com
scanner-elena.top saratov.xrus.org
sdi-pme.com savefrom.com
sdrescher.net scanner-ivan.top
serw.clicksor.com searchlock.com
sexkontakteao.info seeingmeerkat.com
sexobzor.info seksvideoonlain.com
sexspornotub.com sexuria.net
shisha-swag.de shell-pmr.ru
shtaketniki.ru shopcheermakeup.info
siha.de si-unique.com
sisi-go.ru sibtest.ru
sky-mine.ru sigmund-freud.co.uk
skytraf.xyz signoredom.com
softtor.com snegozaderzhatel.ru
spravka130.ru softomix.net
stackthatbucks.com softonicads.com
steelmaster.lv soietvousmaime.fr
stjamesschool.info spacash.com
stpicks.com speedup-my.site
suchenindeutschland.com stats-public.grammarly.io
steebook.com
studentguide.ru
stuffpride.com
sumo.com
supervesti.ru
support.nopeas.sk
suture.co
svtrd.com svtrd.com
tacbibirfa.tk terrafootwear.us
tagil.zrus.org thebluffs.com
taximytishi.ru thejournal.ru
teletype.in tizanidineduringpregnancy.blogspot.com
tgtclick.com tizanidinesideeffectsweightloss.blogspot.com
tizanidinedrugtest.blogspot.com topbestgames.com
tizanidinehydrochloride2mg.blogspot.com tour-line.net
topclickguru.com trion.od.ua
topflownews.com trubywriting.com
torture.ml tvnewsclips.info
trackzapper.com
trafficjunky.net
trafficmonetize.org
tvteleport.ru tvteleport.ru
twu.com.ua twodollarshows.com
ugguk.online uac.net.au
ukrobstep.com
ukrtvir.com.ua
unacittaconte.org unacittaconte.org
uptimebot.net unmaroll.ya.ru
uptimechecker.com venerologiya.com
v24s.net vesnatehno.com
varikozdok.ru videositename.com
vinylvault.co.uk vipps.com.my
vip-file.com vitanail.ru
vipcallsgirls.com vkontaktemusic.ru
voloo.ru vut.com.ru
vseigru.one
wallpapers-best.com
wallpapersdesk.info wallpapersdesk.info
winwotgold.pl watch-movies.ru
wma-x.com watchmygf.net
works.if.ua waterefficiency.co
worldhistory.biz waycash.net
wygraj-teraz.com web-analytics.date
xgftnlrt.bloger.index.hr webadvance.club
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai webix.me
xn--80ak6aa92e.com webnode.me
xn--80aodinpgi.xn--p1ai websitevaluebot.com
xxxkaz.org womensterritory.ru
youdao.com xblognetwork.com
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--d1aifoe0a9a.top
xxxasianporn.net
yellocloud.be
yottos.com
youdesigner.kz
youresponsive.com youresponsive.com
zakazfutbolki.com yourserverisdown.com
zarabiaj-dzis.pl zastenchivosti.net
zarabotat-na-sajte.ru
zdesoboi.com
znakomstvaonlain.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.51 0; 10.1.0.17 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3503 ### Version: V4.2022.12.3504
### Updated: Fri Dec 2 10:03:12 UTC 2022 ### Updated: Fri Dec 2 22:01:09 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3503 ### Version: V4.2022.12.3504
### Updated: Fri Dec 2 10:03:12 UTC 2022 ### Updated: Fri Dec 2 22:01:09 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##