vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.05.3083 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-05-03 22:07:17 +00:00
parent 52ff35bbb3
commit 8a87bece5c
46 changed files with 94543 additions and 94543 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3082 #### Version: V4.2022.05.3083
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.05.3082 #### Version: V4.2022.05.3083
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.05.3082 #### Version: V4.2022.05.3083
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11682
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.05.3082 #### Version: V4.2022.05.3083
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11686
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11686
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11686
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11686
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.1.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.1.219 0;
127.0.0.1 0; 127.0.0.1 0;

11686
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.1.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.1.219 0;
127.0.0.1 0; 127.0.0.1 0;

11686
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.1.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.1.219 0;
127.0.0.1 0; 127.0.0.1 0;

11686
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3082 3083

11686
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11686
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.237.154.221 0; 52.152.141.103 0;

11686
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.1.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.1.219 0;
127.0.0.1 0; 127.0.0.1 0;

11686
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.1.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.1.219 0;
127.0.0.1 0; 127.0.0.1 0;

11686
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.1.219 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.1.219 0;
127.0.0.1 0; 127.0.0.1 0;

11686
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.237.154.221 0; 52.152.141.103 0;

11686
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.237.154.221 0; 52.152.141.103 0;

11686
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 1; 10.1.1.219 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AfD-Verbotsverfahren G-i-g-a-b-o-t
AhrefsBot GrapeFX
Alligator GridBot
CheeseBot Masscan
Download Wonder Piepmatz
EyeNetIE SurveyBot
Leap WWW-Mechanize
NICErsPRO WebBandit
SeoSiteCheckup Zeus
SpaceBison cah.io.community

308
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
80legs 80legs
ADmantX ADmantX
ALittle Client ASPSeek
Aboundex Abonti
Aboundexbot Acunetix
AhrefsBot AhrefsBot
Aipbot
Alexibot Alexibot
AllSubmitter AllSubmitter
Alligator AlphaBot
Anarchy99 Anarchie
Aspiegel Anthill
Attach Apexoo
Asterias
AwarioSmartBot AwarioSmartBot
BBBike
BDCbot BDCbot
BDFetch BDFetch
BackDoorBot BackDoorBot
BackStreet BackStreet
Backlink-Ceck Battleztar Bazinga
BacklinkCrawler Bitacle
Bandit BlackWidow
BetaBot Blow
Bolt BlowFish
Brandprotect Boardreader
BotALot
Buddy Buddy
BuiltBotTough
Bullseye Bullseye
CATExplorador BunnySlippers
BuzzSumo
CCBot CCBot
CODE87
CensysInspect CensysInspect
CheTeam
CheeseBot CheeseBot
CherryPicker Citoid
ChinaClaw
Cliqzbot Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Cogentbot Collector
Copier
CopyRightCheck CopyRightCheck
Copyscape
Craftbot Craftbot
CrunchBot Crawling at Home Project
CrazyWebCrawler
Curious Curious
CyotekWebCopy
DIIbot
DSearch DSearch
DTS Agent
Demon Demon
Deusu
Digincore Digincore
DigitalPebble
Dirbuster Dirbuster
DnBCrawler-Analytics Dispatch
DomCopBot DomCopBot
DomainCrawler DomainStatsBot
DomainSigmaCrawler Dotbot
Dragonfly
Drip Drip
EMail Siphon
EMail Wolf
EasyDL EasyDL
Ebingbong Ebingbong
Ecxi
EirGrabber
EroCrawler
Exabot
Express WebPictures Express WebPictures
Extractor ExtractorPro
Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FemtosearchBot FHscan
Fimap FlashGet
Flunky G-i-g-a-b-o-t
Freeuploader
FyberSpider
GetWeb GetWeb
Grabber Go!Zilla
Grafula GrabNet
GrapeshotCrawler GrapeshotCrawler
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack
Haansoft
HaosouSpider
Havij
Hloader
HybridBot HybridBot
IDBTE4M IRLbot
IDBot IlseBot
IndeedBot Image Fetch
Image Sucker
Indy Library
InfoNaviRobot
Intelliseek
InterGET
Iria
Iskanie
JOC Web Spider JOC Web Spider
JikeSpider JamesBOT
JustView JennyBot
Jyxobot Jorgee
Kenjin Spider
Kinza Kinza
Kozmosbot Larbin
Leap
LeechFTP LeechFTP
LexiBot LexiBot
LibWeb LibWeb
Libwhisker LieBaoFast
LinkWalker LinkScan
LinkpadBot Linkbot
LinksManager LinqiaScrapeBot
Lipperhey Lipperhey
Ltx71 Ltx71
MFC_Tear_Sample
MQQBrowser
MTRobot MTRobot
Mag-Net Majestic SEO
Majestic-SEO Majestic-SEO
Mass Downloader MarkWatch
Masscan Masscan
Mata Hari Mata Hari
MeanPath Bot MauiBot
Morfeus Fucking Scanner MegaIndex.ru
Mozlila Metauri
Microsoft Data Access
Mister PiX
Moblie Safari
Msrabot Msrabot
Musobot NPbot
Name Intelligence
Nameprotect Nameprotect
NearSite
Needle Needle
NetAnts
NetMechanic
NetZIP NetZIP
Netcraft
Nettrack Nettrack
Netvibes Netvibes
Niki-bot Nibbler
Nmap
Not Not
Nuclei Nuclei
Nutch
Offline Explorer Offline Explorer
Offline Navigator OpenVAS
OnCrawl
Openfind Openfind
Openvas OutfoxBot
OrangeBot
OutclicksBot
PECL::HTTP
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer PageAnalyzer
PageGrabber Panscient
PageScorer PeoplePal
Petalbot
Picscout
PictureFinder PictureFinder
Pimonster
Pixray Pixray
PleaseCrawl
ProPowerBot
ProWebWalker
Probethenet
Psbot
Pu_iN Pu_iN
Pump
PxBroker PxBroker
PyCurl PyCurl
RankActive Quick-Crawler
RankActiveLinkBot RankActiveLinkBot
RankurBot RankFlex
Re-re RankingBot
Recorder RankingBot2
RedesScrapy RebelMouse
RepoMonkey Rogerbot
SBIder
SEOkicks SEOkicks
SEOkicks-Robot SEOprofiler
SEOlyticsCrawler SISTRIX
SMTBot SMTBot
SalesIntelligent ScanAlert
ScoutJet
Scrapy Scrapy
ScrepyBot Screaming
ScreenerBot
Searchestate
SearchmetricsBot SearchmetricsBot
Seekport SeoSiteCheckup
Semrush Shodan
SemrushBot
Seomoz
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteSucker
SmartDownload SlySearch
Snake Snake
Snapbot
Snoopy Snoopy
Sociscraper Sogou web spider
Sosospider Sosospider
Sottopop SpankBot
Spanner
Spbot
SputnikBot SputnikBot
Sqlmap Steeler
Sqlworm Swiftbot
Sucker
Sucuri
SuperBot
SuperHTTP
SurveyBot
Suzuran
Szukacz Szukacz
T0PHackTeam T8Abot
Teleport Teleport
TeleportPro Trendictionbot
Telesoft Turingos
Titan Turnitin
True_Robot
TwengaBot
Twice
Typhoeus Typhoeus
URLy.Warning URLy Warning
UnisterBot VelenPublicWebCrawler
Upflow VeriCiteCrawler
VB Project
VidibleScraper
Virusdie
VoidEYE VoidEYE
Voil Voil
Voltron
WASALive-Bot WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV WWW-Mechanize
WISENutbot Wallpapers/3.0
WPScan WallpapersHD
WWW-Collector-E Web Collage
WebBandit Web Enhancer
Web Pix
Web Sauger
WebCollage
WebCopier WebCopier
WebFetch WebEnhancer
WebFuck WebFuck
WebPix
WebSauger WebSauger
WebStripper WebZIP
WebWhacker
Webalta Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Webster Website Quester
Whack Who.is Bot
Whacker WinHTTrack
Widow Woobot
WiseGuys Robot Wotbox
YoudaoBot Wprecon
Zauba Xenu
Zermelo
Zitebot Zitebot
ZoominfoBot
ZyBorg
arquivo.pt arquivo.pt
backlink-check backlink-check
cah.io.community check1.exe
clark-crawler
cognitiveseo cognitiveseo
crawl.sogou.com demandbase-bot
crawler4j domainsproject.org
evc-batch
instabid instabid
isitwp.com internetVista monitor
iubenda-radar linkdexbot
lwp-request
lwp-trivial
magpie-crawler magpie-crawler
meanpathbot meanpathbot
mediawords mediawords
muhstik-scan
plumanalytics plumanalytics
seobility polaris version
seoscanners probe-image-size
seostar
serpstatbot serpstatbot
sexsearcher
sitechecker.pro
sogouspider sogouspider
tAkeOut spyfu
sysscan
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
webpros.com xpymep1.exe
x09Mozilla
zauba.io
zgrab

318
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,249 +1,249 @@
01h4x.com
360Spider
404checker 404checker
ADmantX 80legs
Abonti ALittle Client
Aboundex
Aboundexbot Aboundexbot
Acunetix AfD-Verbotsverfahren
AhrefsBot
AiHitBot
Anarchie Anarchie
Aspiegel Anarchy
Asterias Anthill
Apexoo
Atomseobot
Attach
AwarioRssBot
BBBike
BDCbot
BDFetch BDFetch
BLEXBot
BackDoorBot BackDoorBot
BacklinkCrawler Backlink-Ceck
Barkrowler Bandit
BatchFTP Battleztar Bazinga
BetaBot BlowFish
Bitacle
Boardreader Boardreader
Brandwatch Buck
Buddy Buddy
BuiltBotTough BuiltBotTough
CATExplorador BunnySlippers
Calculon BuzzSumo
CensysInspect CCBot
CheTeam CODE87
Cogentbot Cegbfeieh
Collector CheeseBot
CopyRightCheck Chlooe
Crawling at Home Project Citoid
CrazyWebCrawler Copier
Copyscape
Cosmos
CrunchBot CrunchBot
CyotekWebCopy Curious
DSearch Custo
Digincore DataCha0s
DatabaseDriverMysqli
Demon
DigitalPebble DigitalPebble
Disco
Dispatch
DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot DomainSigmaCrawler
DomainAppender EMail Wolf
DomainCrawler Ecxi
Dragonfly Evil
EasyDL Extractor
Ebingbong EyeNetIE
FDM FHscan
FemtosearchBot FemtosearchBot
Firefox/7.0 Fimap
FlashGet Flunky
Foobot
Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider
G-i-g-a-b-o-t
Genieo Genieo
GetWeb GetWeb
Gigabot Gigabot
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
Gotit
GrabNet GrabNet
Grabber
GrapeFX GrapeFX
GrapeshotCrawler
GridBot
HMView HMView
Havij Haansoft
Heritrix HonoluluBot
Hloader Humanlinks
HybridBot IDBTE4M
IDBot
IlseBot IlseBot
InfoTekies InfoTekies
Iria Intelliseek
IstellaBot InterGET
Jbrofuzz Internet Ninja
InternetSeer
Iskanie
JennyBot
JetCar JetCar
JikeSpider
Joomla Joomla
Jorgee
JustView
Jyxobot Jyxobot
Kenjin Spider Kinza
Keybot Translation-Search-Machine
Kozmosbot Kozmosbot
Lanshanbot Lanshanbot
Leap
LexiBot LexiBot
Lftp
LieBaoFast LieBaoFast
Likse
LinkWalker LinkWalker
LinqiaScrapeBot LinkextractorPro
Lipperhey Spider LinkpadBot
Lmspider LinksManager
MFC_Tear_Sample LinqiaMetadataDownloaderBot
MIDown tool MIIxpc
MSIECrawler MJ12bot
MTRobot MQQBrowser
Mail.RU_Bot Mag-Net
Majestic SEO Majestic12
Majestic-SEO MarkWatch
Mass Downloader Mass Downloader
MauiBot Masscan
Mb2345Browser MeanPath Bot
Metauri MegaIndex.ru
MicroMessenger MicroMessenger
Mojeek
Mojolicious Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Needle
NPbot NetAnts
Name Intelligence
Nameprotect
Navroad
Net Vampire
NetLyzer NetLyzer
NetMechanic NetMechanic
NetSpider
Netcraft
Netvibes Netvibes
Nibbler NextGenSearchBot
Niki-bot
NimbleCrawler NimbleCrawler
Not Nimbostratus
Nutch Offline Navigator
OpenLinkProfiler OnCrawl
Openfind Openfind
Openvas PECL::HTTP
OrangeBot
OutclicksBot
OutfoxBot
PHPCrawl PHPCrawl
PageAnalyzer POE-Component-Client-HTTP
PageGrabber Page Analyzer
PageThing.com PageThing.com
Pandalytics Papa Foto
Pavuk
PeoplePal
Petalbot
PictureFinder
Piepmatz
Pockey Pockey
ProPowerBot
Psbot Psbot
PxBroker Pu_iN
Quick-Crawler Pump
PyCurl
RSSingBot RSSingBot
RankFlex RankActive
RankActiveLinkBot
RankingBot
RankingBot2
RankurBot RankurBot
RealDownload
Reaper
RebelMouse
Recorder Recorder
Ripper RedesScrapy
SBIder SEOkicks-Robot
SEOkicks Scrapy
SISTRIX Semrush
Scanbot Seomoz
ScoutJet SiteCheckerBotCrawler
Screaming
ScrepyBot
Searchestate
Seekport
SemanticJuice
SeobilityBot
Shodan
Siphon
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker SiteSucker
Siteimprove
Sitevigil Sitevigil
SlySearch Snake
Snapbot
SocialRankIOBot
Sociscraper Sociscraper
Sogou web spider Sosospider
SpankBot
Spanner Spanner
Spinn3r Spinn3r
SputnikBot SputnikBot
Sqlmap
Sqlworm
Sqworm Sqworm
Steeler
Sucker Sucker
Sucuri
SuperBot SuperBot
SuperHTTP
Surfbot Surfbot
SurveyBot SurveyBot
Swiftbot
Szukacz
TeleportPro
Telesphorep
The Intraformant The Intraformant
Thumbor TheNomad
Titan Toweyabot
Toata Tracemyfile
True_Robot Trendiction
Twice Turingos
Typhoeus Turnitin
URLy Warning Vagabondo
Upflow
VCI
Vacuum
VeriCiteCrawler VeriCiteCrawler
VidibleScraper VoidEYE
Virusdie WPScan
Voltron WWWOFFLE
WBSearchBot Wallpapers/3.0
WWW-Collector-E Web Auto
WWW-Mechanize Web Collage
WWW::Mechanize Web Pix
Wallpapers Web Sucker
WallpapersHD
Web Fetch
Web Fuck
Web Sauger
WebAuto
WebCollage
WebEnhancer WebEnhancer
WebFuck WebFuck
WebLeacher
WebSauger WebSauger
WebStripper WebStripper
WebSucker WebSucker
WebWhacker
WebZIP WebZIP
Webalta Webalta
Webshag Website Quester
Webster Webster
Whacker
Whatweb Whatweb
Woobot Widow
Xenu WiseGuys Robot
Wprecon
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zauba
Zermelo Zermelo
Zeus Zeus
ZoomBot Zitebot
coccocbot ZumBot
adscanner
autoemailspider
cah.io.community
check1.exe
cognitiveseo cognitiveseo
crawl.sogou.com
dataforseo.com dataforseo.com
demandbase-bot evc-batch
domainsproject.org
eCatch
gopher gopher
heritrix
ips-agent ips-agent
isitwp.com isitwp.com
iubenda-radar iubenda-radar
magpie-crawler linkdexbot
meanpathbot lwp-trivial
mediawords mediawords
muhstik-scan
netEstate NE Crawler
page scorer
plumanalytics
polaris version polaris version
probe-image-size probe-image-size
ripz
satoristudio.net
scan.lol scan.lol
seocompany.store seocompany.store
seoscanners
serpstatbot
sexsearcher sexsearcher
sogouspider spyfu
sysscan tAkeOut
ubermetrics-technologies.com trendiction.com
trendiction.de
voyagerx.com voyagerx.com
webprosbot webprosbot
x22Mozilla x22Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
fotosfotos.eu 1001watch.com.ua
moz.com birzha-truda.eu
nastroyke.net delayreferat.ru
nyfinance.ml gcup.ru
russkie-gorki.ru istizanidineanarcoticdrug.blogspot.com
se-welding.ru mzdish.site
socialsignals24.com repeatlogo.co.uk
unlimitdocs.net seo-smm.kz
vkmusics.ru youandcredit.ru
wovis.site zolotoy-lis.ru

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03p.info 01apple.com
08800.top
12bet.com 12bet.com
2345.com 1flag.co.za
2drittel.de 1kinobig.ru
2rich4bitches.com 1millionusd.xyz
5-steps-to-start-business.com 2020iscoming.info
777-club.ru 24-ak.ru
7a2rnuey1tw9ar.ru 7makemoneyonline.com
7search.com 7xc4n.com
8jn.dating-app.ru 991mostfm.co.id
91abcw.com a.frcls.fr
98oi.ru aa9bd78f328a6a41279d0fad0a88df1901.com
a14download.com abcdeg.xyz
a96527gi.beget.tech adhome.biz
aadroid.net adsland.men
absolute-s.ru adultfriendfinder.com
academia-nsk.org adviceforum.info
adrunnr.com
adultfullhd.com
aero2.ru aero2.ru
afora.ru afslankpillen2017nl.eu
airlandsea.info agadelha.com.br
akama.com ahrntal.verymes.xyz
aksonural.ru aktobe.xkaz.org
alessandraleone.com akusherok.ru
akvamaster.dp.ua
albumsuper.info
allblogroll.com allblogroll.com
allboard.xobor.de alloysteel.ru
allmarketsnewdayli.gdn
allproblog.com allproblog.com
allwomen.info allstatesugarbowl.org
amigobulls.com alphahoverboards.com
amospalla.es amazingninja.com
andpolice.com ameblo.top
animal-drawings.com amehdaily.com
animebox.com.ua analytics-ads.xyz
anonymousfox.co anonymousfox.co
apiallgeniusinfo-a.akamaihd.net api.stathat.com
arclk.net apiappenableinfo-a.akamaihd.net
asacopaco.tk apibetweenlinesn-a.akamaihd.net
asiavirtualsolutions.com apidiamondatanet-a.akamaihd.net
asrv-a.akamoihd.net apimountainbikei-a.akamaihd.net
atdedinotuho.tk apispringsmartne-a.akamaihd.net
appleid-verification.com
asdfz.pro
asrvvv-a.akamaihd.net
atleticpharm.org
atofilms.com
audiobangout.com audiobangout.com
avek.ru auto.rusvile.lt
aviav.ru autolombard-krasnodar.ru
avtolombard-krasnodar.com autorn.ru
azbukafree.com avitocars.ru
backiee.com avkzarabotok.info
bad-stars.net avramstroy.ru
baladur.ru avtoarenda.by
bang-hotties.com ayakino.net
banksrf.ru azte.ch
basedecor.ru bangla-chat-uk-paralud.ga
beachpics.com bankofthewext.com
beauby.ru bannerbank.ru
barnfurnituremart.com
bbtec.net
beachtoday.ru
beatifullandscape.co
bestimagecoollibrary.com
bestmobilityscooterstoday.com
bestwaystogetpaid.us bestwaystogetpaid.us
betonka.pro bestwebsitesawards.com
bio-market.kz bestwrinklecreamnow.com
bitcoins-live.ru betteroffers.review
beyan.host.sk
bget.ru
bigcareer.info
bigcities.org
bimmerpost.com
bin-brokers.com
bitcoinpile.com
bjanshee.ru bjanshee.ru
blobar.org blogig.org
bluerobot.info bmsco.co
boanonihaca.tk bnt-team.com
bolitgorlo.net
bombla.org bombla.org
braintobe.win bongiornos.info
bumskontakte.org brainxs.us
buyk.host.sk brandedleadgeneration.com
buytizanidineonlinenoprescription.blogspot.com brateg.xyz
bycontext.com brothers-smaller.ru
bytimedance.ru bryansk.zrus.org
bzero1jewelry.net buckspar.gq
cah.io.community buqyxa.rincian.info
calc-for-credit.ru burger-tycoon.com
cam-kontakte.org c-gainsbourg.com
camsex.xxx-cam.webcam calstaterealty.us
canacopegdl.com capture-room.com
candy-glam-hp.com cardiosport.com.ua
capacitacionyaprendizaje.com cardsharp1.ru
carrauterie.be cartierbraceletsreplica.pw
cartierbracelet.xyz cartierlovebracelet.xyz
cartierreplica.top ccbill.com
cellfun.mobi celebsopics.com
cenokos.ru chastnoeporno.com
chatroulette.world chatroulette.life
cheapjerseysfootballshop.com cheap-trusted-backlinks.com
ci.ua checkhit.com
cialis-samples.com chocolateslim-en-espana.com
christianlouboutinreplica.win
cialisovercounteratwalmartusa.com cialisovercounteratwalmartusa.com
citysecurity.nu cienum.fr
cleanallvirus.com cinemaenergy-hd.ru
cleanpcnow.com civilwartheater.com
cobaltpro.ru cleansearch.net
coderstate.com clodo.ru
colobit.biz club-musics.ru
compliance-andrew.xyz cntravelre.com
compliance-margo.top compliance-don.xyz
concealthyself.com compliance-george.xyz
constantaservice.net cookielawblog.wordpress.com
cordstrap.cc coolgamechannel.com
country-chic.ru customcollegeessays.net
covadhosting.biz dairyindia.in
craftinsta.ru darkbooks.org
cutt.us
cyse.tk
danschawbel.com
datadepths.com
datsun-do.com
decorazilla.com
demon-tweeks.com demon-tweeks.com
den-noch24.ru
derevesendeco.com
detalizaciya-tut.biz detalizaciya-tut.biz
deutschehobbyhuren.net detki-opt.ru
dieta-personalna.pl detoxmed24.ru
diminishedvalueoforegon.com detsky-nabytek.info
directivepub.com dev78.dioniqlabb.se
dnsrsearch.com dichvuvesinhhanoi.com
doctissimo.fr dieswaene.com
dogsrun.net dipstar.org
dolohen.com dneprsvet.com.ua
domination.ml doctormakarova.ru
donvito.unas.cz doktoronline.no
drivotracker.com dorratex.tn
drupa.com downloadeer.net
eager-nash.188-93-233-196.plesk.page doxysexy.com
easytuningshop.ru dynainbox.com
econom.co e-buyeasy.com
egvar.net eas-seo.com
el-nation.com edge.sharethis.com
elektrozigarette2.wordpressy.pl electrik-avenue.com
elektrozigaretten2.yn.lt elektroniksigaraankara.info
eloconcream.blogspot.com elektrozigarette2.webs.com
encodable.com elektrozigaretten1.tumblr.com
elvel.com.ua
energydiet24.ru
englishgamer.com
eonpal.com eonpal.com
epicbrogaming.com erotik-kostenlos.net
erectile.bid eshop4u.jp
ertelecom.ru essay-writing.work
escort-russian.com etotupo.ru
escortplius.com euronis-free.com
essay-edu.biz europages.com.ru
essayassist.com excaliburfilms.com
estibot.com exchanges-bet.com
evaashop.ru expolicenciaslatam.co
evidencecleanergold.com ezaz.info
exonline.info f0815.de
expertnaya-ocenka.ru f1nder.org
extrabot.com faithe.top
extractorandburner.com fapgon.com
extremal-blog.com fatmaelgarny.com
ezofest.sk feargames.ru
familyphysician.ru felizporno.com
fenoyl.batcave.net
fertilitetsradgivningen.se
fetishinside.com fetishinside.com
filefilter.weebly.com
filerockstar298.weebly.com filerockstar298.weebly.com
financepoints.eu fivedwld.cf
fingerprintjs.com fix-website-errors.com
fitfloponline.store flash4fun.com
flashlarevista.com forex-indextop20.ru
floridahuntingfishingadventures.com forsex.info
foodgid.net free-laptop-reward.com
forminecrafters.ru freshsuperbloop.com
fortwosmartcar.pw funtoonez.com
forzeronly.com fxgallery.com
fototravel.eu
freakycheats.com
freeuploader.ml
freshberry.com.ua
frivgame250.com
fuckingawesome.com
fungamelands.com
fungirlsgames.net
fxund.us
gambarkatabaru.com
game300.ru
gameonasia.com gameonasia.com
gardene.ru gamevalue7.weebly.com
gdcentre.ru
genericlowlatencyasiodriverhq.aircus.com
gerhardhealer.com
get-free-social-traffic.com
getmyads24.com getmyads24.com
getoutofdebtfree.org greatdealshop.com
ghernnqr.skyrock.com grizzlysgrill.com
gkvector.ru gsmlab.pl
glavtral.ru gtopstats.com
go2mike.ru hahashka.ru
googlemare.com haikuware.com
gotorussia.com hd720kino.ru
gox.com.ua hentai-manga.porn
gq-catalog.gq herehloadibs.cf
gratuitxblcodes.com hermesbelts.xyz
greatzip.com hermesreplica.pw
growshop.es
gta-top.ru
hausfrauensex18.com
headpress.ru
healthcarestore.info
heatpower.ru
helvetia.com.ua
hidefiles.org hidefiles.org
highspeed5.net himazin.info
hledejvshopech.cz histats.com
homeandhealth.ru hmywwogw.bloger.index.hr
hobild.net
homedo.fabpage.com
homegardenlova.com
hoste.octopis.com
hostsshop.ru hostsshop.ru
hotcore.info hotchatdate.com
hunthillfarmtrust.org hoztorg-opt.ru
hyhj.info hustoon.over-blog.com
hyiphunter.org hyiphunter.org
ictizanidinehcl4mg.blogspot.com hystersister.com
iideaidekonkatu.info igrovyeavtomaty777.ru
ilovevitaly.com ilo134uloh.com
imgata.com imadedinner.net
in-tandem.co infobanks.ru
inbabes.sexushost.com infowarcraft.ru
interesnie-faktu.ru intellektmedia.at
investzalog.ru interior-stickers.ru
iphantom.com invivo.hu
islamtoday.co.za iomoio.net
javitas.info irkutsk.online-podarki.com
javlibrary.cc iskalko.ru
jeremyeaton.co itrevolution.cf
jerseywholesalechinabiz.com ivanovo.zrus.org
jjbabskoe.ru iwantedmoney.com
jobgirl24.ru iwanttodeliver.com
jolic2.com janavibekken.no
jose.mulinohouse.co japfm.com
jrpmakati.com javrip.net
juliadiets.com job.icivil.ir
kalandranis.gr joessmogtestonly.com
karadene.com justprofit.xyz
kerei.ru kareliatobacco.ru
ketoanhanoi.info katjimej.blog.fc2.com
keyhantercume.com kazrent.com
kimcurlrvsms.com kemerovo.zrus.org
konica.kz keywordglobal.co.za
konkursowo-24.pl kinky-fetishes.com
konpax.com kinostorm.net
koptims.tiu.ru kiskinhouse.com
kosmetyki.tm.pl klikbonus.com
kosova.de kliksaya.com
krafte.ru koleso24.com.ua
krasivoe-hd.com krynica.info
kupit-adenu.ru kumuk.info
kw21.org kvartir-remont.biz
labvis.host.sk
ladov.ru
lafourmiliaire.com
lakomka.com.ua
lanasshop.ru lanasshop.ru
lapitec.eu lancheck.net
laxdrills.com lankarns.com
lazy-z.com law-check-eight.xyz
ledis.top leadn.pl
lenvred.org letmacwork.world
lescinq.com li-er.ru
levaquin750mg.blogspot.com linkdebrideur.xyz
likesuccess.com liver-chintai.org
linkredirect.biz liverpool.gsofootball.com
localflirtbuddies.com liveshoppersmac.com
locustdesign.co loadingpages.me
lovi-moment.com.ua lodki-pvh.dp.ua
m-google.xyz lolnex.us
m.b00kmarks.com lunamedia.co
magazintiande.ru lunchrock.co
maju.bluesalt.co m0r0zk0-krava.ru
manipulyator-peterburg.ru mackeeper-land-672695126.us-east-1.elb.amazonaws.com
manualterap.roleforum.ru magda-gadalka.ru
marmitaco.cat magiadinamibia.blogspot.com
master-muznachas.ru makedo.ru
matb3aa.com makemoneyonline.com
mecash.ru marblestyle.ru
medmajor.ru med-zdorovie.com.ua
mega-polis.biz.ua mega-bony-2017.pl
megagrabber.ru mezaruk.info
message-warning.net mhi-systems.ru
metaxalonevstizanidine.blogspot.com middlerush-a.akamaihd.net
michaelkorssaleoutletonline.net mir-limuzinov.ru
mirzonru.net mirzonru.net
misswell.net mncrftpcs.com
mobicover.com.ua mobicover.com.ua
moesonce.com
moetomnenie.com
monarhs.info
moncleroutletonline.xyz moncleroutletonline.xyz
monetizer.com-01.site moneytop.ru
money-every-day.com morepoweronmac.trade
montazhnic.ru moscow-clining.ru
mototsikl.org most.gov.iq
mturkcontent.com moz.com
musezone.su myblogregistercm.tk
musicstock.me mydownlodablefiles.com
muzaporn.com mypets.by
myhealthcare.com narkologiya-penza.ru
mykings.pw naturalbreakthroughsresearch.com
namecrumilchlet.tk naturalpharm.com.ua
narkologiya-orel.ru
negral.pluto.ro
nero-us.com nero-us.com
nerudlogistik.ru
nextbackgroundcheck.gq
nextconseil.com nextconseil.com
nhl17coinsforps3.gratisblog.biz nl.netlog.com
nikitsyringedrivelg.pen.io notaria-desalas.com
nina.az nudo.ca
nippon-bearings.ru nvformula.ru
nomuos.it obuv-kupit.ru
noscrapleftbehind.co okayimage.com
notebook-pro.ru omoikiri-japan.ru
nw-servis.ru one-gear.com
o333o.com onemactrckr.com
obiavo.com online-zaymy.ru
okel.co onlineku.com
olympescort.com optikremont.ru
omgtnc.com ostrovtaxi.ru
onefilms.net pandroid.co
onlainbesplatno.ru panicatack.com
online-x.ru paparazzistudios.com.au
online247.ml partners-ship.pro
onlyforemont.ru pensplan.com
openlibrary.org
orel-reshka.net
oyster-green.com
p-business.ru
parajumpersjakkesalgnorge.info
parlament.biz
pathwhelp.org
paydayonlinecom.com
perm.xrus.org
pflexads.com
picquery.com
picswe.com
picturesify.com picturesify.com
pijoto.net pills24h.com
pinwallpaper.xyz pinup-casino1.ru
pio.polytopesexempt.com piski.top
pisanieprac.info pivka.xyz
pkr1hand.com pixgood.com
plastgran.com platezhka.net
polska-poezja.com platinumdeals.gr
polybuild.ru polimga.pw
popugaychiki.com popugauka.ru
porndroids.com popup-hgd.xyz
porno.simple-image.com.ua pornodojd.ru
pornoxxx.com.mx pornogad.com
portside.xyz pornoplen.com
poshiv-chehol.ru pornosok.ru
pravoholding.ru posible.net
predominant-invent.tk pospr.waw.pl
preparevideosafesystem4unow.space pr0fit-b0x.com
preventheadacheguide.info pretty-mart.com
pricheskaonline.ru princevc.com
privatamateure.com prizeestates.cricket
producm.ru professionaldieselcare.com
prostitutki-belgoroda.org proftests.net
prostitutki-tyumeni.org progress-upakovka.ru
pskcijdc.bloger.index.hr promodj.com
propranolol40mg.blogspot.com
prostitutki-kiev.org
prostitutki-novgoroda.org
proxyradar.com
ptr.ruvds.com ptr.ruvds.com
putitin.me quelle.ru
qld10000.net quizzitch.net
qwarckoine.com rabot.host.sk
ragecash.com rapidgator-porn.ga
ranksays.com realnye-otzyvy.info
rapidokbrain.com rebrand.ly
rapidsites.pro redirlock.com
realitykings.com relax.ru
realmonte.net renewablewealth.com
recipedays.com resant.ru
rednise.com rethinkwasteni.info
remmling.de rubanners.com
renhacklids.tk rus-teh.narod.ru
royalads.net rvi.biz
ruf777.com
rusprostitute.com
s-iwantyou.com s-iwantyou.com
safe-app.net sakhboard.ru
samara.rosfirm.ru sammlungfotos.online
samoiedo.it
sampleletters.net
sanjosestartups.com sanjosestartups.com
sashagreyblog.ga sape.top
scanner-fred.top sarahmilne.top
scanner-jess.top sbdl.no
scansafe.net sbprabooks.com
search.pch.com scalerite.co.za
searchtooknow-a.akamaihd.net scanner-julianna.top
semprofile.com scripted.com
sethrollins.net scrnet.biz.ua
sex-tracker.com search-goo.com
sexuria.net searchkut.com
sexvporno.ru security60-e.com
sexy.babes.frontend-stack.top seo-2-0.com
sharebutton.org seocheckupx.com
shareyards.com seoexperimenty.ru
seoholding.com
sexpornotales.net
sextracker.com
shiksabd.com
shitmovs.com shitmovs.com
shmetall.com.ua shop.acim.org
shoesonlinebuy.xyz shoppingmiracles.co.uk
shoplvlv.us shoppytoolmac.com
sisi-go.ru shopwme.ru
si-unique.com
signoredom.com
signx.info
similardeals.net
site.ru
siteheart.net siteheart.net
sitiz.club sitevaluation.com
sivs.ru slujbauborki.ru
slonechka.ru smart-balancewheel.com
smartshoppymac.com snegozaderzhatel.ru
snip.tw sosdepotdebilan.com
snowplanes.com sotkal.lark.ru
social-buttons.com spacash.com
softomix.com spasswelt.net
speechfoodie.com special-porn.com
spinnerco.ca sports-supplements.us
sptslmtrafms.com sribno.net
spywarespy.com stanthonyscatholicchurch.org
stardevine.com strangeduckfilms.com
startufa.ru stroilka.info
sterva.cc studentguide.ru
streamin.to stuff-about-money.com
strfls.com styro.ru
sugarkun.com suggest-keywords.com
sugvant.ru sunflowerdrawingpaintings.blogspot.com
sumo.com sweepstakes.rewardit.com
suralink.com swsociety.se
surcentro.com t3chtonic.com
sureone.pro taqywu51.soup.io
surflinksmedical.com
svolze.com
symbaloo.com
syvertsen-da.no
t-bygg.com
tacbelarus.ru
takeprofitsystem.com
taranerymagesswa.blogspot.com taranerymagesswa.blogspot.com
tds-advert002.info taraz.xkaz.org
teguh.info templates.radiodigital.co
templates.franklinfire.co testbotprocessor44.com
theendivechronicles.com thebestphotos.eu
timer4web.com thebluffs.com
tizanidine4mg.blogspot.com thenews-today.info
tizanidinehcl4mgtablet.blogspot.com tilido.com
tizanidinehydrochloride4mgstreetvalue.blogspot.com time-japan.ru
tizanidinenarcotic.blogspot.com tizanidinefibromyalgia.blogspot.com
tizanidinerestlesslegsyndrome.blogspot.com tizanidinehclsideeffects.blogspot.com
tizanidineshowupondrugtest.blogspot.com tizanidinevscyclobenzaprine.blogspot.com
tizanidinesleepaid.blogspot.com topcar-krasnodar.ru
tn811.us topsy.com
toolsky.com torrent-to-magnet.com
top-study.work totu.info
top10registrycleaners.com tracklead.net
trackzapper.com tradgardspartner.se
trafficfactory.biz trafficstars.com
trkur.com traffmonster.info
tupper-posuda.ru truebeauty.cc
turbabitload.weebly.com trustedmaccleaner.com
turn-up-life.life
tvand.ru
twiclub.in
twittruth.com
ua-company.ru ua-company.ru
udsgame.online uhodzalijami.ru
ukkala.xyz ukkala.xyz
umaseh.com ukrtvory.in.ua
ultimatesetnewfreeallsoftupgradesystems.pw
umg-stroy.ru
unacittaconte.org
unimodemhalfduplefw.pen.io unimodemhalfduplefw.pen.io
usbggettwku.ga unitygame3d.com
usiad.net unmaroll.ya.ru
varikoz24.com upproar.com
vclicks.net us-america.ru
vegascosmetics.ru valsalud.com
velpanex.ru videochat.mx
ventelnos.com videochat.ph
verymes.xyz vip-parfumeria.ru
via-energy-acquistare.com vodkoved.ru
viddyoze.com vonradio.com
videooko.weebly.com vostoktrade.info
videtubs.pl vroze.com
viel.su vseuznaem.com
vip.51.la
vipromoffers.com
visionwell.com.cn
vuclip.com vuclip.com
vvon.co.uk
vzglyadriv.kg
vzlomsn.org vzlomsn.org
vzubah.com wallpaperaccess.com
w7s.ru wardreapptokone.tk
webcamdevochka.com webalan.ru
website-analyzer.info weblibrary.win
website-speed-checker.site weburok.com
wejdz-tu.pl wesharepics.info
whereiskentoday.com westermarkanjou.se
withstandingheartwarming.com wetgames.ru
wnoz.de white-truck.life
womensplay.net wholesalejerseys-cheapest.com
womensterritory.ru wildnatureimages.com
worldinternetauthority.com wonderfulflowers.biz
wowas31.ucoz.ru writersgroup580.web.fc2.com
www.solartek.ru writingservices17.blogspot.ru
wsgames.ru
wygraj-skiny.win
xgftnlrt.bloger.index.hr
xjlottery.com
xmnb.net xmnb.net
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xn--90acenikpebbdd4f6d.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xn--b1ag5cfn.xn--p1ai
xtubeporno.net xn--d1abj0abs9d.in.ua
xwatt.ru xn--e1afanlbnfckd7c3d.xn--p1ai
xxxmania.top xxx-treker.ru
yadro.ru xxxguitars.com
yaderenergy.ru
yhit.press yhit.press
yoluxuryevents.com yoluxuryevents.com
yougame.biz yoursite.com
youradulthosting.com youthreaders.com
youtubedownload.org youtube-downloader.savetubevideo.com
zarajbuilders.com za-music.mymobiplanet.com
zigzog.ru zarenica.net
zoodrawings.com zoogdinsney.com
zscalerone.net zvooq.eu

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01apple.com 08800.top
2000k.ru 5u.com
9icmzvn6.website 9med.net
aa08daf7e13b6345e09e92f771507fa5f4.com a1security.com.ua
aab94f698f36684c5a852a2ef272e031bb.com aa625d84f1587749c1ab011d6f269f7d64.com
abtasty.com
acheterviagrafr24.com acheterviagrafr24.com
acortarurl.es adf.ly
ad-words.ru admatic.com.tr
affrh2015.com
ahrefs.com ahrefs.com
alfabot.xyz akuhni.by
alive-ua.com alert-jdh.xyz
allwomen.info alldezservice.kz
anal-acrobats.com allforminecraft.ru
anniemation.com allknow.info
apartmentbay.ru allwidewallpapers.com
antons-transporte.de
ap.senai.br
apccargo.com apccargo.com
arkartex.ru artefakct.com
artpicso.com atelielembrancaqueencanta.com.br
atlasvkusov.ru automobile-spec.com
auto-zapchasti.org bankcrediti.ru
autowebmarket.com.ua bestofferswalkmydogouteveryday.gq
avek.ru bget.ru
azbukadiets.ru bizlist.com.de
bala.getenjoyment.net blackcurranthumidifiers.site
barbourjackets.online blackplanet.com
bard-real.com.ua blog-hits.com
basisches-wasser.net blpmovies.com
belsetirehafi.tk brainsandeggs.blogspot.com
bestadbid.com
bestdraws.com
bigcities.org
binaryoptionscops.info
bioscorp.ru
bjanshee.ru
boeuklad.com
bouda.kvalitne.cz
braip.com.br braip.com.br
brakehawk.com
brillianty.info brillianty.info
buyessay3.blogspot.ru bristolhotel.com
bycontext.com cakesplus.com.au
candytiner.com candycrushshop.com
cartierlovestore.xyz
cartierreplica.xyz
castingbank.ru
cb.iphantom.com
centrdebut.ru
cheapmoncler.win cheapmoncler.win
check-host.net christianlouboutinshoes.xyz
cjmc.info clean-start.net
cleanpcnow.com compliance-don.top
cmhomestayagency.com compliance-elena.top
compliance-irvin.top conciergegroup.org
creditmoney.com.ua cookie-law-enforcement-ee.xyz
custom-electric-guitar.com creams.makeforum.eu
ddospanel.com daptravel.com
defenderxtactical.com dawlenie.com
dodlive.mil dev78.dioniqlabb.se
dojki365.online diminishedvalueoforegon.com
domain-tracker.com dkvorota.ru
domainroam.win domainanalyzing.xyz
dorratex.tn downloaddy.net
dotmass.top drillsaw.ru
downloader12.ru drugspowerstore.com
driving.kiev.ua dvr.biz.ua
drivotracker.com dvrlists.com
e-kwiaciarz.pl ec-file.info
elektrischezigarette1.onsugar.com ekspresihati.info
elektronischezigarette2.mex.tl ekto.ee
eloconcream.blogspot.com elektrischezigarette2.devhub.com
enhand.se elektrischezigaretten2.beeplog.com
epicbrogaming.com elexies.info
erotiktreff24.info embedle.com
escort-russian.com evogarage.com
etenininrade.ga ezigarettenkaufen2.dreamwidth.org
fetlifeblog.com face.hostingx.eu
figuringmoneyout.com fakehandbags.xyz
forum.poker4life.ru ferieboligkbh.dk
forumprofi.de filmi-v.online
freemags.cc findanysex.com
fullfileaccess.com finuse.com
funnel.co.za flooringinstallation-edmonton.com
gameplexcity.com forum-engineering.ru
gdebestkupit.ru forum.tvmir.org
given2.com foxweber.com
globetrotting-culture.ru free-social-buttons.com
gnuetella.com freesitetest.com
goodly.pro froggytube.com
gsasearchengineranker.pw fx-brokers-review.com
gtopstats.com gazobeton-p.com.ua
harmonyglen.us gcup.ru
heartofpayne.xyz get-free-traffic-now.com
hebr.myddns-flir.com girls-ufa.ru
hentaiheroes.com global-ics.co.za
high-speed1.net globatur.ru
hollywoodactress.info glopages.ru
howtoclean.club googst2.ru
hystersister.com guardlink.org
icaseclub.ru hatdc.org
ie.57883.net hazardky.net
ilo134uloh.com hdhc.ru
imfamous.info hdpixent.com
impresagaia.it healthcarestore.info
increasewwwtraffic.info helvetia.com.ua
istizanidineanarcoticdrug.blogspot.com highspeed5.net
ivearchenceinflu.cf histats.com
iweblist.info hobild.net
kazinogames.lv houseofrose.com
kidd.reunionwatch.com ilovevitaly.com
kinobest.pl imagine-ex.co
kinogonew.ru imgarit.pw
kinoplen.ru imgata.com
kinotorka.ru insurple.com
kirov.zrus.org interesnie-faktu.ru
ku6.com intim-uslugi.info
lankarns.com invitefashion.com
larose.jb2c.me ivanstroi.ru
lego4x4.ru ix20.ru
letslowbefast.today jackwolfskinoutlet.online
likesdesign.com jaxcube.info
linkrr.com jerseyssportsshop.com
lion.misba.us karaganda.xkaz.org
littlecity.ch kit-opt.ru
loginduepunti.it krassh.ru
lol-smurfs.com kretpicf.pw
lonely-mature.com lapolis.it
longadventure.com larger.io
louboutinreplica.xyz laserpen.club
lovelycraftyhome.com leadn.pl
macrotek.ru leftofcentrist.com
mail.allnews24.in linkwithin.com
marblestyle.ru live-sexcam.tk
max-eclat.men liveinternet.ru
michellblog.online lkbennettstore.online
missvietnam.org lockerz.com
mncrftpcs.com locksmith.jp
mojpregled.com map028.com
most-kerch.org marmitaco.cat
moyaterapiya.ru master-muznachas.ru
msk.afora.ru masthopehomes.com
my-aladin.com matchpal-a.akamaihd.net
mydirtyhobby.com mecash.ru
myhealthcare.com menstennisforums.com
nagdak.ru metarip.ru
naturalbreakthroughsresearch.com mister-shop.com
noumeda.com mixed-wrestling.ru
nudepatch.net mnimmigrantrights.net
nudo.ca mobplayer.ru
ohmyrings.com monthlywinners.com
okout.ru mypets.by
olympescort.com nationalbreakdown.com
page2rss.com newstraveller.ru
pagesense.com nhl17coins.exblog.jp
partner-pop.men nordvpn.com
partnerads.men novgorod.xrus.org
pathwhelp.org nrjmobile.fr
pc-test.net obiavo.uz
perfection-pleasure.ru olvanto.ru
picturesify.com onemantrip.com
ontargetseo.us
onthemarch.co
ops.picscout.com
orsonet.ru
osb.se11.ru
ozas.net
par-fallen.ga
paramountmarble.co.uk
pdns.cz
perfectplanned.com
pigrafix.at pigrafix.at
pinkduck.ga pihl.se
pinsdaddy.com play-movie.pl
pixshark.com porngalleries.top
playtap.us pornoforadult.com
pliks.pl pornosok.ru
podshipniki-ntn.ru portal-eu.ru
pornhub-forum.ga preg.marketingvici.com
pornmania.pl profitfx.online
pornogratisdiario.com projefrio.com.br
pornokajf.com prostitutki-kharkova.org
pornokorol.com prostitutki-kiev.org
prointer.net.ua qitt.ru
prosmibank.ru quangcaons.com
pushdata.sendpulse.com quickbuck.com
questionmarque.ch razyboard.com
quickcashlimited.com rennlist.com
rapidokbrain.com risparmiocasa.bz.it
recordpage-a.akamaihd.net rockingclicks.com
rfid-locker.co rockprogblog.com
rfserial.net royalcar-ufa.ru
rome2rio.com rozalli.com
rostov.xrus.org rvzr-a.akamaihd.net
rubanners.com
ryetaw.com ryetaw.com
s-forum.biz samo-soznanie.ru
salutmontreal.com satellite.maps.ilovevitaly.com
sawin.beth.webd.pl sbf441.com
sbprabooks.com seeingmeerkat.com
scanner-andrew.top sextracker.de
scanner-donald.top sexy-screen-savers.com
scanner-jack.top sexy.babes.frontend-stack.top
scanner-mark.top sharebutton.org
searchtooknow-a.akamaihd.net sigmund-freud.co.uk
seo18.su skincrate.net
shop.acim.org sluganarodu.ru
si-unique.com social-buttons.com
silvermature.net speedup-my.site
smsactivator.ru stardevine.com
snegozaderzhatel.ru steamoff.net
socialbookmarksubmission.org streha-metalko.si
socialtrade.biz summerlinhomes411.info
solmarket.by surflinksmedical.com
sophang8.com thaismartloan.com
studentguide.ru theautoprofit.ml
superkanpo.com thecoral.com.br
tackletarts.co thegameriders.com
takethatad.com tilido.com
teastory.co timhost.ru
thebestweightlosspills.ovh tizanidinehcl2mgsideeffects.blogspot.com
theguardlan.com tizanidinesideeffects.blogspot.com
themeforest.net
tizanidineandgabapentin.blogspot.com
tizanidineorflexeril.blogspot.com
tk-assortiment.ru tk-assortiment.ru
tksn.ru token-lab.org
top10registrycleaners.com tozup.com
topclickguru.com
trackmedia101.com
tracxn.com
traffic100.com
tri-slona.org tri-slona.org
tsstcorpcddvdwshbbdriverfb.aircus.com trustedmaccleaner.com
tuckpointingmasonrysystems.com tube8.com
u-cheats.ru tubo360.com
uginekologa.com ugguk.online
urccvfmc.bloger.index.hr umnovocaminho.com
uvozdeckych.info valkiria-tk.ru
vacances-voyages.info vipms.ru
vbtracker.net virtuagirl.com
vgoloveboli.net virtualbb.com
videokrik.net vseigry.fun
visa-pasport.ru vykupavto-krasnodar.ru
vsesubwaysurfers.com webpromotion.ae
webcamtalk.net webtherapy.ru
welcomeauto.ru woman-tampon.ru
wovis.site worldtraveler.world
writersgroup580.web.fc2.com wosik-dach.service-for-web.de
x-true.info wufak.com
xlolitka.com ww1943.ru
xmladserver.com wygraj-teraz.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xn--h1aakne2ba.xn--p1ai
xn----7sbahjd3btneuw1joc.xn--p1ai xolodremont.ru
xn----7sbho2agebbhlivy.xn--p1ai xrus.org
xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai
yaderenergy.ru
yaminecraft.ru
yebocasino.co.za
your-bearings.com
youradulthosting.com youradulthosting.com
youresponsive.com youraticles.pl
za-music.mymobiplanet.com yourporn.com
zastroyka.org yoursearch.me
zazagames.org ytmnd.com
zeleznobeton.ru zambini.ru
znaturaloriginal.com zone-kev717.info
zoodrawings.com
zoolubimets.ru
zootravel.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.4 0; 10.1.1.219 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3082 ### Version: V4.2022.05.3083
### Updated: Tue May 3 10:03:49 UTC 2022 ### Updated: Tue May 3 22:01:34 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.05.3082 ### Version: V4.2022.05.3083
### Updated: Tue May 3 10:03:49 UTC 2022 ### Updated: Tue May 3 22:01:34 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##