vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3445 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-01 22:07:24 +00:00
parent 14c0080689
commit 6d5ec2773c
46 changed files with 82284 additions and 82284 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3444 #### Version: V4.2022.11.3445
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3444 #### Version: V4.2022.11.3445
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.10.3444 #### Version: V4.2022.11.3445
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10150
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.10.3444 #### Version: V4.2022.11.3445
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10154
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10154
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10154
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10154
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

10154
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

10154
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

10154
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3444 3445

10154
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10154
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.85.83.161 0; 20.245.244.65 0;

10154
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

10154
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

10154
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

10154
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.85.83.161 0; 20.245.244.65 0;

10154
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.85.83.161 0; 20.245.244.65 0;

10154
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AhrefsBot BDFetch
FDM Boardreader
Mojolicious Buck
SentiBot CSHttp
Sqworm Libwhisker
Teleport NetAnts
WebWhacker Offline Navigator
Site Sucker
SpankBot
arquivo-web-crawler arquivo-web-crawler
magpie-crawler
xpymep1.exe

306
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
404enemy 404enemy
ALittle Client ALittle Client
Aboundex
AhrefsBot AhrefsBot
AiHitBot
Alexibot Alexibot
Alligator
AlphaBot AlphaBot
Anarchie Anarchy
Anarchy99 Anarchy99
Ankit Atomseobot
Anthill
Apexoo
Asterias
Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike
BDCbot
BackDoorBot
BackStreet
BackWeb BackWeb
Backlink-Ceck Backlink-Ceck
BacklinkCrawler
Bandit
Barkrowler Barkrowler
BatchFTP BatchFTP
Battleztar Bazinga
BetaBot
Bigfoot
Bitacle
Blackboard Blackboard
Blow Blow
BlowFish BlowFish
BuiltBotTough BotALot
Bullseye CCBot
BuzzSumo CODE87
CensysInspect CensysInspect
CheTeam CheTeam
CheeseBot Chlooe
Cogentbot Citoid
Copier Cloud mapping
Copyscape Cocolyzebot
CopyRightCheck
Curious
Custo
DTS Agent DTS Agent
Demon
Deusu Deusu
Digincore Devil
Discobot DittoSpyder
Dispatch DnBCrawler-Analytics
Domains Project DomCopBot
Dotbot DomainAppender
DomainSigmaCrawler
Download Wonder Download Wonder
Dragonfly
ECCP/1.0 ECCP/1.0
EMail Wolf EMail Siphon
EasyDL
Ebingbong Ebingbong
EirGrabber
EroCrawler
Exabot
Express WebPictures Express WebPictures
ExtLinksBot Extreme Picture Finder
ExtractorPro Fimap
FHscan Flunky
Firefox/7.0
Foobot
Fuzz
FyberSpider FyberSpider
Fyrebot Fyrebot
GT::WWW GalaxyBot
GermCrawler Getintent
Go!Zilla Gigabot
Gotit
GrabNet GrabNet
Grafula Grabber
GrapeFX HTTrack
HTTP::Lite Harvest
HaosouSpider Havij
Humanlinks HonoluluBot
HybridBot Id-search
IRLbot
IlseBot IlseBot
Image Fetch Image Fetch
Image Sucker Image Sucker
Indy Library
InterGET InterGET
Iskanie Iria
Jbrofuzz JamesBOT
Jetty Jetty
JikeSpider JikeSpider
Joomla Jorgee
JustView Jyxobot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Keyword Density Keyword Density
Kozmosbot Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Leap
LeechFTP LeechFTP
LeechGet LeechGet
Lftp
Lightspeedsystems
Likse Likse
LinkextractorPro LinkScan
LinkpadBot LinkWalker
Linkbot
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
MIDown tool Litemage_walker
MIIxpc MFC_Tear_Sample
MSFrontPage MJ12bot
MQQBrowser
MSIECrawler MSIECrawler
Mail.RU_Bot
Majestic-SEO Majestic-SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch
Masscan
Mata Hari Mata Hari
MauiBot MeanPath Bot
Mb2345Browser Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Microsoft Data Access MegaIndex.ru
Mister PiX Metauri
MicroMessenger
Microsoft URL Control
Minefield
Moblie Safari
Mojeek Mojeek
Mojolicious
Morfeus Fucking Scanner
Mozlila
Mr.4x3 Mr.4x3
Msrabot Msrabot
Name Intelligence
Nameprotect Nameprotect
Navroad
Needle Needle
Net Vampire
NetMechanic
NetSpider
NetZIP NetZIP
Netcraft Netcraft
NextGenSearchBot Nettrack
Netvibes
Nibbler Nibbler
Nmap Niki-bot
Octopus Nikto
Ninja
Not
Offline Explorer Offline Explorer
OnCrawl OnCrawl
OpenLinkProfiler
Openfind Openfind
Openvas OrangeSpider
OutfoxBot OutclicksBot
PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
Page Analyzer
PageGrabber PageGrabber
PageScorer PageThing.com
Pandalytics Papa Foto
Panscient Piepmatz
Petalbot Pixray
Pi-Monster
Picscout
PictureFinder
Pimonster
PleaseCrawl PleaseCrawl
ProWebWalker
Probethenet Probethenet
Pu_iN Pump
PxBroker PxBroker
PyCurl
QueryN Metasearch QueryN Metasearch
RankFlex Rankivabot
RankingBot
RealDownload RealDownload
Reaper Reaper
RebelMouse RebelMouse
Recorder Recorder
Ripper RedesScrapy
Rogerbot RocketCrawler
SBIder SEOkicks
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats Searchestate
SalesIntelligent
ScanAlert
Screaming
ScreenerBot
ScrepyBot
SearchmetricsBot
Seekport
SemanticJuice SemanticJuice
SentiBot Seomoz
SeoSiteCheckup
Shodan Shodan
SiteLockSpider Siphon
Site Sucker Site Sucker
Sitebeam SiteCheckerBotCrawler
Siteimprove Snake
Sitevigil
SlySearch
Snoopy
SocialRankIOBot SocialRankIOBot
Sosospider
Sottopop
SpaceBison SpaceBison
SpankBot
Spinn3r
SputnikBot
Sqlmap
Sqlworm
Sucuri
SuperBot SuperBot
Surfbot Surfbot
Suzuran Szukacz
Swiftbot T0PHackTeam
Telesphorep Telesoft
TheNomad Telesphoreo
Titan TightTwatBot
Toweyabot Toweyabot
Trendictionbot True_Robot
TurnitinBot Turnitin
TwengaBot
Twice Twice
URLy.Warning Typhoeus
VB Project Upflow
VCI VelenPublicWebCrawler
Vagabondo VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie WPScan
WEBDAV WWW-Collector-E
WISENutbot WWW-Mechanize
WWW::Mechanize
Wallpapers Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE Web Auto
Web Collage Web Sauger
Web Fuck WebAuto
WebEnhancer WebFetch
WebGo IS WebGo IS
WebLeacher WebLeacher
WebWhacker WebSauger
WebZIP Webalta
Webshag Webshag
Whack WebsiteExtractor
Whacker Webster
WiseGuys Robot Whatweb
Zade Who.is Bot
Zermelo WinHTTrack
Zeus Wonderbot
Xaldon_WebSpider
Zitebot Zitebot
ZumBot ZoomBot
adscanner archive.org_bot
arquivo.pt clark-crawler
backlink-check com.plumanalytics
coccocbot crawler.feedback
cognitiveseo
dataforseo.com dataforseo.com
demandbase-bot
domainsproject.org domainsproject.org
eCatch
gopher
instabid
internetVista monitor internetVista monitor
ips-agent isitwp.com
iubenda-radar
linkdexbot
magpie-crawler magpie-crawler
meanpathbot
mediawords mediawords
page scorer page scorer
pcBrowser pcBrowser
s1z.ru ripz
satoristudio.net
scalaj-http
scan.lol scan.lol
seobility
seocompany.store seocompany.store
seoscanners seostar
sitechecker.pro sogouspider
siteripz
sp_auditbot sp_auditbot
spyfu tAkeOut
trendiction.com
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webprosbot webprosbot
x09Mozilla
xpymep1.exe
zauba.io zauba.io
zgrab

290
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404enemy 404enemy
AIBOT ADmantX
Aboundex Abonti
Acunetix Aboundexbot
AfD-Verbotsverfahren
AiHitBot AiHitBot
AllSubmitter
Alligator Alligator
AlphaBot
Anarchie Anarchie
Anarchy Anarchy
Anarchy99
Anthill Anthill
Apexoo Aspiegel
Asterias Asterias
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BBBike
BDCbot
BDFetch BDFetch
BackDoorBot BackStreet
BackWeb BackWeb
BacklinkCrawler
Bandit Bandit
Barkrowler Battleztar Bazinga
BatchFTP
BetaBot
Bigfoot Bigfoot
Bitacle
Blow Blow
BlowFish BlowFish
Brandwatch Brandwatch
Buck
Buddy
BunnySlippers
BuzzSumo
CATExplorador CATExplorador
CCBot
CSHttp CSHttp
Calculon Cegbfeieh
CheTeam
CheeseBot CheeseBot
CherryPicker CherryPicker
Chlooe Claritybot
Citoid
Cocolyzebot Cocolyzebot
Cogentbot Cogentbot
Collector Collector
CopyRightCheck
Copyscape Copyscape
Cosmos Craftbot
Crescent Crescent
CrunchBot
Curious Curious
Custo Custo
CyotekWebCopy DSearch
DIIbot
DTS Agent
Demon
DigitalPebble DigitalPebble
Dirbuster
Disco
Discobot Discobot
Dispatch
DittoSpyder DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DomCopBot DnyzBot
DomainStatsBot Download Wonder
Dotbot Drip
Dragonfly EMail Wolf
ECCP/1.0
EMail Siphon
Ebingbong Ebingbong
Ecxi Ecxi
EirGrabber Evil
Extreme Picture Finder Express WebPictures
EyeNetIE FemtosearchBot
Fimap Flunky
Foobot Foobot
FrontPage GetRight
Fyrebot
Getintent
Gigabot Gigabot
Go!Zilla Gotit
Grafula Grabber
GrapeFX GridBot
HMView HMView
HTTrack Haansoft
Havij HaosouSpider
Hloader Harvest
HonoluluBot HonoluluBot
IRLbot IDBot
Image Sucker Image Fetch
Indy Library IndeedBot
InfoNaviRobot Intelliseek
InfoTekies Internet Ninja
InterGET InternetSeer
Iskanie JennyBot
Jbrofuzz
JetCar
Jetty Jetty
JustView
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
LNSpiderguy Keyword Density
Kinza
Kozmosbot
LWP::Simple LWP::Simple
Lanshanbot Larbin
LibWeb
Libwhisker Libwhisker
LieBaoFast LieBaoFast
LinkScan Lightspeedsystems
LinkWalker
Linkbot
LinkextractorPro LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
MQQBrowser MFC_Tear_Sample
MIIxpc
MJ12bot
MSFrontPage
MTRobot MTRobot
Magnet Magnet
Majestic-SEO Mail.RU_Bot
MauiBot MarkWatch
MeanPath Bot Mb2345Browser
Meanpathbot Meanpathbot
Mediatoolkitbot
Metauri
Microsoft URL Control Microsoft URL Control
Minefield Mister PiX
Moblie Safari Moblie Safari
Navroad MolokaiBot
Net Vampire Mr.4x3
Nameprotect
NearSite
Nessus
NetLyzer NetLyzer
NetZIP NetMechanic
Netvibes Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler
NimbleCrawler NimbleCrawler
Nuclei
Nutch Nutch
Offline Explorer OpenLinkProfiler
Offline Navigator
Openfind Openfind
OrangeBot OutclicksBot
OrangeSpider
OutfoxBot
PECL::HTTP
PHPCrawl
Page Analyzer Page Analyzer
PageThing.com PageGrabber
Panscient Pandalytics
Papa Foto
Pavuk Pavuk
PeoplePal Pi-Monster
Petalbot Picscout
Picsearch Picsearch
PictureFinder
Piepmatz Piepmatz
Pimonster Pixray
ProPowerBot ProWebWalker
Pump Probethenet
RSSingBot PxBroker
RankActive RankFlex
RankActiveLinkBot
RankingBot2
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SEOkicks SBIder
SEOstats SEOstats
SalesIntelligent SISTRIX
ScanAlert SMTBot
ScoutJet Scanbot
ScrepyBot Scrapy
Searchestate Searchestate
SearchmetricsBot SemrushBot
Seekport SeobilityBot
Shodan
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
SiteSnagger
SiteSucker SiteSucker
Sitebeam
Siteimprove Siteimprove
Snapbot SmartDownload
Snake
Snoopy Snoopy
SocialRankIOBot
Spanner Spanner
Spbot Spbot
Spinn3r SputnikBot
Sqlworm Stripper
Sqworm Sucker
SurveyBot Sucuri
T8Abot SuperBot
TeleportPro SuperHTTP
Suzuran
Szukacz
Telesphoreo Telesphoreo
Telesphorep TheNomad
The Intraformant
Thumbor Thumbor
Titan TightTwatBot
Toata Trendiction
Toweyabot
Trendictionbot
Turingos
Turnitin Turnitin
Typhoeus TwengaBot
URLy Warning URLy Warning
VB Project V-BOT
VCI VCI
Vacuum
Vagabondo Vagabondo
VidibleScraper VoidEYE
Virusdie
Voltron Voltron
WASALive-Bot WEBDAV
WBSearchBot WISENutbot
Wallpapers WWW::Mechanize
WeSEE
Web Enhancer Web Enhancer
Web Fuck Web Fuck
Web Sauger Web Pix
Web Sucker
WebBandit
WebCollage WebCollage
WebFetch WebEnhancer
WebFuck WebFuck
WebImageCollector WebGo IS
WebLeacher WebLeacher
WebReaper
WebStripper WebStripper
WebZIP WebWhacker
Webalta
WebmasterWorldForumBot
Webshag Webshag
Whacker Who.is Bot
Wonderbot Wonderbot
Woobot Woobot
Wotbox Wotbox
Wprecon
Xaldon WebSpider Xaldon WebSpider
YoudaoBot Xenu
Zauba
Zitebot
ZoominfoBot ZoominfoBot
ZumBot ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler arquivo.pt
autoemailspider
backlink-check
cah.io.community cah.io.community
coccocbot check1.exe
com.plumanalytics clark-crawler
crawl.sogou.com crawl.sogou.com
demandbase-bot dataforseo.com
domainsproject.org
eCatch eCatch
evc-batch evc-batch
gopher instabid
heritrix ips-agent
internetVista monitor
isitwp.com isitwp.com
linkdexbot linkdexbot
meanpathbot magpie-crawler
mediawords
muhstik-scan muhstik-scan
pcBrowser
polaris version polaris version
ripz
satoristudio.net satoristudio.net
seoscanners seobility
sitechecker.pro serpstatbot
sexsearcher
sp_auditbot
spyfu spyfu
trendiction.com sysscan
tAkeOut
voyagerx.com voyagerx.com
webmeup-crawler webmeup-crawler
x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
cenoval.ru allblogroll.com
clmforexeu.com monclerjacketsoutlet.pw
freelifetimefuckbook.com porndroids.com
lacloop.info pushdata.sendpulse.com
ladsblue.com rankia.com
pathwhelp.org snts.shell-pmr.ru
ploenjitmedia.azurewebsites.net thecoral.com.br
resultshub-a.akamaihd.net uac.net.au
telemetryverification.net viagra.pp.ua
tengohydar.tk wovis.site

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 01apple.com
0912701309f8ce.com 0n-line.tv
0daymusic.org 108shot.com
1000n1.ru
1008.su
11235813.webzdarma.cz
11pikachu.ru 11pikachu.ru
13tabs.com 1j7740kd.website
2000k.ru 23kw.ru
2itech.ru 2x2fan.ru
4ureyesonly.com 4webmasters.com
76brighton.co.uk 5-steps-to-start-business.com
777-club.ru 5elementov.ru
7wind.ru 7xc4n.com
8xv8.com 83net.jp
9med.net 98oi.ru
a1security.com.ua a96527gi.beget.tech
aa14ab57a3339c4064bd9ae6fad7495b5f.com aaa38852e886ac4af1a3cff9b47cab6272.com
aa81bf391151884adfa3dd677e41f94be1.com aad1f4acb0a373420d9b0c4202d38d94fa.com
abclauncher.com aarbur.com
adclickthru.net abtasty.com
adconscious.com addbin.men
adelly.bg adman.gr
adsloads.com adnotbad.com
adultfriendfinder.com adpremium.org
adultmeetonline.info adprotect.net
advertisingtag.net ads.gold
advokateg.xyz agreda.pluto.ro
agadelha.com.br alfa-img.com
agecheckadult.com all4invest.info
agroeconom.kz allcredits.su
airlandsea.info
ald2014.org
alf-img.com
alphacarolinas.com
alveris.ru
amanda-porn.ga
amazon-adsystem.com amazon-adsystem.com
animenime.ru amehdaily.com
apiadanaknet-a.akamaihd.net amospalla.es
apiwebwebgetcom-a.akamaihd.net anal-acrobats.hol.es
applicationg29.com analytics-ads.xyz
aramaicmedia.org apibatbrowsecom-a.akamaihd.net
arius.tech apiluckyleapnet-a.akamaihd.net
arpelsreplica.xyz apiportalmorecom-a.akamaihd.net
arquivo.pt app-ready.xyz
arraty.altervista.org appmsr.org
approved.su
appsecurityr.com
architecturebest.com
artel116.ru
articlesdirectoryme.info articlesdirectoryme.info
asscenihotosidea.blogspot.co.za asrvvv-a.akamaihd.net
asscenihotosidea.blogspot.com aucoinhomes.com
atelielembrancaqueencanta.com.br audiobangout.com
autoloans.com awency.com
autolombard-krasnodar.ru
autorn.ru
avirasecureserver.com
azbuka-mo.ru
b-styles.xyz b-styles.xyz
bagcionderlab.com b00kmarks.com
bala.getenjoyment.net babs.com.ua
batanga.net baksman.com
belsetirehafi.tk balkanfarma.ru
best-seo-solution.com baltgem.com
bestcoin.cc bannerads.de
bestdraws.com bashtime.ru
bestplacetobuyeriacta.jw.lt bbsoldes.fr
bestserials.com beauty-b0x.pl
bfz.biz beauty-bracelet.com
big-boards.info beauty-things.com
bimatoprost-careprost.com belinka.com.ua
biteg.xyz bensbargains.net
bkgr.se bestwebsitesawards.com
blackwitchcraft.ru beta.hotkeys.com
blogqpot.com betshuckclean.com
blogrankers.com betteroffers.review
bombla.org bg6s0.com
brainxs.us bioca.org
bristolhostel.com biocn.dx.am
btc4u.ru biographiya.com
bizfly.info
blackplanet.com
bookhome.info
boom-celebs.com
brains2.biz
brainzpod.win
bravegirlsclub.com
breastaugmentation.co.za
brendbutik.ru
bugshoot.cn bugshoot.cn
bumble.cheapwebsitehoster.com bumskontakte.org
businesxxl.com burn-fat.ga
buyessay3.blogspot.ru buytizanidineonlinenoprescription.blogspot.com
buyparajumpers.online candy-glam-hp.com
bycontext.com cartierbracelet.xyz
byme.se casablancamanor.co.za
bzero1jewelry.net caveavins.fr
cacheimages.com cbox.ws
callawaygolfoutlet.online cdncash.org
cardul.ru cejewelry.xyz
casinorewards.info cerev.info
celejihad.info certifywebsite.win
cheapestjerseysonwholesale.com cete.ru
cheapmoncler.pw cfcl.co.uk
chelnytruck.ru chatroulette.world
cherrypointplace.ca cheapbelstaff.online
ci.ua cheatcode-lita12.rhcloud.com
cialisndbrx.com children-learningreading.info
classiquebijoux.ru cineacademy.ru
clickhype.com clickaider.com
clipartnew.com clickcash.com
club-samodelkin.ru coldfilm.ru
coderstate.com collegerentals.com
coin-hive.com colobit.biz
compliance-walt.top communisave.co.za
converse.ddsoldes.fr compliance-barak.xyz
cookie-law-enforcement-ee.xyz compliance-jack.top
cookie-law-enforcement-ff.xyz connectionstrenth.com
copblock.org contentsexpress.com
counter.bloke.com contextualyield.com
covadhosting.biz coolwallpapers-hd.com
culad.com countercrazy.com
custom-electric-guitar.com covid-schutzmasken.de
craftburg.ru
crystalslot.com
dame-ns.kz
daooda.com
darkages.info
ddlmega.net ddlmega.net
decenttools.com de.zapmeta.com
delfin-aqua.com.ua dealitright.click
delta-line.men delta-line.men
dinkolove.ya.ru demian.kz
doeco.ru diegolopezcastan.com
domain2008.com discountbarbour.online
domaincrawler.com djstools.com
domination.ml domik-derevne.ru
doska-vsem.ru dprtb.com
downloadmefiranaratb1972.xpg.com.br drugspowerstore.com
dstroy.su
dverimegapolis.ru
e-biznes.info
e-commerce-seo1.com
e2click.com
e705.net
eachdayisagift.review
editors.choice6912650.hulfingtonpost.com
education-cz.ru education-cz.ru
ekn-art.se elidelcream.weebly.com
electrik-avenue.com enskedesquashclub.se
elektrischezigarette1.onsugar.com eonpal.com
elektrischezigarette2.devhub.com erotische-geschichten-xxl.com
elektrischezigaretten2.beeplog.com esfchat.tk
elektrozigaretten1.tumblr.com
emediate.eu
epicbrogaming.com
epngo.bz
eropho.com
erotiktreff24.info
ertelecom.ru
essay-edu.biz essay-edu.biz
etotupo.ru estibot.com
eu-cookie-law.blogspot.com euronis-free.com
eurosamodelki.ru
excaliburfilms.com
exoneration-project.us exoneration-project.us
expresstoplivo.ru expertblog.info
ezigarettenkaufen1.pagina.gr extads.net
ezigaretteshop.webs.com ezigarettenkaufen1.deviantart.com
familyholiday.ml famix.xyz
fashionindeed.ml fickkontaktehobbyhuren.com
fatmaelgarny.com fighrofacciufreesig.ga
fdzone.org filesvine.com
fetroshok.ru finansov.info
fickenbumsen.net
fidelityfunding.com
fil.ru
findpics.pw findpics.pw
findsexguide.com firma-legion.ru
fivedwld.cf fm-upgrade.ru
floating-share-buttons.com fordsonmajbor.cf
formulaf1results.blogspot.com fototravel.eu
foto-sisek.porngalleries.top
foto-telok.net
foxinsocks.ru foxinsocks.ru
freakycheats.com free-share-buttons.com
freetangodownload.com freelotto.com
freewlan.info frighteningremain.cf
freza-sverlo.ru funnymama.com
gamewrath.com
gaming-journal.com gaming-journal.com
gdebestkupit.ru gaygalls.net
gentamicineyedrops.blogspot.com gearsadspromo.club
getaclueamerica.com geckoandfly.com
ghazel.ru gilsonchiro.xyz
gktt.ru glall.ru
glavtral.ru globatur.ru
globalsurfari.com go2jump.org
googlemare.com goldpanningtools.com
gorabagrata.ru goodhousekeeping.com
gosmeb.ru goodwinmetals.co
gosreg.amchs.ru googst2.ru
gotwebsite1.com gowreckdiving.com
gox.com.ua gpms.org.my
graphics8.info gsmlab.pl
grtyi.com guiadeserraazul.com
gsou.cf guigyverpo.cf
hanink.biz.ly gunsvicceadadebt.tk
hdwallpapers-free.com gyffu.com
hdxnxxtube.mobi haveinc.xyz
health-medical-portal.info healbio.ru
hetmanship.xyz hidefiles.org
hitsbox.info
hledejvshopech.cz
home.myplaycity.com home.myplaycity.com
homegardenlova.com homeandhealth.ru
host-tracker.com homeart.space
hotblog.top hothot.ru
hotcore.info hscsscotland.com
hotenergy.ru hybrid.ru
house.sieraddns.com hyhj.info
hoverboard360.de iblogpress.xyz
howlongdoestizanidinestayinyoursystem.blogspot.com ideibiznesa2015.ru
husky-shop.cz imagez.co
hyipmanager.in
hystersister.com
ibmdatamanagement.co
ilovevitaly.xyz
imgarcade.com
imgarit.pw
iminent.com
imitex-plus.ru imitex-plus.ru
imperia31.ru imperia31.ru
impisr.ru instabid.tech
import-sales.com interferencer.ru
impotentik.com interfucks.net
inboundlinks.win irkutsk.zrus.org
incomekey.net isitpaleo.info
indianmedicaltourismshop.com iwantedmoney.com
infazavr.ru johnrobertsoninc.com
instasexyblog.com journalhome.com
investzalog.ru kambasoft.com
iphantom.com karaganda.xkaz.org
iqoption-bin.com kareliatobacco.ru
iscblog.info keywordbasket.com
istanbulit.com kicknights.gq
istizanidineanopiate.blogspot.com kinogonew.ru
izhevsk.xrus.org kinopolet.net
izoll.ru kiprinform.com
jacago.com
james13prix.info
janavibekken.no
javlibrary.cc
javspace.net
joessmogtestonly.com
johannesburgsingles.co.za
jwss.cc
kadashihotel.com
karachev-city.ru
kellyonline.xyz
keywordglobal.co.za
keywordsuggests.com
kfon.eu
kidskunst.info
kinky-fetishes.com
kinohit1.ru
kirov.zrus.org
kiskinhouse.com kiskinhouse.com
komputernaya-pomosh-moscow.ru kletkimehan.ru
krasula.pp.ua klitimg.pw
kursy-ege.ru kmgamex.cf
lacave.ntic.fr knowyournextmove.com
lapitec.eu kollekcioner.ru
lashstudia.ru konoplisemena.com
letsrepair.in kovesszucs.atw.hu
liffa.ru krasivoe-hd.net
liverpool.gsofootball.com kutikomi.net
livingroomdecor.info kvartir-remont.biz
llastbuy.ru kvartiry-remont.ucoz.ru
locatellicorretor.com.br kw21.org
lodki-pvh.dp.ua kwzf.net
loginduepunti.it lanadelreyfans.us
lonfon.xyz larger.io
lovelycraftyhome.com lasvegaslockandsafe.com
mabdoola.blogspot.com leadn.pl
machicon-akihabara.info ledis.top
mackeeper-land-672695126.us-east-1.elb.amazonaws.com lemon-ade.site
madot.onlinewebshop.net lesbian.xyz
magnetic-bracelets.ru letto.by
manipulyator-peterburg.ru limtu.ifmo.ru
map028.com linkredirect.biz
marketingvici.com lion.bolegapakistan.com
mastershef.club littlesunraiser.com
medicine-4u.org livesex-amateure.info
medicovi.com lollypopgaming.com
medtherapy.ru lonely-mature.com
meduza-consult.ru lookingglassemb.com
megahdporno.net louboutinreplica.xyz
meget.co.za mackeeper-center.club
mejoresfotos.eu macotool.com
metallosajding.ru managerpak204.weebly.com
mhi-systems.ru marketland.ml
mirobuvi.com.ua maslenka.kz
misandesign.se matchpal-a.akamaihd.net
mnogabukaff.net matpre.top
mnogolok.info mebeldekor.com.ua
moi-glazki.ru med-zdorovie.com.ua
monarchfind-a.akamaihd.net mediaoffers.click
monclercheap.xyz medicines-choice.com
moneyviking-a.akamaihd.net medmajor.ru
mexicotravelnet.com
mhtr.be
mirtorrent.net
mister-shop.com
mixtapetorrent.com
modabutik.ru
moesen-ficken.com
moesonce.com
mojaocena.com
monclerboots.xyz
money-every-day.com
monthlywinners.com monthlywinners.com
moxo.com montredemarque.nl
morocco-nomad-excursions.com
moyaterapiya.ru
mozello.ru mozello.ru
mp3films.ru mrinsidesales.com
mp3ritm.top msk-diplomat.com
multgo.ru musicstock.me
mvpicton.co.uk muz-baza.net
my-cash-bot.co mwtpludn.review
myseoconsultant.com myhydros.org
mzdish.site myliveblog.ru
najaden.no myonlinepayday.co
nalogovyy-kodeks.ru nailsimg.com
nastydollars.com nardulan.com
neks.info netfacet.net
neobux-bg.info next-dentists.tk
netallergy.ru nhl17coinsforps3.gratisblog.biz
newyorkhotelsmotels.info nodup.ru
nibbler.silktide.com noumeda.com
nicefloor.co.uk nuit-artisanale.com
nippon-bearings.ru nzfilecloud.weebly.com
nordstar.pro oda.as
nuup.info
o-o-8-o-o.com
oddamzadarmo.eu
odesproperty.com
odywpjtw.bloger.index.hr
office-windows.ru office-windows.ru
office2web.com office2web.com
offside2.5v.pl
offtime.ru offtime.ru
olvanto.ru onemantrip.com
olympescort.com onlinedomains.ru
oneshotdate.com onlinewritingjobs17.blogspot.ru
onko-24.com oren-cats.ru
online-sbank.ru orhonit.com
opinionreelle.com osb.se11.ru
optom-deshevo.ru palvira.com.ua
orenburg-gsm.ru partner-print.men
orion-code-access.net pc-virus-d0l92j2.pw
oxford-book.com.ua pekori.to
ozoz.it photokitchendesign.com
paighambot.com picurams.pw
painting-planet.com piluli.info
paleohub.info
perederni.net
petsblogroll.com
photochki.com
pio.polytopesexempt.com pio.polytopesexempt.com
pixell.club pixshark.com
pixgood.com pliks.pl
plastgranar.nu podshipniki-ntn.ru
plastjulgranar.se popmarker.com
poppen-nw.net popunder.ru
poquoson.org porngalleries.top
porndairy.in porno.simple-image.com.ua
pornhubforum.tk pornogad.com
pozdravleniya-c.ru pornoreino.com
pr-ten.de pornotubexxx.name
pr0fit-b0x.com potoideas.us
praisong.net preventheadacheguide.info
presleycollectibles.com printdirectforless.com
pricheskaonline.ru privatefx.all4invest.info
printie.com projectforte.ru
privat-girl.net prosperent.com
prodvigator.ua prostitutki-belgoroda.org
prombudpostach.com.ua prostitutki-tolyatti.org
prostitutki-almata.org prostitutki-tyumeni.org
purplesphere.in prostitutki-yaroslavlya.org
quickloanbank.com prpops.com
rada.ru pukaporn.com
rankings-analytics.com pussyspace.net
rebuildermedical.com qitt.ru
redbottomheels.xyz rainbowice.ru
registrydomainservices.com randki-sex.com
relax.ru razleton.com
relayblog.com recordpage-a.akamaihd.net
renecaovilla.online redirectingat.com
renhacklids.tk reimageplus.com
replicalouboutin.xyz reklamuss.ru
rewardit.com remontvsamare.su
rfid-locker.co rent2spb.ru
ribieiendom.no rock-cafe.info
richinvestmonitor.com rockprogblog.com
ring4rhino.com roll123.com
robertefuller.com rome2rio.com
royalcar-ufa.ru royal-betting.net
rubanners.com ruclicks.com
ruex.org.ua runofilms.ru
ruscopybook.com russia-today-video.ru
ruspdd.com russkoe-zdorovie.ru
sampleletters.net rustic-quiver.win
scanner-jess.top samo-soznanie.ru
scansafe.net sarm.tk
scrapy.org sayyoethe.blogspot.co.za
sdelatmebel.ru scanner-alex.top
search-error.com scanner-alexa.top
searchinquire.com scanner-brian.top
see-your-website-here.com scanner-elena.top
servethis.com scanner-fred.top
scanner-mary.top
schoolfiles.net
scottbywater.com
se.bnt-team.com
selectads.men
seo-platform.com
seo-traffic-ranking.info
seotoolsagency.com
serpstat.com
seryeznie-znakomstva.ru
sexad.net
sexvporno.ru sexvporno.ru
shakhtar-doneck.ru sexy-screen-savers.com
shijian.ac.cn shama-rc.net
shopperifymac.com shariki-zuma-lines.ru
shoppingjequiti.com.br sharpchallenge.com
sierraapps.com shivafurnishings.com
silverage.ru sibdevice.ru
simplesite.com sim-service.net
site-analyzer.com site5.com
site-speed-check.site siteripz.net
site.ru sitevaluation.org
siteexpress.co.il sitevalued.com
skachat-besplatno-obrazcy.ru skatestick.bid
sloopyjoes.com slim.sellany.ru
smart-scripts.com snomer1.ru
smartadserver.com snsdeainavi.info
smartshoppymac.com socialbookmarksubmission.org
snyatie-lomki-v-stacionare.ru socialsignals24.com
sobecjvuwa.com.ru sovinsteel.ru
social-search.me specialfinanceoffers.com
socialbutton.xyz sptslmtrafms.com
softomix.net stair.registrydomainservices.com
soochi.co sticken.co
sophang8.com stopnarco.ru
soundfrost.org streetfire.net
space-worry.ml stretchingabuckblog.com
spb.afora.ru strfls.com
spinnerco.ca stroilka.info
spy-sts.com studiokamyk.com.pl
sr-rekneskap.no superinterstitial.com
ssconstruction.co superoboi.com.ua
start.myplaycity.com surintech.ac.th
stats-collector.org sweepstakes.rewardit.com
stickers-market.ru swinon.site
strigkaomsk.ru tackletarts.co
sukarame.net tagil.zrus.org
supermama.top tamada69.com
supervesti.ru taqywu51.soup.io
surfbuyermac.com templates.radiodigital.co
svarkagid.com tfxiq.com
svolze.com thegalerie.eu
swingerseiten.com theporndude.com
takeflyte.com thin.me.pn
tako3.com titan-cloud.life
tds-advert002.info tivolibasket.it
tedy.su tizanidinedrugtest.blogspot.com
teenfuck.tv tizanidinehcl4mgisitanarcotic.blogspot.com
teguh.info tizanidinewithdrawal.blogspot.com
template-kid.com
terrafootwear.us
thegioixekhach.com
thepantonpractice.co.uk
tizanidineandgabapentin.blogspot.com
tnaionline.org
toloka.hurtom.com toloka.hurtom.com
tonivedu.it top-l2.com
topkarkas.com topflownews.com
torontoplumbinggroup.com torrents.life
tozup.com track.deriv.com
trafficmania.com trafficgenius.xyz
tsyndicate.com trion.od.ua
tversvet.ru trustedhealthtips.com
twincitiescarservice.com try-rx.com
twojebook.pl tryrating.com
udayavani.com tt-ipd.info
ukrup.com turkeyreport.tk
ultimateclassicrock.com tv-spoty.info
unitexindia.com uamtrk.com
unitygame3d.com ukkelberg.no
ussearche.cf unpredictable.ga
valueclick.com upupa.net
urccvfmc.bloger.index.hr
uzungil.com
vabasa.inwtrade.com
vadimkravtcov.ru
vavilone.com
vclicks.net
vektorpress.ru vektorpress.ru
veselokloun.ru velobikestock.com
video-chat.love versaut.xxx-cam.webcam
vids18.site via-energy-cumpara.com
video-chat.cn
videosbox.ru
viel.su
vikistars.com
villacoloniale.com villacoloniale.com
vipromoffers.com vinsit.ru
vizag.kharkov.ua vip-parfumeria.ru
vkontaktemusic.ru virtualbb.com
vostoktrade.info visionwell.com.cn
wallpapers-best.com vmnmvzsmn.over-blog.com
watsonrealtycorp.com volume-pills.biz
webads.co.nz vote-up.ru
webix.biz vut.com.ru
webpromotion.ae w3data.co
weblo.com
wechatdownload10.com
wesharepics.com wesharepics.com
whatistizanidine2mg.blogspot.com westen-z.life
whereiskentoday.com westermarkanjou.se
whipme.yopoint.in wladimirpayen.com
whiteelephantwellington.com works.if.ua
wingsofrefuge.net wtsindia.in
winner7777.net x-diesel.org
winterclassichockeyjerseys.com x-lime.net
ww1943.ru xdoza.com
ww2awards.info xep.info
x-diesel.info xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai
xn----9sbubg3ambdfl1j.xn--p1ai xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn--c1acygb.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai xnxx-n.com
xn--b1addnj3cah.xn--p1ai xxxdatinglocal.us
xn--b1agm2d.net xxxrus.org
xxart.ru yes-com.com
xz618.com yoluxuryevents.com
yachts-cruise.info yopoint.in
yourporngay.com you-shall-not-pass.is74.ru
ytmnd.com yoursite.com
zaimhelp.ru zaimhelp.ru
zapiszto.pl zapiszto.pl
zeikopay.com zeg-distribution.com
zhuravlev.info zeroredirect6.com
znakom.sibtest.ru zhorapankratov7.blogspot.com
znakomstva-moskva77.ru zirondelli.it
zocaparj.kz
zoogdisany.com
zooggames.com zooggames.com
zoolubimets.ru zoompegs.com
zootoplist.com zrelaya.pw
zrus.org
zscaler.net
zscalerone.net zscalerone.net
zzbroya.com.ua zvooq.eu
zzlgxh.com

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1001desert.com 000free.us
256bit.by 00webcams.com
2728fb936f0.com 0lovespells0.blogspot.com
34.gs 101billion.com
4istoshop.com 4inn.ru
8lufu.com 7minuteworkout.com
999webdesign.com 8kisses.eu
abcdefh.xyz 8xv8.com
abwa.tk aa9d046aab36af4ff182f097f840430d51.com
aab94f698f36684c5a852a2ef272e031bb.com
aarbur.com
abcguru.xyz
abctoppictures.net
acads.net
activepr.ru
actulite.com actulite.com
adconscious.com
adktrailmap.com adktrailmap.com
adzerg.com allknow.info
adzpower.com allkrim.com
alevinefesleri.com allproblog.com
aliexpresscashback.club android-vsem.org
alkoravto.ru ap.senai.br
allcredits.su arenanews.com.ua
ameblo.jp atleticpharm.org
angel1777.kz autotop.com.ua
angry-fermi-7633.arukascloud.io avirasecureserver.com
arabsexxxtube.com baixar-musicas-gratis.com
arabseyes.com bambi.ck.ua
asrv-a.akamoihd.net bannerconnect.net
autodo.info beachfix.co
avkzarabotok.com
balook.com
bank.uz
bankmib.ru
banksrf.ru
batanga.net
bedcapdealers.com
bestdraws.com bestdraws.com
betteroffers.review beztuberkuleza.ru
betterscooter.com bio-market.kz
bjorkbacken.nu bizru.info
bookmark4you.com blog-hits.com
bouda.kvalitne.cz blog.koorg.ru
business-made-fun.com blogrankers.com
buy-forum.ru bscodecs.com
buyessayonline19.blogspot.ru btc4u.ru
cardsdumps.com bvlgariring.xyz
cartierring.xyz cakesplus.com.au
centraletermice.us canvas.gloverid.site
cheap-pills-norx.com casinopinup-wins.com
cheapjerseysbizwholesale.us ccjp.eu
chelyabinsk.xrus.org cgi2.nintendo.co.jp
children-learningreading.info cheap-trusted-backlinks.com
cleansearch.net cheapbarbour.online
compliance-jessica.top chinese-amezon.com
compliance-olga.top circlesl.com
coolgramgoods.com cjs.com.ru
coprofam.org clips.ua.ac.be
credit.co.ua com-onlinesupport.host
crynet.cc compliance-brian.top
csgo4.win compliance-josh.top
dailyfinancefix.com conciergegroup.org
dailyrank.net confib.ifmo.ru
craftinsta.ru
cranly.net
credit-respect.ru
culturevie.info
darodar.com
date-withme.com date-withme.com
dellalimov.com
dfwu1019.info dfwu1019.info
discountliv.com digilander.libero.it
dktr.ru docsportal.net
documentbase.net domoysshop.ru
dojki-hd.com donna7753191.ru
e-buyeasy.com dvrlists.com
econom.co ecxtracking.com
efnor-ac.com edwinkonijn.com.au
excitacion.info elektrozigarette1.dreamwidth.org
eyes-on-you.ga eloconcream.blogspot.com
ezofest.sk eluxer.net
ff30236ddef1465f88547e760973d70a.com eugenevaultstorage.com
filmi-onlain.info exchangeit.gq
finemanteam.com extlinks.com
fitfloponline.store failingmarriege.blogspot.com
folowsite.com fatmaelgarny.com
formularz-konkurs.tk fidelityfunding.com
foxydeal.com filmi-v.online
frbizlist.com financehint.eu
freejabs.com finuse.com
freelotto.com free-gluten.ru
frequiry.com free-traffic.xyz
front.to galeria-zdjec.com
fun-mobi.pl gay-site.store
galaxy-family.ru gayxperience.com
ge0ip.net gcup.ru
gearcraft.us gelezki.com
germetiki.com.ua geoads.com
ghostvisitor.com gq-catalog.gq
glavtral.ru guardlink.com
goldandcard.ru guitar-master.org
goodhousekeeping.com headpress.ru
googlemare.com hermesbirkinhandbagoutlets.com
gosreg.amchs.ru heygidday.biz
gotwebsite1.com hitsbox.info
gourcy.altervista.org home-task.com
graphid.com hotcore.info
hd-film.pl housediz.com
hermesreplica.pw husky-shop.cz
hikesearch.net i4track.net
hosting-tracker.com imagine-ex.co
huimin764128.com imperia31.ru
ico.re inspiring-desperate.tk
ilmen.net io9.com
infokonkurs.ru iqupdatetmz.win
intellektmedia.at irkutsk.zrus.org
ipchicken.com kakablog.net
istanbulit.com kamagragelusa.net
jrcigars.com karusel-market.ru
kashubadesign.ru kemerovo.zrus.org
keyhantercume.com keywesthideaways.co
kinky-fetishes.com keywordbasket.com
kino-rf.ru keywordhut.com
kongoultry.net kinozapas.org
kosmetyki.tm.pl konpax.com
kriokomora.info kunstaktien.info
kurwa.win law-check-nine.xyz
linkarena.com lexixxx.com
lovasszovetseg.hu louboutinreplica.xyz
love-baby.cz
lowephotos.info
luckyshop.net.ua luckyshop.net.ua
magazin-pics.ru luxmagazine.cf
maglid.ru machicon-ueno.info
medicine-4u.org marcogrup.com
meds-online24.com marmotstore.online
megakino.net medosmotr-ufa.ru
meltwater.com meltwater.com
memberty.com meteostate.com
metabar.ru microsoftportal.net
minharevisao.com midst.eu
misandesign.se mil-stak.com
moje-recenze.cz missclub.info
moneytop.ru mobstarr.com
morf.snn.gr mockupui.com
moxo.com mojaocena.com
musicstock.me monetizationking.net
mybackgroundlandscape.blogspot.com monsterdivx.com
mydirtyhobby.com morpicert.pw
mygameplus.com mp3ringtone.info
myghillie.info myiptest.com
mylesosibirsk.ru na-telefon.biz
narkologiya-penza.ru namecrumilchlet.tk
narkologiya-voronezh.ru netfacet.net
naturtreenspicerx.pw nikitsyringedrivelg.pen.io
niki-mlt.ru nsatc.net
noclegonline.info oakleyglassesonline.us
nofreezingmac.work oda.as
nordstar.pro oddamzadarmo.eu
nottyu.xyz onclickpredictiv.com
oakridgemo.com online-zaymy.ru
obiavo.com openlibrary.org
okeinfo.online orhonit.com
onemactrckr.com ownshop.cf
oneminutesite.it paceform.com
online.ktc45.ru page2rss.com
online247.ml partnerads.men
pattersonsweb.com partnersafe.men
pb-dv.ru petitions.whitehouse.gov
penisvergrotendepillennl.ovh picswe.com
phrcialiled.com pistonclasico.com
picsfair.com pixell.club
pills24h.com playmsn.com
pinwallpaper.xyz poffet.net
pisanieprac.info polska-poezja.com
pod-muzyku.club polytopesexempt.com
porn-w.org pomoc-drogowa.cba.pl
pornosexrolik.com pornoslive.net
pornosmola.info promodj.com
pornotubexxx.name qexyfu.bugs3.com
produkto.net questionmarque.ch
pshare.biz qwertty.net
pskcijdc.bloger.index.hr rcb101.ru
ptr.ruvds.com realresultslist.com
pukaporn.com rednise.com
pussyfleet.com refudiatethissarah.info
putevka24.ru registratciya-v-moskve.ru
pyrodesigns.com.au reklamuss.ru
raavidesigns.com repeatlogo.co.uk
rebrand.ly reversing.cc
redirectingat.com roma-kukareku.livejournal.com
remmling.de
rightenergysolutions.com.au
riralmolamsaca.tk
risparmiocasa.bz.it
ros-ctm.ru
royal-betting.net royal-betting.net
rvzr-a.akamaihd.net rukino.org
sarahmilne.top rumamba.com
sarf3omlat.com runofilms.ru
scanner-viktor.top runtnc.net
search.alot.com rvtv.ru
sexpornotales.com sale-japan.com
sexvporno.ru samoiedo.it
shymkent.xkaz.org santasgift.ml
sildenafilcitratemed.com saugatuck.com
sitiz.club scanner-jane.top
sluganarodu.ru se-welding.ru
snow.nvr163.com seasaltwithfood.com
snowplanes.com segol.tv
socialmonkee.com sexkontakte-seite.com
softwaretrend.net slavia.info
songplanet.ru smadihome.com
sonnikforme.ru social-button.xyz
sptslmtrafms.com sophang8.com
stanthonyscatholicchurch.org space4update.pw
statustroll.com spb.afora.ru
studiofmp.com sports-supplements.us
sugarkun.com stroymonolit.su
sugarlyflex.pw superoboi.com.ua
telefonsex-ohne0900.net svarbit.com
telesvoboda.ru symbaloo.com
thefotosgratis.eu szamponrevita.pl
tizanidinehclsideeffects.blogspot.com tamada69.com
tizanidinevssoma.blogspot.com telegramdownload10.com
tizanidinewithdrawal.blogspot.com telsis.com
tovaroboom.vast.ru thebestphotos.eu
tula.howotorg.ru thebluenoodle.com
twelvevisionspartyofcolorado.com thedownloadfreeonlinegames.blogspot.com
tyumen.xrus.org thegameriders.com
unblocksit.es tizanidinehydrochloride2mg.blogspot.com
usbggettwku.ga tizanidineoral.blogspot.com
valoresito.com toloka.hurtom.com
virus-respirators.com traffixer.com
vkonche.com tripper.de
vuclip.com tsstcorpcddvdwshbbdriverfb.aircus.com
webpromotion.ae tversvet.ru
websites-reviews.com ucsol.ru
weddingdresses.xyz uhdtv.website
wesharepics.com umnovocaminho.com
white-truck.life vapeface.club
wineitudes.wordpress.com vbtracker.net
wjgony.com verymes.xyz
word-vorlagen.net via-energy-acquistare.com
worldis.me vikistars.com
wovis.site visionwell.com.cn
wrona.it watchmygf.net
xclicks.net wedding-salon.net
xedserver.com whatsappbot.flyland.ru
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai witclub.info
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai word-vorlagen.xyz
xn----7sbho2agebbhlivy.xn--p1ai xkaz.org
xxlargepop.com xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
yginekologa.com xtrafficplus.com
yopoint.in xzlive.com
youradexchange.com yebocasino.com
zhorapankratov7.blogspot.com youdesigner.kz
zirondelli.it yourothersite.com
zs2vm.top youtubedownload.org
zskdla.site ypmuseum.ru
zaimite.ru
zakazvzloma.com
zigarettenonlinekaufen2.bloog.pl
zlatnajesen.com
zryydi.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.229 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3444 ### Version: V4.2022.11.3445
### Updated: Mon Oct 31 22:01:16 UTC 2022 ### Updated: Tue Nov 1 22:01:31 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3444 ### Version: V4.2022.11.3445
### Updated: Mon Oct 31 22:01:16 UTC 2022 ### Updated: Tue Nov 1 22:01:31 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##