vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.09.2563 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-09-01 10:09:02 +00:00
parent 97c22d5e59
commit df6937177d
46 changed files with 63213 additions and 63213 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2562 #### Version: V4.2021.09.2563
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2562 #### Version: V4.2021.09.2563
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2562 #### Version: V4.2021.09.2563
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7770
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2562 #### Version: V4.2021.09.2563
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [624](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7774
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7774
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

7774
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7774
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 1; 10.1.0.137 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 0; 10.1.0.137 0;
127.0.0.1 0; 127.0.0.1 0;

7774
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 1; 10.1.0.137 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 0; 10.1.0.137 0;
127.0.0.1 0; 127.0.0.1 0;

7774
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 1; 10.1.0.137 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 0; 10.1.0.137 0;
127.0.0.1 0; 127.0.0.1 0;

7774
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2562 2563

7774
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

7774
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.151.21.234 0; 40.69.159.17 0;

7774
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 1; 10.1.0.137 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 0; 10.1.0.137 0;
127.0.0.1 0; 127.0.0.1 0;

7774
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 1; 10.1.0.137 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 0; 10.1.0.137 0;
127.0.0.1 0; 127.0.0.1 0;

7774
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 1; 10.1.0.137 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 0; 10.1.0.137 0;
127.0.0.1 0; 127.0.0.1 0;

7774
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.151.21.234 0; 40.69.159.17 0;

7774
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.151.21.234 0; 40.69.159.17 0;

7774
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 1; 10.1.0.137 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BatchFTP CODE87
GridBot CSHttp
Harvest DBLBot
Mozlila Mail.RU_Bot
RealDownload NearSite
OnCrawl
RepoMonkey
Screaming Screaming
Snapbot Swiftbot
Steeler Who.is Bot
WebZIP
Whack

288
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404checker 404checker
404enemy 80legs
ADmantX Aboundex
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot AiHitBot
Alexibot AllSubmitter
Alligator AlphaBot
Anthill Anarchie
Aspiegel AwarioRssBot
Attach
AwarioSmartBot AwarioSmartBot
BDFetch BDFetch
BLEXBot
BackStreet BackStreet
Backlink-Ceck
Badass Badass
Barkrowler Bandit
Battleztar Bazinga Bigfoot
Bitacle Bitacle
Black Hole
Blow Blow
BlowFish Bolt
Boardreader BotALot
Buck Buck
Bullseye Bullseye
BuzzSumo CODE87
CATExplorador Cegbfeieh
Calculon CheTeam
CazoodleBot Cliqzbot
CheeseBot
Chlooe
Claritybot
Cloud mapping Cloud mapping
Cogentbot Collector
CopyRightCheck Copier
Crescent Craftbot
CrunchBot Crawling at Home Project
Curious
CyotekWebCopy CyotekWebCopy
DBLBot DBLBot
DTS Agent DIIbot
Deusu DSearch
Devil Demon
Digincore Digincore
Dirbuster Disco
DnyzBot
DomainAppender
DomainSigmaCrawler DomainSigmaCrawler
Domains Project DomainStatsBot
Dotbot Dotbot
Download Wonder Download Wonder
Dragonfly
Drip Drip
ECCP/1.0
EMail Siphon EMail Siphon
EirGrabber EasyDL
Exabot Ebingbong
Extractor EroCrawler
Extreme Picture Finder Express WebPictures
EyeNetIE ExtractorPro
FHscan
Fimap
Firefox/7.0 Firefox/7.0
Flunky
Freeuploader Freeuploader
FyberSpider FrontPage
G-i-g-a-b-o-t Fuzz
Fyrebot
GT::WWW GT::WWW
GalaxyBot GalaxyBot
GermCrawler Genieo
GetWeb GetRight
Getintent
Gigablast
Gigabot Gigabot
Go-Ahead-Got-It
GoZilla
Gotit
Grafula
GrapeFX GrapeFX
HEADMasterSEO HEADMasterSEO
Haansoft HMView
Harvest Harvest
IDBTE4M Humanlinks
IDBot HybridBot
Image Sucker IRLbot
IndeedBot InfoTekies
InfoNaviRobot Intelliseek
InterGET Iskanie
Internet Ninja JOC Web Spider
InternetSeer
JamesBOT JamesBOT
JennyBot
JetCar JetCar
Joomla
JustView
Kenjin Spider
Keyword Density Keyword Density
Larbin Kozmosbot
Leap Leap
LeechFTP LeechFTP
LexiBot
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse Likse
LinkScan
LinkWalker LinkWalker
Linkdexbot Linkbot
LinkpadBot LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Lipperhey MFC_Tear_Sample
Lipperhey Spider
Litemage_walker
Lmspider
Ltx71
MQQBrowser MQQBrowser
MSIECrawler MSIECrawler
MTRobot Magnet
Mag-Net Majestic SEO
Mail.RU_Bot Majestic12
MarkMonitor MauiBot
Mass Downloader
Mb2345Browser
MegaIndex.ru
Metauri Metauri
MicroMessenger
Microsoft URL Control Microsoft URL Control
Mister PiX
Moblie Safari Moblie Safari
Mojolicious
Mozlila
Mr.4x3 Mr.4x3
Nessus Msrabot
NPbot
Name Intelligence
Nameprotect
Needle
NetSpider
NetZIP
Netcraft Netcraft
Nettrack
Nibbler Nibbler
Nimbostratus Nikto
Nuclei NimbleCrawler
Offline Explorer Nmap
OutfoxBot Offline Navigator
PHPCrawl OpenLinkProfiler
PageAnalyzer OrangeBot
PECL::HTTP
POE-Component-Client-HTTP
Page Analyzer
PageScorer PageScorer
Pandalytics Papa Foto
Panscient Pavuk
Petalbot PictureFinder
Pi-Monster
Picscout
Pimonster Pimonster
PleaseCrawl
Pockey Pockey
ProWebWalker ProPowerBot
Psbot Psbot
Pu_iN Pu_iN
PxBroker QueryN Metasearch
PyCurl
Quick-Crawler Quick-Crawler
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot
ReGet RankurBot
RealDownload Re-re
Reaper Reaper
Recorder Recorder
RepoMonkey RepoMonkey
RocketCrawler
Rogerbot Rogerbot
SBIder SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SISTRIX SISTRIX
SMTBot SMTBot
SalesIntelligent
Scanbot
ScoutJet ScoutJet
Screaming
ScrepyBot ScrepyBot
SearchmetricsBot SearchmetricsBot
SemrushBot Semrush
SeobilityBot SeoSiteCheckup
Shodan
SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam Sitebeam
Sitevigil Sitevigil
Snoopy SlySearch
SmartDownload
Snake
SocialRankIOBot SocialRankIOBot
Sosospider Spinn3r
Spammen Steeler
Spbot
SputnikBot
Sucker
Sucuri
SuperBot
Surfbot
SurveyBot SurveyBot
Suzuran Suzuran
T0PHackTeam Swiftbot
T8Abot
Teleport Teleport
TeleportPro TeleportPro
Telesoft
Telesphoreo
Telesphorep
The Intraformant The Intraformant
TheNomad
Thumbor Thumbor
TightTwatBot TightTwatBot
Titan Toata
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Trendictionbot Trendictionbot
Turnitin Turnitin
TurnitinBot
TwengaBot TwengaBot
Typhoeus
URLy Warning URLy Warning
Upflow URLy.Warning
VB Project
Vagabondo Vagabondo
VidibleScraper VidibleScraper
WBSearchBot Virusdie
WEBDAV Voil
WASALive-Bot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
Wallpapers Wallpapers
WallpapersHD WallpapersHD
Web Collage
Web Enhancer Web Enhancer
Web Fuck Web Fetch
Web Sauger Web Sucker
WebAuto WebAuto
WebPix WebCollage
WebCopier
WebFuck
WebReaper
WebStripper WebStripper
WebSucker WebWhacker
WebZIP Webalta
Webshag WebsiteExtractor
Website Quester Webster
Whack Who.is Bot
Widow
WiseGuys Robot
Wonderbot Wonderbot
Xaldon WebSpider Wprecon
ZyBorg ZyBorg
arquivo-web-crawler arquivo-web-crawler
backlink-check cah.io.community
clark-crawler clark-crawler
coccocbot-web
com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j crawler4j
dataforseo.com evc-batch
eCatch facebookscraper
instabid meanpathbot
lwp-trivial mediawords
netEstate NE Crawler
page scorer page scorer
pcBrowser pcBrowser
plumanalytics plumanalytics
s1z.ru probe-image-size
satoristudio.net satoristudio.net
scalaj-http
scan.lol scan.lol
seobility seostar
seoscanners serpstatbot
siteripz sitechecker.pro
sysscan spyfu
tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com trendiction.de
voyagerx.com
x09Mozilla x09Mozilla
x22Mozilla xpymep1.exe
zgrab

262
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 01h4x.com
404checker
80legs
ASPSeek ASPSeek
Abonti
Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren
Alexibot Alexibot
AllSubmitter AlphaBot
Anarchy99 Asterias
Apexoo
Aspiegel
Attach Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot BBBike
BDFetch
BackStreet
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Bandit
Barkrowler Barkrowler
Battleztar Bazinga BatchFTP
BetaBot
Bigfoot Bigfoot
Black Hole Black Hole
BlowFish BlowFish
Bolt Bolt
BotALot
Brandprotect Brandprotect
Buck
Buddy Buddy
Bullseye CATExplorador
BunnySlippers CODE87
CSHttp Calculon
CheTeam
CheeseBot
CherryPicker CherryPicker
Cliqzbot Cloud mapping
Cogentbot Cocolyzebot
Copyscape Copier
Crescent CopyRightCheck
Cosmos
CrunchBot CrunchBot
Custo Curious
DTS Agent
Deusu Deusu
Digincore Digincore
DigitalPebble
Disco Disco
Discobot Discobot
Discoverybot Discoverybot
DittoSpyder Dispatch
DnyzBot DnyzBot
DomCopBot DomainCrawler
DomainSigmaCrawler Download Wonder
DomainStatsBot EMail Siphon
Domains Project EasyDL
Dragonfly EirGrabber
Drip
Ebingbong
Evil Evil
Exabot
Express WebPictures
ExtLinksBot ExtLinksBot
Extractor Ezooms
ExtractorPro
EyeNetIE
FDM
FemtosearchBot
Fimap Fimap
Firefox/7.0 Firefox/7.0
Flunky
Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider
Fyrebot Fyrebot
GT::WWW GT::WWW
Genieo
GermCrawler
GetRight
GetWeb GetWeb
Gigablast Gigabot
Go!Zilla Go-Ahead-Got-It
Gotit Gotit
Grabber
Grafula Grafula
GrapeshotCrawler
HEADMasterSEO HEADMasterSEO
Harvest HTTrack
HybridBot Heritrix
Hloader
Humanlinks
IDBTE4M IDBTE4M
Iblog
Image Sucker Image Sucker
IndeedBot
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek InterGET
Iria InternetSeer
Iskanie Iskanie
Jbrofuzz
JetCar JetCar
Joomla Jetty
Jyxobot Keyword Density
Kenjin Spider Kozmosbot
Kinza
LNSpiderguy LNSpiderguy
Lanshanbot
Larbin Larbin
Leap Leap
LeechFTP Lftp
LibWeb
LieBaoFast LieBaoFast
Likse
LinkextractorPro LinkextractorPro
LinqiaScrapeBot LinkpadBot
LinqiaRSSBot
Lipperhey
Litemage_walker
Ltx71
MQQBrowser MQQBrowser
MSIECrawler Mag-Net
Magnet Magnet
Majestic SEO Majestic SEO
Majestic-SEO
Majestic12
MarkMonitor
MarkWatch MarkWatch
Masscan Mass Downloader
MauiBot MeanPath Bot
Mb2345Browser
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru MicroMessenger
Metauri
Mister PiX
Mojeek Mojeek
Mojolicious Mojolicious
Mozlila Mozlila
Mr.4x3 Mr.4x3
Musobot
NPbot NPbot
Nameprotect
Navroad
NearSite NearSite
Needle NetAnts
Nessus NetLyzer
Net Vampire Netcraft
Nikto Nimbostratus
Ninja Ninja
Nuclei Nuclei
Nutch
Octopus Octopus
Offline Navigator OpenVAS
Openfind
Openvas Openvas
OutclicksBot OutclicksBot
POE-Component-Client-HTTP OutfoxBot
PageScorer PHPCrawl
Page Analyzer
PageAnalyzer
PageThing.com PageThing.com
Panscient Panscient
Papa Foto PeoplePal
Picscout Picscout
Picsearch Picsearch
Pixray Piepmatz
Pimonster
ProPowerBot ProPowerBot
ProWebWalker
Pump
PxBroker
PyCurl PyCurl
Quick-Crawler RankActive
RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankFlex
RankingBot2
Rankivabot Rankivabot
RankurBot
Re-re Re-re
ReGet Ripper
RealDownload SBIder
Reaper SEOkicks
RebelMouse
SMTBot SMTBot
SalesIntelligent
Scanbot
Scrapy Scrapy
ScreenerBot
Searchestate
SentiBot SentiBot
SeoSiteCheckup
Seomoz
Shodan Shodan
SiteExplorer SiteCheckerBotCrawler
SiteSucker
Sitebeam Sitebeam
Siteimprove Siteimprove
SocialRankIOBot SocialRankIOBot
Sottopop Sociscraper
SpankBot Sosospider
Spinn3r Spbot
Sqlmap Sqlmap
Sqlworm Sqlworm
Sqworm Sqworm
Steeler Steeler
Stripper
Sucker Sucker
SuperHTTP Sucuri
Surfbot Surfbot
SurveyBot
Swiftbot
Szukacz Szukacz
Telesphoreo
Telesphorep Telesphorep
The Intraformant
Thumbor Thumbor
Titan Titan
Toweyabot Tracemyfile
Turingos Trendiction
Turnitin
TurnitinBot TurnitinBot
Twice TwengaBot
URLy Warning URLy Warning
URLy.Warning
UnisterBot
VB Project VB Project
Vagabondo VoidEYE
VelenPublicWebCrawler
Voil Voil
Voltron Voltron
WASALive-Bot WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWWOFFLE
Wallpapers/3.0
WallpapersHD
Web Collage Web Collage
Web Fetch Web Fetch
Web Fuck
Web Pix
Web Sucker Web Sucker
WebCollage WebBandit
WebEnhancer
WebFuck WebFuck
WebLeacher WebReaper
WebPix WebSauger
WebStripper
WebWhacker
WebZIP WebZIP
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Website Quester Website Quester
Webster WebsiteExtractor
WebsiteQuester
Whacker Whacker
Widow Wonderbot
WiseGuys Robot
Woobot Woobot
Wprecon Zauba
Zermelo
Zitebot Zitebot
ZmEu
ZoomBot ZoomBot
ZyBorg ZyBorg
archive.org_bot archive.org_bot
autoemailspider arquivo.pt
backlink-check
cah.io.community cah.io.community
clark-crawler check1.exe
cognitiveseo
com.plumanalytics com.plumanalytics
crawler4j crawler4j
dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org
eCatch
evc-batch evc-batch
gopher gopher
ips-agent heritrix
internetVista monitor
isitwp.com isitwp.com
magpie-crawler
meanpathbot meanpathbot
mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
pcBrowser pcBrowser
polaris version plumanalytics
s1z.ru probe-image-size
scalaj-http scalaj-http
seobility
seoscanners seoscanners
seostar serpstatbot
sexsearcher spyfu
sogouspider
sp_auditbot
tAkeOut
trendiction.com trendiction.com
voyagerx.com ubermetrics-technologies.com
webpros.com
webprosbot
x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
afmuseum.com 5elementov.ru
autodo.info checktext.org
cookie-law-enforcement-aa.xyz christianlouboutinshoes.xyz
efnor-ac.com depositfiles-porn.ga
honyaku.yahoofs.jp div.as
linerdrilling.com ge0ip.net
novgorod.xrus.org nisuturnetdgu.tk
onlyporno.ru notebook-pro.ru
td-33.ru seemoreresultshu-a.akamaihd.net
webinstantservice.com yourporngay.com

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 12bet.com
01apple.com 14b.info
123cha.com 1stat.ru
12u.info 2nt.ru
18ps.ru 3weekdiet.com
1qingdao.com 3xforum.ro
273-fz.ru 4kepics.com
28n2gl3wfyb0.ru
45en.ru
4pp13.com 4pp13.com
4webmasters.com aa9bd78f328a6a41279d0fad0a88df1901.com
72-news.com abcdeg.xyz
acgs.tk academia-nsk.org
adelly.bg acads.net
adultactioncam.com adcash.com
advocatemsk.ru adloads.net
airmaxshoes-2016.com adminshop.com
adtech.de
advokateg.xyz
afmuseum.com
afora.ru
akama.com
akita.kz
aksonural.ru
aktivator-windows10.blogspot.com
albamargroup.com albamargroup.com
albuteroli.com alessandraleone.com
alfa-img.com all-streaming-media.com
ameblo.top all4invest.info
animal36.com alloysteel.ru
apimountainbikei-a.akamaihd.net aludecor.info
apiqualitinknet-a.akamaihd.net anal-acrobats.hol.es
apishops.ru anatomiy.com
aosexkontakte.net
appfastplay.com
appfixing.space
applicationg29.com
appmsr.org
approvedlocal.co.za approvedlocal.co.za
arvut.org arabgirls.adultgalls.com
asacopaco.tk arewater.com
atleticpharm.org aruplighting.com
atmagroup.ru asia-forum.ru
auspolice.com asrvvv-a.akamaihd.net
availit.weebly.com asseenontv.ru
aviav.ru.com atley.eu.pn
avon-ukraine.com automatic-seo.com
awency.com autorn.ru
azartclub.org aviva-limoux.com
azartniy-bonus.com avtointeres.ru
azazaporn.com babyfactory.fr
azbuka-mo.ru
babs.com.ua
bagsonsale.online bagsonsale.online
balois.worldbreak.com balook.com
bamo.xsl.pt bausparen.bz.it
bank.uz bear.gotcher.us
banksrf.ru
baoxaydung.com.vn
bayareaaeroclub.org
beachpics.com
beget.tech beget.tech
beta.hotkeys.com best-coupon-offer.com
betterscooter.com best-way.men
betwinservice.com bestmarriages.com
beyan.host.sk bestofpicture.com
billiard-classic.com.ua bestwaystogetpaid.us
birzha-truda.eu bidr.trellian.com
bitcoinremote.com big-boards.info
bizzliving.com bimatoprost-careprost.com
bjgugu.net.ua binomo.com
blackbabesporn.com bjetjt.com
blog.yam.com blogsfunky672.weebly.com
bloglag.com bluesman.nu
blpmovies.com bmw.afora.ru
bonkers.name boom-celebs.com
bonvillan.com brandov.ru
boo-arts.com breastaugmentation.co.za
braip.com.br bronzeaid-a.akamaihd.net
bravegirlsclub.com buqayy0.livejournal.com
builtwith.com
bupropion-sr-150-mg.us
burger-tycoon.com
buron.pw buron.pw
buybest1.biz buyhoverboard.com
buzzzg.men bvlgaribracelet.xyz
callcafe.info callawaygolfoutlet.online
cand.jp candcstuccoandstone.com
candlewooddentalcentre.com.au
canoncdriverq3.pen.io canoncdriverq3.pen.io
canyougethighofftizanidine.blogspot.com cartierlovestore.xyz
carapics.com cartiertrinity.xyz
cartechnic.ru cenoval.ru
cartierlove.xyz centraletermice.us
casite-513049.cloudaccess.net cete.ru
ccbill.com cheap-pills-norx.com
cellfun.mobi cheapjerseysbizwholesale.us
cherrypointplace.ca check-this-out-now.online
cinemaenergy-hd.ru checkm8.com
cityadspix.com chimiver.info
clean-virus-mac.com cialischsrx.com
clevernt.com cjs.com.ru
clickcash.com click2pawn.com
clicksotrk.com clickbank.net
coinsspb.com cocolyze.com
comeondog.info collegerentals.com
communisave.co.za compliance-alex.top
compliance-elena.xyz compliance-alexa.xyz
computernetworksonline.com compliance-fred.xyz
compliance-olga.top
compliance-willy.top
comsysnet.com comsysnet.com
connectionstrenth.com cookingmeat.ru
cookie-law-enforcement-ff.xyz cool-wedding.net
costablue.xyz coolgramgoods.com
customcatchcan.com copro.pw
cv.wallhade.co cornomase.win
cyclobenzaprinevstizanidine.blogspot.com country-chic.ru
daymusam.com crazyprotocol.com
deutschland123.de cubs-jerseys.us
diffbot.com customboxes4less.com
dimfour.com cvety24.by
divci-hry.info d0t.ru
djstools.com dailyfinancefix.com
dnsrsearch.com darmebel.com.ua
docstoc.com davidovic.info
domainanalyzing.xyz ddpills.com
domainroam.win dealitright.click
download-walpaperhd.blogspot.com deda-moroza-zakaz.ru
downloaddy.net dengi-pod-zalog-nedvizhimosti.ru
drugstoreforyou.com detskie-konstruktory.ru
dyt.net dfiles.me
dirtpics.pw
djekxa.ru
dms-sw.ru
dneprsvet.com.ua
doeco.ru
dojki365.online
domaincdn.xyz
download-of-the-warez.blogspot.com
downloadeer.net
downtuptv.gq
doxyporno.com
drillsaw.ru
drugspowerstore.com
drunkenstepfather.com
drunkmoms.net
dwomlink.info
eandsgallery.com eandsgallery.com
ecommerce-seo.com earl-brown.info
eduinfosite.com eleimgo.pw
efkt.jp elektrischezigaretten1.blogse.nl
ekn-art.se elektrozigaretten1.postbit.com
el-nation.com
elitedollars.com
elmacho.xyz elmacho.xyz
energydiet-info.ru eloxal.ru
eorogo.top empathica.com
erectile.bid enews.tech
essay-writing.work epngo.bz
estibot.com eshop4u.jp
europages.com.ru expolicenciaslatam.co
euroskat.ru extremal-blog.com
evrotekhservis.ru ezigarettenkaufen1.pagina.gr
executehosting.com fanrto.com
expdom.com
extstat.com
ez8motelseaworldsandiego.com
ezigarettenshop2.myblog.de
fandlr.com
fashion-mk.net
fashion-stickers.ru fashion-stickers.ru
fbdownloader.com faz99.com
fickenbumsen.net felizporno.com
film-one.ru fidelityfunding.com
financehint.eu financeloan.us
flirt4free.com findacheaplawyers.com
floridahuntingfishingadventures.com fireads.men
folowsite.com fix-website-errors.com
forum.doctissimo.fr flavors.me
frbizlist.com flipper.top
free-deals.faith foodcrafts.website
free-stock-illustration.com footbalive.org
freemags.cc fotosfotos.eu
freewebs.com franklinfire.co
freewlan.info freemaintenancesysforpcandmac.top
freshberry.com.ua freenode.info
ftns.ru futbolkisales.ru
fuckingawesome.com gambarkataku.co
fun-mobi.pl gardene.ru
funnypica.com gearcraft.us
fxgallery.com geckoandfly.com
gamebackyard.com generalporn.org
gaygalls.net germetiki.com.ua
geile-lelly.eu getpopunder.com
get-seo-domain.com gettpromos.com
getprismatic.com ghazel.ru
gidonline.one gkvector.ru
girlgamerdaily.com glastecfilms.com.my
gktt.ru global-ics.co.za
go2mike.ru go2mike.ru
golaya.pw gobongo.info
goldandcard.ru
goodhumor24.com goodhumor24.com
gopro-online.info googst2.ru
graphicwe.org goosefishpost.bid
graphid.com gotorussia.com
gourcy.altervista.org
greamimgo.pw greamimgo.pw
groupmoney.ru greatdealshop.com
gstatey.net grizzlysgrill.com
guiadeserraazul.com gsasearchengineranker.top
gwhwpxbw.bloger.index.hr gsasearchengineranker.xyz
handicapbathtubarea.com gsmtlf.ru
handicapvansarea.com gwebtools.com
hannasolution.ru hanink.biz.ly
hauleddes.com hazardky.net
hd720kino.ru help.tpu.ru
helvetia.com.ua herehloadibs.cf
hentaiheroes.com historichometeam.com
hermesreplica.pw hit-men.men
herokuapp.com hornymatches.com
hobbyhuren24.net horoshieokna.com
homelygarden.com hotelcrocenzi.sm
hotenergy.ru hotloans.ru
hotsocialz.com housemilan.ru
houseofrose.com hundejo.com
howtoclean.club i4track.net
htmlcorner.com igithab.com
humanelydrew.com infodocsportal.com
hunterboots.online inome.com.ua
hybrid.ru
ico.re
igtools.club
ihc2015.info
imgarit.pw
immigrational.info
imperia31.ru
impisr.edunsk.ru
inboundlinks.win
intellektmedia.at
interferencer.ru
interior-stickers.ru interior-stickers.ru
intermesh.net investsuccess.org
irunfar.com investyb.com
iskalko.ru iqoption.pro
iweblist.info irkutsk.online-podarki.com
j33x.com istizanidineacontrolledsubstance.blogspot.com
javatex.co.id janavibekken.no
javspace.net javrip.net
je7.us joessmogtestonly.com
jikoman.info
johnnyhaley.top johnnyhaley.top
joingames.org juliadiets.com
kam-dom.ru kalb.ru
kanimage.com kambasoft.com
kaz.kz kamorel.com
ketoanhanoi.info keywordglobal.co.za
kino-key.info kiprinform.com
kinoduh.ru kirov.zrus.org
kladrus.ru komputernaya-pomosh-moscow.ru
kleine-titten.biz konica.kz
knigonosha.net
knowyournextmove.com
kollesa.ru
kolotiloff.ru
kongoultry.net
koronirealestate.gr
kostenlos-sexvideos.com kostenlos-sexvideos.com
kovesszucs.atw.hu krafte.ru
kovrenok.ru
krasivye-devushki.net
krassh.ru krassh.ru
labvis.host.sk ks1234.com
laparfumotec.com kvartiry-remont.ucoz.ru
larose.jb2c.me la-fa.ru
letmacwork.world labelwater.se
lexaprogeneric.link ladsup.com
liffa.ru lafourmiliaire.com
liky.co.ua landinez.co
linkbolic.com landreferat.ru
linkwithin.com lapitec.eu
lion.playtap.us lapolis.it
lion.snapmap.us larchik.net
liveinternet.ro ledpolice.ru
liveinternet.ru letolove.ru
livesex-amateure.info letslowbefast.today
liveshoppersmac.com likesuccess.com
lkbennettstore.online linkrr.com
llastbuy.ru liquimondo.com
localflirtbuddies.com live-cam6.info
louboutinreplica.pw lmrauction.com
love-baby.cz lonely-mature.com
luchshie-topcasino.ru m1media.net
luckyday.world magazin-pics.ru
lulea-auktionsverk.se magda-gadalka.ru
lutherstable.org managerpak204.weebly.com
luxup.ru marketingtechniques.info
m3gadownload.pl martlinker.com
marktforschung-stuttgart.com maslenka.kz
mebelcomplekt.ru master-muznachas.ru
medanestesia.ru mcadamssupplyco.com
medictube.ru medicines-choice.com
meetingrainstorm.bid medkritika.ru
meltwater.com mega-bony2017.pl
member-quiz.com member-quiz.com
menstennisforums.com metarip.ru
metalonly.info metascephe.com
methodsmarketing.com michellblog.online
mexicosleevegastrectomy.com mini-modus.ru
michaelkorssaleoutletonline.net mirzonru.net
mil-stak.com mmgq.ru
millionare.com moesonce.com
mini.7zap.com mokrayakiska.com
miniads.ca moneyzzz.ru
mir-limuzinov.ru moviemail-online.co.uk
missvietnam.org moyaterapiya.ru
mobifunapp.weebly.com msk-diplomat.com
moivestiy.biz mttwtrack.com
monclercoats.xyz
monetizationking.net
montazhnic.ru
mrpornogratis.xxx
musicpro.monster musicpro.monster
muz-baza.net
mvpicton.co.uk mvpicton.co.uk
myblogregistercm.tk
mycaf.it
mydoctorok.ru mydoctorok.ru
myliveblog.ru myfreetutorials.com
mystats.xyz mypornfree.ru
myyour.eu myyour.eu
na-telefon.biz naj-filmy24.pl
nakrutka.cc nash-krym.info
narco24.me naughtyconnect.com
nationalbreakdown.com nevansk.ru
neobux-bg.info niki-mlt.ru
netfacet.net nikitabuch.com
new-apps.ru ningessaybe.me
newstudio.tv nodding-passion.tk
niroo.info nordstar.pro
nufaq.com nudo.ca
odesproperty.com obsessionphrases.com
odoratus.net offer.wpsecurity.website
odywpjtw.bloger.index.hr oil-td.ru
ofanda.com onclkads.com
okout.ru online247.ml
onblastblog.online onlyporno.ru
onemantrip.com ooomeru.ru
oneminutesite.it oops-cinema.ru
onstrapon.purplesphere.in openstat.com
ordernorxx.com optibuymac.com
orgasmus-virtual.com ovirus.ru
outshop.ru paleohub.info
ownshop.win panchro.xyz
pandarastore.top
panouri-solare-acoperis.com
par-fallen.ga
paramountmarble.co.uk paramountmarble.co.uk
parfusale.se
partnerads.men partnerads.men
pattersonsweb.com partnersafe.men
pcimforum.com partybunny.ru
pelfind.me pawli.eu
pepperstyle.ru perfection-pleasure.ru
phrcialiled.com petitions.whitehouse.gov
pic2fly.com picmoonco.pw
pics-group.com pics-group.com
pijoto.net pills24h.com
pl-top.pl pipki.r.acdnpro.com
playlott.com pistonclasico.com
popcash.net pix-hd.com
poppen-nw.net pix24x7.com
popunder.ru placid-rounded-coast.glitch.me
porn-w.org plastgran.com
pornofeuer.com play-movie.pl
pornotubexxx.name polcin.de
portadd.men popup-fdm.xyz
portnoff.od.ua pornhub-forum.ga
predominant-invent.tk pornohub.me
privacyassistant.net pornowarp.info
privhosting.com portal-eu.ru
pro-poly.ru pozdrawleniya.ru
producm.ru pretty-mart.com
professionalwritingservices15.blogspot.ru printdirectforless.com
profitkode.com privetsochi.ru
prom23.ru prlog.ru
promover.org pussyspace.net
prostitutki-belgoroda.org pwwysydh.com
prostitutki-odessa.org rebrand.ly
pshare.biz regionshop.biz
pskcijdc.bloger.index.hr registratciya-v-moskve.ru
pulse33.ru registrydomainservices.com
quickbuck.com remmling.de
radiodigital.co remybutler.fr
randki-sex.com restorator-msk.ru
ranksignals.com revistaindustria.com
rapidokbrain.com rff-cfal.info
rating-casino2021.ru robertefuller.com
realtytimes.com royalcar-ufa.ru
rebelmouse.com roznica.com.ua
redhotfreebies.co.uk
rednise.com
reklamuss.ru
rethinkwasteni.info
rfserial.net
rialp.getenjoyment.net
rn-to-bsn.com
robot-forex.biz
rocis.site
royal-investments.net
rozalli.com
rp9.ru
running-line.ru running-line.ru
ruscopybook.com russkoe-zdorovie.ru
russia-tao.ru sadaholding.com
russian-postindex.ru samolet.fr
sa-live.com savefrom.com
sammlungfotos.online
sanjosestartups.com
sayyoethe.blogspot.co.za
sbtdesign.co.uk
scanner-alex.top scanner-alex.top
scrapinghub.com scanner-julianna.top
seeresultshub-a.akamaihd.net scanner-olga.top
seoanalyses.com searchaddis.com
seocheckupx.com searchengineranker.email
seoexperimenty.ru searchimage.co
seokicks.de searchmywindow-a.akamaihd.net
service-core.ru sei80.com
sex-sex-sex5.com selectads.men
semalt.com
semxiu.com
seobility.net
sex-dating.co
sex-foto.pw
sex.hotblog.top
sexpornotales.net
sextracker.be
sexuria.net sexuria.net
sharebutton.to sexvporno.ru
shemalegalls.blogporn.in sherlock.se
sierraapps.com shopcheermakeup.info
sigmund-freud.co.uk simon3.ru
similardeals.net sims-sims.ru
site3.free-share-buttons.com site-speed-checker.site
siteexpress.co.il site5.com
slomm.ru skylta.com
sladkoevideo.com
slavkokacunko.de
slayerlife.com
slowmac.tech
slutloadlive.com
smallseotools.com smallseotools.com
snip.to snjatie-geroinovoy-lomki.ru
snomer1.ru sockshare.net
soblaznu.net solnplast.ru
socialmonkee.com songoo.wz.cz
socialseet.ru space2019.top
sosdepotdebilan.com spaceshipad.com
spb.afora.ru spbchampionat.ru
spin2016.cf sport-video-obzor.ru
spy-sts.com srdrvp.com
srgwebmail.nl stockquotes.wooeb.com
stairliftsarea.com stop-zavisimost.com
starpages.net strana-krasoty.ru
staynplay.net stroyhelp-dv.ru
steamoff.net supermesta.ru
sticken.co
stickers-market.ru
stromerrealty.com
sundrugstore.com
superfish.com
svarkagid.com svarkagid.com
svbur.ru
swagbucks.com
swinon.site
t-bygg.com t-bygg.com
tds-advert005.info tacbibirfa.tk
technika-remont.ru takethatad.com
technopellet.gr tattooha.com
telefonsexkostenlos.tk tengohydar.tk
telegraf.by terrafootwear.us
thedownloadfreeonlinegames.blogspot.com
theendivechronicles.com theendivechronicles.com
thefarmergame.com
thefds.net
thenews-today.info
thesmartsearch.net
thewebsitetemplate.info thewebsitetemplate.info
tizanidine6mg.blogspot.com thewomenlife.com
tizanidinedruginteractions.blogspot.com titan-cloud.life
tizanidinedrugtest.blogspot.com tivolibasket.it
tizanidinehydrochloride2mg.blogspot.com tizanidineandgabapentin.blogspot.com
tizanidineinfo.blogspot.com tizanidinecost.blogspot.com
tomatis.gospartner.com tizanidinehcl4mgtabinfo.blogspot.com
toondinsey.com tizanidineorflexeril.blogspot.com
tizanidinewithdrawal.blogspot.com
tk-assortiment.ru
todohr.com
toonfamilies.net
top10-online-games.com top10-online-games.com
topsiteminecraft.com top10-way.com
tracksurf.daooda.com top250movies.ru
tradgardspartner.se topcar-krasnodar.ru
trafaret74.ru topquality.cf
traffic2cash.org traffic2cash.org
tramadolandtizanidine.blogspot.com traffpartners.com
trubywriting.com trichizobswiv.agddns.net
truck-land.life truck-rece.life
tubeline.biz trustl.life
tula.howotorg.ru ttsq.fr
tupper-posuda.ru tuckermktg.com
tversvet.ru tula.mdverey.ru
u-cheats.ru turbabitload.weebly.com
ugguk.online twelvevisionspartyofcolorado.com
unionmarkt.de twinderbella.com
twu.com.ua
uasb.ru
uchil.net
unlimitdocs.net
upstore.me
url-img.link
urldelivery.com
uroki.net uroki.net
uytmaster.ru userequip.com
v24s.net valsalud.com
varbergsvind.se valueclick.com
vektorpress.ru veselokloun.ru
via-energy-acquistare.com videosbox.ru
via-energy-cumpara.com viel.su
villacoloniale.com vladimir.xrus.org
visionwell.com.cn
voloomoney.com voloomoney.com
vykup-avto-krasnodar.ru vsesubwaysurfers.com
w-journal.ru vvpg.ru
watchinf.com walkme.com
wechatdownload10.com wallpapersdesk.info
westsextube.com warezaccess.com
wholesalenfljerseys.us.com warningwar.ru
wholinkstome.com webjam.com
win-spy.com wesharepics.info
wnhjavlhezp.gq windowssearch-exp.com
woman-tampon.ru wineitudes.wordpress.com
works.if.ua winwotgold.pl
worldhistory.biz workle.website
wosik-dach.service-for-web.de wygraj-teraz.com
wowas31.ucoz.ru
wowcasinoonline.ooo
x-diesel.com x-diesel.com
xfluro.com xerox-douglas.cf
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xn----7sbho2agebbhlivy.xn--p1ai
xn--80abgj3a5acid6ghs.top xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn--80ahvj9e.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xnxx-n.com xn--80ak6aa92e.com
xsion.net xsfetish.org
xvideosbay.com xwatt.ru
xxxasianporn.net xxxguitars.com
yougotanewdomain.com yellocloud.be
za-fun-offer.com yes-com.com
zaidia.xhost.ro yoshkarola.zrus.org
zootravel.com youdao.com
zvuker.net yoursite.com
yubikk.info
yugo-star.ru
zajm-pod-zalog-nedvizhimosti.ru
zarenica.net
zdesformula.ru
zeroredirect12.com
znakomstva-piter78.ru
zolotoy-lis.ru
zootoplist.com
zryydi.com
zs2vm.top
zskdla.site

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
03p.info 1000n1.ru
101billion.com 3w24.com
108shot.com 7fon.ru
123locker.com aac500b7a15b2646968f6bd8c6305869d7.com
1pamm.ru albamargroup.com
24videos.tv allforminecraft.ru
3weekdiet.com allsilver925.co.il
7search.com anaseracresar.tk
aad1f4acb0a373420d9b0c4202d38d94fa.com anlimebel.kiev.ua
activecampaign.dreamhosters.com arendatora.ru
adanih.com artavenuegardenstudios.com
adclickthru.net artdeko.info
albumsuper.info autonew.biz
algarveglobal.com avitocars.ru
all4invest.info avon-novosib.ru
allkrim.com bablonow.ru
anapa-inns.ru best-coupon-offer.com
android-style.com bigcareer.info
apibetweenlinesn-a.akamaihd.net bitporno.sx
appleid-verification.com bizcheapjerseyswholesalechina.com
aquarium-pro.ru bizfly.info
aussie-prizes.com blacktwhite.com
autodo.info bobba.dzaba.com
availit.weebly.com bonkers.name
avek.ru brandbucket.com
azartmix.com brus-vsem.ru
babespcs.com burger-tycoon.com
balook.com buydissertation.net
bangla-chat-uk-paralud.ga buyparajumpers.online
banksrf.ru buypillsorderonline.com
batanga.net callawaygolfoutlet.online
beatifullandscape.co canrole.com
betteroffers.review
bird1.ru
bitcoin-ua.top
blackbabesporn.com
blog.koorg.ru
bochemit.com.ua
bonux.nextview.ru
brains2.biz
bscodecs.com
budilneg.xyz
buron.pw
buyessay3.blogspot.ru
buyfriend.ru
buzzzg.men
callmd5map.com
capsderfudd.tk capsderfudd.tk
caulderwoodkennels.com carfax.com.ua
cdncash.net carrauterie.be
cfjump.com cartierlove2u.com
cleanmypc.com caveavins.fr
clevernt.com citysecurity.nu
com-supportcenter.website cjmc.info
compliance-mary.top classicakuhni.ru
confib.ifmo.ru classiquebijoux.ru
cookingmeat.ru coindirect.io
copenergo.ru com-onlinesupport.host
copypaste.traderzplanet.in compliance-alexa.xyz
cpajunkies.com contentlook.co
creativehutindia.com copblock.org
creditwell.ru counter.bloke.com
customcollegeessays.net cubook.supernew.org
dbmkfhqk.bloger.index.hr custom-electric-guitar.com
delta-line.men
demian.kz demian.kz
djonwatch.ru designdevise.com
dokfilms.net detskie-zabavi.ru
dostavka-v-krym.com detsky-nabytek.info
drillsaw.ru diarioaconcagua.com
duplicashapp.com dikqlyremy.info
e2click.com dolohen.com
easycommerce.cf domainsphoto.com
empathica.com dorratex.tn
enthuse.computernetworksonline.com dumuelave.xyz
erotik-kostenlos.net dvr.biz.ua
eupornstar.info dwomlink.info
excaliburfilms.com earl-brown.info
exoclick.com ecookna.com.ua
extlinks.com eldiariodeguadalajara.com
ezigarettenkaufen1.deviantart.com elektrischezigarettekaufen2.cowblog.fr
femdom.twiclub.in encodable.com
fillmewithhappiness.com englate.com
flooringinstallation-edmonton.com etotupo.ru
forum-engineering.ru europages.com.ru
fotzen-ficken.com everypony.ru
freecamdollars.com excitacao.com
freelotto.com extener.org
gambarxkata.co familyholiday.ml
game-top.su fashionavenuegame.com
gaming-journal.com finteks.ru
gembird.com fortunejack.com
ggiaro.com forum.tvmir.org
glavtral.ru fuck-paid-share-buttons.xyz
goodwriterssales.com fuckbuddybestgilf.info
googlsucks.com fuckingawesome.com
gsasearchenginerankerdiscount.com funponsel.com
hawaiisurf.com fx-brokers-review.com
hdpixent.com fym.com.ru
hollywoodactress.info gagrasector.ru
hostnow.men gg.zzyjxs.com
hotloans.ru glcomputers.ru
hoverboard360.de globalscam.ga
hyiphunter.org gne8.com
iboss.com goldandcard.ru
increasewwwtraffic.info googglet.com
infowarcraft.ru googst2.ru
inmate-locator.us graphid.com
internetproviderstucson.com greatdealshop.com
intim-uslugi.info herokuapp.com
ip-guide.com hit-men.men
isoveti.ru homegardenlova.com
jabimgo.pw housekuba.org
justbcause.com hplaserjetpdriver8y.pen.io
kazka.ru ico.re
krasnodar-avtolombards.ru ilovevitaly.com
imgata.com
iqupdatetmz.win
islandminingsupply.wordpress.com
isotoner.com
juliaworld.net
jwcialislrt.com
kakablog.net
karachev-city.ru
kemerovo.zrus.org
klin3952.ru
kochanelli.com
kotaku.com
krafte.ru
krasivye-devushki.net
kuzinsp.ru kuzinsp.ru
lacave.ntic.fr
lampokrat.ws lampokrat.ws
lazy-z.com lanadelreyfans.us
landscaping.center
laptop-4-less.com
larutti.ru
laserpen.club
libertybilisim.com libertybilisim.com
listiseltemournlan.gq life-instyle.com
liumimgx.pw lion.bolegapakistan.com
liveshoppersmac.com lkbennettoutlet.online
liveu.infoteka.hu lollypopgaming.com
lodki-pvh.dp.ua love-baby.cz
losangeles-ads.com macrotek.ru
lostaruban.com mcar.in.ua
lowephotos.info medictube.ru
luxmagazine.cf meetlocalchicks.com
m.b00kmarks.com megavolt.net.ua
mafcards.ru misterjtbarbers.com
managerpak204.weebly.com mixx.com
manve.info mobicover.com.ua
megagrabber.ru monclerjacketsoutlet.pw
memberty.com monclervests.xyz
metallosajding.ru moscow.online-podarki.com
mikozstop.com mrlmedia.net
mojaocena.com mrmoneymustache.com
monarchfind-a.akamaihd.net multgo.ru
moonci.ru mydearest.co
most-kerch.org mydeathspace.com
mp3downloadhq.com myfreemp3.eu
mydownlodablefiles.com mygameplus.ru
myphotopipe.com nerudlogistik.ru
myshopmatemac.com npoet.ru
netvouz.com nudo.ca
nippon-bearings.ru okuos.com
nootrino.com online-sbank.ru
offer.camp opinionreelle.com
okmusic.jp paighambot.com
okout.ru painting-planet.com
oneshotdate.com partner-high.men
openlibrary.org pelfind.me
paradontozanet.ru photo.houseofgaga.ru
partner-pop.men picturesify.com
peekyou.com piulatte.cz
pensplan4u.com pornhub-ru.com
perederni.net porno-chaman.info
phelissota.xyz porno-transsexuals.ru
php-market.ru pornowarp.info
pinstake.com pourvous.info
pirateday.ru privatamateure.com
piuminiita.com privhosting.com
pizdopletka.club progress-upakovka.ru
pizza-imperia.com
placid-rounded-coast.glitch.me
popunder.net
pornofiljmi.com
porto.abuilder.net
pr-ten.de
preg.marketingvici.com
priceg.com
prombudpostach.com.ua prombudpostach.com.ua
puteshestvennik.com promoheads.com
rankings-analytics.com pronorm.fr
ranksonic.org propranolol40mg.blogspot.com
remontgruzovik.ru pshare.biz
rethinkwasteni.info qcstrtvt.bloger.index.hr
rff-cfal.info quickloanbank.com
robotixix.com rarbg.to
scmor.ilxc.cc rating-bestcasino.com
scoopquest.com razamicroelectronics.com
sdi-pme.com realmonte.net
searchimage.co realnye-otzyvy.info
secret.xn--oogle-wmc.com recipedays.com
servingnotice.com registrysweeper.com
sextracker.de restorator-msk.ru
sexy.babes.frontend-stack.top reversing.cc
shodanhq.com risparmiocasa.bz.it
slavkokacunko.de rubanners.com
sluganarodu.ru ryetaw.com
socialsignals24.com sadaholding.com
sptslmtrafms.com sale-japan.com
ssconstruction.co savememoney.co.za
streetfire.net scanner-brian.top
strv.se seansonline24.pl
suhanpacktech.com secretscook.ru
supermodni.com.ua shmetall.com.ua
superoboi.com.ua shtaketniki.kz
superstarfloraluk.com sierraapps.com
swingerseiten.com simple-share-buttons.com
syvertsen-da.no sisi-go.ru
tagil.zrus.org skanninge.se
telesvoboda.ru skuteczna-dieta.co.pl
tizanidinehcl4mgisitanarcotic.blogspot.com socas.pluto.ro
tizanidinehydrochloride2mg.blogspot.com social-vestnik.ru
toy-shop.top sribno.net
trumpetedextremes.com stmassage.ru
trustedmaccleaner.com stockquotes.wooeb.com
twelvevisionspartyofcolorado.com strana-krasoty.ru
stroiminsk.org
studentguide.ru
sudexpert66.ru
swagbucks.com
sweepstakes.rewardit.com
swiped.su
technopellet.gr
teesdaleflyballclub.co.uk
thefds.net
tivolibasket.it
tizanidineformigraines.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
trafficfactory.biz
trafficjunky.com
try-rx.com
tubeoffline.com
u-cheats.ru
ultimateclassicrock.com ultimateclassicrock.com
uzungil.com ultimatesetnewfreeallsoftupgradesystems.pw
uni.me
unitygame3d.com
v-doc.co v-doc.co
vbtracker.net v24s.net
videochat.mx vapeface.club
videonsk.com vchulkax.com
vip-dom.in veopornogratis.xxx
vipromoffers.com videochat.ph
visa-pasport.ru webkeyit.com
visionwell.com.cn webscouter.net
vkmusics.ru wejdz-tu.pl
voloslove.ru whiteproduct.com
vsdshnik.com wholinkstome.com
wait3sec.org workle.website
wallpapersdesk.info x-lime.com
warezkeeper.com xaylapdiendanang.com
web-analytics.date xingzi-vision.com
weburok.com xn--80abgj3a5acid6ghs.top
welck.octopis.com xn--80ahdheogk5l.xn--p1ai
woman-orgasm.ru youbloodyripper.com
wordkeyhelper.com youradexchange.com
www.ehscloud.cn yourads.website
www.jbetting.com youtoner.it
xmronta.com zacreditom.ru
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai zarepta.com
xn----ctbigni3aj4h.xn--p1ai zeroredirect6.com
xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--80aafb2a.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
xn--b1addnj3cah.xn--p1ai
xxxmania.top
xz618.com
yaaknaa.info
yoursearch.me
yugk.net
zetmaster.ru
zhongwenlink.com
znakomstvaonlain.ru
zvetki.ru
zx6.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.211 0; 10.1.0.137 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2562 ### Version: V4.2021.09.2563
### Updated: Tue Aug 31 22:01:15 UTC 2021 ### Updated: Wed Sep 1 10:03:11 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2562 ### Version: V4.2021.09.2563
### Updated: Tue Aug 31 22:01:15 UTC 2021 ### Updated: Wed Sep 1 10:03:11 UTC 2021
### Bad Bot Count: 624 ### Bad Bot Count: 624
################################################### ###################################################
### Version Information ## ### Version Information ##