vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.08.2538 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-08-20 10:09:02 +00:00
parent 856f43aaa0
commit f41ae4ba57
46 changed files with 66125 additions and 66125 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2537 #### Version: V4.2021.08.2538
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2537 #### Version: V4.2021.08.2538
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2537 #### Version: V4.2021.08.2538
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8132
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2537 #### Version: V4.2021.08.2538
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

8136
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8136
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

8136
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

8136
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

8136
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

8136
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

8136
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2537 2538

8136
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

8136
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.234.178.255 0; 52.191.11.132 0;

8136
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

8136
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

8136
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

8136
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.234.178.255 0; 52.191.11.132 0;

8136
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.234.178.255 0; 52.191.11.132 0;

8136
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 1; 10.1.0.176 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Barkrowler MSFrontPage
Brandprotect Msrabot
CATExplorador RebelMouse
Firefox/7.0 Siteimprove
RealDownload Sociscraper
Stripper Sogou web spider
TwengaBot TeleportPro
Webster URLy.Warning
Xaldon_WebSpider Wprecon
facebookscraper Zauba

288
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404checker 404checker
AIBOT 80legs
Abonti Aboundex
AfD-Verbotsverfahren Aboundexbot
Acunetix
AhrefsBot
Aipbot Aipbot
AlphaBot Anarchie
Anarchy AwarioSmartBot
Attach BBBike
AwarioRssBot BLEXBot
BDFetch
BackStreet BackStreet
BackWeb Backlink-Ceck
BacklinkCrawler
Badass Badass
BatchFTP
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bitacle Bitacle
Black Hole Black Hole
BlackWidow
Boardreader
Bolt Bolt
BotALot
Brandprotect Brandprotect
Buck Brandwatch
Buddy CATExplorador
BunnySlippers CCBot
CODE87 Calculon
CazoodleBot
CensysInspect CensysInspect
CheTeam CheTeam
CherryPicker CheeseBot
ChinaClaw ChinaClaw
Cocolyzebot Cocolyzebot
Cogentbot Cogentbot
Collector
Copier Copier
CrazyWebCrawler Crescent
CrunchBot CrunchBot
Curious Curious
Custo
DBLBot DBLBot
DIIbot DatabaseDriverMysqli
DTS Agent Deusu
Demon Devil
Digincore
Dirbuster Dirbuster
Disco Discoverybot
DomCopBot Dispatch
DomainAppender DomainSigmaCrawler
DomainStatsBot
Domains Project Domains Project
Dotbot Dotbot
Download Wonder Download Wonder
Drip Dragonfly
ECCP/1.0
EMail Siphon EMail Siphon
EasyDL
Ebingbong Ebingbong
EirGrabber EirGrabber
EroCrawler Exabot
Express WebPictures Express WebPictures
ExtLinksBot ExtLinksBot
ExtractorPro Extreme Picture Finder
EyeNetIE Fimap
FHscan
Firefox/7.0 Firefox/7.0
FlashGet FlashGet
Foobot Flunky
FyberSpider FrontPage
GT::WWW
GalaxyBot GalaxyBot
Genieo Genieo
GoZilla GetRight
Grabber Getintent
Go!Zilla
Go-Ahead-Got-It
Gotit
GrabNet
Grafula Grafula
GrapeFX GridBot
HEADMasterSEO HTMLparser
HTTP::Lite
Haansoft Haansoft
HaosouSpider HaosouSpider
Havij
Hloader Hloader
IRLbot Humanlinks
Image Fetch HybridBot
Image Sucker IDBTE4M
IndeedBot IDBot
Indy Library InfoNaviRobot
InfoTekies InterGET
Internet Ninja JamesBOT
InternetSeer Jetty
Iria
Iskanie
IstellaBot
JennyBot
JikeSpider JikeSpider
Joomla
Jorgee
Jyxobot Jyxobot
Kenjin Spider
Keyword Density
Kinza Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot
Larbin Larbin
Leap LeechGet
Libwhisker
Lightspeedsystems Lightspeedsystems
Likse Likse
LinkScan
LinkWalker
Linkdexbot Linkdexbot
LinkextractorPro
LinkpadBot
LinksManager LinksManager
Lipperhey Spider Lipperhey
Litemage_walker Lmspider
MIDown tool
MJ12bot MJ12bot
MSIECrawler MQQBrowser
MTRobot Mag-Net
Majestic SEO Mail.RU_Bot
Masscan Majestic12
Mata Hari Mass Downloader
MauiBot MauiBot
MeanPath Bot Mb2345Browser
Meanpathbot Meanpathbot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru MicroMessenger
Microsoft URL Control Microsoft URL Control
Moblie Safari Mojeek
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mr.4x3 Musobot
NPbot Name Intelligence
Nameprotect Nameprotect
Navroad NearSite
Needle
Nessus Nessus
NetAnts Net Vampire
NetLyzer NetSpider
NetZIP
Nettrack
NextGenSearchBot
Nibbler Nibbler
Niki-bot
Nikto Nikto
Nimbostratus Nimbostratus
Ninja
Nmap
Nutch
Offline Explorer Offline Explorer
Offline Navigator OpenVAS
OnCrawl Openvas
OrangeBot OutfoxBot
OutclicksBot
PHPCrawl PHPCrawl
POE-Component-Client-HTTP
Page Analyzer
PageScorer PageScorer
Pandalytics Pandalytics
Pavuk
PeoplePal PeoplePal
Petalbot
Pi-Monster Pi-Monster
Picscout
Picsearch Picsearch
Pixray Pixray
RSSingBot Psbot
RankActive PxBroker
RankingBot PyCurl
QueryN Metasearch
Quick-Crawler
RankFlex
RankingBot2
RankurBot RankurBot
Reaper
RepoMonkey RepoMonkey
Ripper Ripper
RocketCrawler
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler SEOprofiler
SISTRIX
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScreenerBot Scanbot
ScrepyBot ScoutJet
Semrush Scrapy
SemrushBot SemrushBot
SeobilityBot Seomoz
Shodan Shodan
SiteCheckerBotCrawler
SiteExplorer SiteExplorer
SiteSnagger SiteSnagger
Sitebeam Sitebeam
Siteimprove Siteimprove
SlySearch SlySearch
Snapbot SmartDownload
Sottopop Snoopy
Spanner Sogou web spider
Spbot
Spinn3r
SputnikBot SputnikBot
Sqlmap Sqlmap
Sqlworm
Sqworm Sqworm
Steeler
Sucker Sucker
SuperHTTP SuperHTTP
SurveyBot SurveyBot
Szukacz Suzuran
Swiftbot
T8Abot T8Abot
TeleportPro Telesoft
The Intraformant
TheNomad TheNomad
TightTwatBot TightTwatBot
Titan Toweyabot
Toata Turingos
True_Robot
Turnitin
TurnitinBot TurnitinBot
Twice
Typhoeus Typhoeus
URLy Warning URLy Warning
URLy.Warning
UnisterBot
Upflow
V-BOT
VB Project
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
WASALive-Bot VeriCiteCrawler
WBSearchBot Voil
WEBDAV
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
Wallpapers Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Fuck Web Fetch
WebCopier WebCopier
WebFetch WebEnhancer
WebFuck WebGo IS
WebImageCollector WebSucker
WebLeacher WebmasterWorldForumBot
WebPix
WebReaper
WebSauger
WebStripper
Website Quester Website Quester
WinHTTrack WebsiteQuester
WiseGuys Robot Whack
Wotbox Who.is Bot
Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Xenu Xenu
Zitebot YoudaoBot
ZoominfoBot Zeus
arquivo.pt ZmEu
autoemailspider ZoomBot
ZumBot
check1.exe
clark-crawler
coccocbot-web
crawl.sogou.com crawl.sogou.com
crawler4j crawler.feedback
demandbase-bot
evc-batch evc-batch
facebookscraper facebookscraper
gopher ips-agent
magpie-crawler isitwp.com
mediawords lwp-request
oBot muhstik-scan
pcBrowser netEstate NE Crawler
plumanalytics plumanalytics
polaris version
satoristudio.net
scalaj-http scalaj-http
seostar scan.lol
serpstatbot serpstatbot
sitechecker.pro sexsearcher
sogouspider
sp_auditbot sp_auditbot
webmeup-crawler trendiction.de
webpros.com ubermetrics-technologies.com
x09Mozilla webprosbot
zauba.io zauba.io

296
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com ADmantX
360Spider
404checker
404enemy
AIBOT AIBOT
Abonti ASPSeek
Aboundex Aboundexbot
Acunetix Alexibot
AhrefsBot
Alligator
Anarchie Anarchie
Apexoo Asterias
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot AwarioSmartBot
BDFetch BDCbot
BackDoorBot BLEXBot
BackStreet
BackWeb BackWeb
Bandit Backlink-Ceck
BatchFTP BatchFTP
Battleztar Bazinga
Bigfoot
Bitacle Bitacle
BlackWidow
Blackboard
Blow Blow
Boardreader BlowFish
Bolt Brandwatch
Brandprotect
Buddy Buddy
BuiltBotTough Bullseye
BunnySlippers BuzzSumo
CODE87 CSHttp
Calculon
Cegbfeieh
CheTeam CheTeam
ChinaClaw ChinaClaw
Chlooe Cogentbot
Copier
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Crescent CrazyWebCrawler
CrunchBot
Curious Curious
CyotekWebCopy Custo
DBLBot DIIbot
DSearch
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Deusu Deusu
Digincore Devil
DigitalPebble
Dirbuster Dirbuster
Discoverybot Disco
DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project Download Wonder
Dotbot Drip
Dragonfly ECCP/1.0
EMail Siphon EMail Siphon
Ebingbong EMail Wolf
Ecxi Ecxi
EroCrawler
Evil Evil
ExtLinksBot Extreme Picture Finder
ExtractorPro FDM
EyeNetIE
FemtosearchBot FemtosearchBot
Fimap Fimap
FlashGet Firefox/7.0
Foobot
Fuzz Fuzz
Genieo Fyrebot
GermCrawler GetRight
GetWeb
Getintent
Gigablast
Go!Zilla Go!Zilla
Gotit GrapeFX
GrapeshotCrawler
HTMLparser HTMLparser
HTTP::Lite
HTTrack HTTrack
Heritrix
Humanlinks Humanlinks
HybridBot
IDBTE4M IDBTE4M
IDBot IndeedBot
Iblog
Indy Library Indy Library
InfoTekies
Internet Ninja
InternetSeer
Iria
Iskanie Iskanie
Jbrofuzz IstellaBot
JennyBot JamesBOT
Jetty Jetty
Jorgee JikeSpider
JustView Jyxobot
Keyword Density Kenjin Spider
Kinza
Kozmosbot Kozmosbot
LWP::Simple LWP::Simple
Larbin Libwhisker
Lftp LieBaoFast
LibWeb
Likse
LinkWalker LinkWalker
Linkbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot Lipperhey Spider
LinqiaRSSBot Litemage_walker
MFC_Tear_Sample Lmspider
MIIxpc MIIxpc
MSIECrawler MSIECrawler
Mag-Net MTRobot
Magnet
Majestic SEO Majestic SEO
Majestic12 Majestic12
MarkMonitor MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
Meanpathbot MauiBot
Metauri MegaIndex.ru
MicroMessenger
Microsoft URL Control Microsoft URL Control
Mister PiX Mister PiX
Mojeek Musobot
Mojolicious NICErsPRO
Name Intelligence
Navroad
NearSite NearSite
Net Vampire NetLyzer
NetAnts
NetSpider
Netcraft Netcraft
Nettrack Nettrack
Niki-bot
Nikto Nikto
Ninja NimbleCrawler
Offline Explorer Nimbostratus
OnCrawl Nuclei
OutclicksBot Octopus
Offline Navigator
Openvas
OrangeSpider
OutfoxBot OutfoxBot
Page Analyzer PECL::HTTP
POE-Component-Client-HTTP
PageGrabber PageGrabber
PageScorer
PageThing.com PageThing.com
Pandalytics
Panscient Panscient
Papa Foto Papa Foto
PeoplePal PeoplePal
Petalbot
Picscout
Picsearch Picsearch
PictureFinder
Piepmatz
Pixray Pixray
PleaseCrawl
ProPowerBot ProPowerBot
Probethenet ProWebWalker
Psbot
Pump Pump
QueryN Metasearch QueryN Metasearch
RSSingBot Quick-Crawler
RankActive
RankFlex
RankingBot2
Rankivabot
ReGet ReGet
Reaper Reaper
RepoMonkey
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SEOkicks SBIder
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOprofiler
SISTRIX SEOstats
SalesIntelligent
ScanAlert ScanAlert
Scanbot
ScoutJet
ScreenerBot ScreenerBot
ScrepyBot SearchmetricsBot
Semrush
SentiBot SentiBot
SeoSiteCheckup
Seomoz Seomoz
Siphon Shodan
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker Sitevigil
Siteimprove SmartDownload
SlySearch Snapbot
Sosospider Sociscraper
Sogou web spider
Sottopop Sottopop
SpaceBison SpankBot
Spinn3r Spanner
Sqlworm Spbot
Sqworm
Steeler Steeler
Stripper Sucuri
Sucker SuperBot
SuperHTTP SuperHTTP
Surfbot
SurveyBot
Suzuran
Swiftbot
Szukacz Szukacz
T0PHackTeam T0PHackTeam
T8Abot Teleport
TeleportPro The Intraformant
Telesphoreo
TheNomad
Titan Titan
Toata
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Trendiction Trendiction
True_Robot Turingos
Turnitin Twice
URLy Warning URLy.Warning
UnisterBot Upflow
VCI
Vacuum
Vagabondo Vagabondo
VeriCiteCrawler VeriCiteCrawler
VidibleScraper Virusdie
Voltron VoidEYE
WASALive-Bot WASALive-Bot
WBSearchBot WBSearchBot
WWW-Mechanize WISENutbot
WPScan
WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Web Pix
WallpapersHD Web Sauger
Web Enhancer Web Sucker
Web Fuck WebAuto
WebBandit WebImageCollector
WebCollage WebPix
WebCopier WebZIP
WebEnhancer WebmasterWorldForumBot
WebFetch
WebGo IS
WebLeacher
WebSucker
WebWhacker
Webshag
Website Quester
Whack Whack
WinHTTrack Whacker
WiseGuys Robot
Wonderbot Wonderbot
Woobot Wotbox
Wprecon Xaldon_WebSpider
Zitebot YoudaoBot
ZyBorg Zade
archive.org_bot Zauba
backlink-check ZmEu
com.plumanalytics ZoomBot
crawl.sogou.com arquivo.pt
crawler.feedback autoemailspider
facebookscraper check1.exe
demandbase-bot
gopher gopher
internetVista monitor heritrix
ips-agent ips-agent
lwp-request lwp-trivial
magpie-crawler
mediawords mediawords
netEstate NE Crawler
plumanalytics plumanalytics
polaris version probe-image-size
ripz
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scalaj-http sexsearcher
sogouspider sogouspider
sp_auditbot sp_auditbot
spyfu spyfu
sysscan trendiction.com
ubermetrics-technologies.com trendiction.de
voyagerx.com voyagerx.com
webprosbot webprosbot
xpymep1.exe
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
aviav.org 1688.com
bazaronline24.ru apidiamondatanet-a.akamaihd.net
cheapestjerseys-wholesale.com epngo.bz
i-service.kz inboundlinks.win
my-aladin.com madot.onlinewebshop.net
s8-nowy-wygraj.comli.com mrmoneymustache.com
seeresultshub-a.akamaihd.net onclkads.com
ww1943.ru rvi.biz
xchat26.myfreecams.com sex-tracker.com
xvideosbay.com wttavern.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1688.com 00webcams.com
178evakuator178.ru 08800.top
1flag.co.za 0912701309f8ce.com
2000k.ru 0lovespells0.blogspot.com
1bet.com
404.mn
4inn.ru 4inn.ru
4ur.click 5000-cotydzien.com
51.la
5elementov.ru
5i2.net 5i2.net
aa81bf391151884adfa3dd677e41f94be1.com 5kstore.com
absugars.com 5tey7463.weebly.com
adbetclickin.pink 70casino.online
addtoadd.men 72-news.com
adf.ly 8jn.dating-app.ru
aa9d046aab36af4ff182f097f840430d51.com
abcdefh.xyz
adidas.frwebs.fr
adminshop.com
adnotbad.com
adpremium.org
adultfriendfinder.com
adzpower.com adzpower.com
agadelha.com.br affordablewebsitesandmobileapps.com
all4invest.ru afftrack001.com
alphahoverboards.com afslankpillen2017nl.eu
amazon-adsystem.com ahhjf.com
an-donut.com airlandsea.info
animal-drawings.com akama.com
animebox.com.ua alevinefesleri.com
apidigidocketnet-a.akamaihd.net alldezservice.kz
apishops.ru alloysteel.ru
apple.com-webbrowsing-security.science alphavisions.net
approved.su ample-awards-today.us
arcteryxsale.online ampower.me
arenda-yeisk.ru android4fun.org
arendakvartir.kz anonymous-redirect.com
arewater.com aosexkontakte.net
arkartex.ru apparel-offer.com
arvut.org apper.de
asrv-a.akamoihd.net applyneedy.xyz
atleticpharm.org arendatora.ru
availit.weebly.com astana.xxxkaz.org
atlasvkusov.ru
atmovs.com
autoplate.info
aviapanda.ru aviapanda.ru
avitocars.ru aviav.co
avtochehli.by aviva-limoux.com
azinoofficial777.ru avon-ukraine.com
baersaratov.ru avtoarenda.by
baikaleminer.com azbukadiets.ru
beachpics.com backuperwebcam.weebly.com
begalka.xyz bagcionderlab.com
bestfortraders.com bala.getenjoyment.net
betonka.pro bannerswap.com
betterhdporn.com battle.net
beyan.host.sk bbsoldes.fr
bgviagrachrx.com beachfix.co
bidbuy.co.kr beauty-bracelet.com
biketank.ga becuo.com
bizru.info besthoro.ru
bizzliving.com bestssaker.com
bjorkbacken.nu bigpenisguide.com
bloggedporn.com bigpictures.club
bloggerads.net bikini-image.com
bloglag.com bimatoprost-careprost.com
blue-square.biz blog-hits.com
bochemit.com.ua blpmovies.com
brainzpod.win bonanza-fortune.men
brandehk.dk bookhome.info
brg8.com boostmyppc.com
bugof.gq braindaily.xyz
bum.com.ru braip.com.br
btnativenav.com
buchananshardware.com
buntube.net
bus-offer.com
buscarfoto.com buscarfoto.com
business-suggested.tk buyantiviralwp.com
buybest1.biz buyessayonline19.blogspot.ru
cactussoft.biz buzzurl.jp
calvet.altervista.org cakemediahq.com.au
camdolls.com callawaygolfstore.online
candy-glam-hp.com camel-beach.com
carmuffler.net candlehandmade.com
capacitacionyaprendizaje.com
carina-sy.de
cars-modification.net cars-modification.net
carson.getenjoyment.net cartierloveringreplica.xyz
cartierring.xyz centrumcoachingu.com
casablancamanor.co.za ceotrk.com
cascadelink.org
casinorewards.info
castingbank.ru
cdn.walkme.com
cdncash.com
cdncash.org
censys.io
cheapbarbour.online cheapbarbour.online
cheapestjerseys-wholesale.com cheapmoncler.pw
cheapjerseysbizwholesale.us cheapwebsitehoster.com
chocolateslim-en-france.com cigarpass.com
christianlouboutinshoes.xyz cineacademy.ru
cialischstgerts.com classiquebijoux.ru
cibpenonptib.flu.cc clevernt.com
cjmc.info clickbank.net
cleanersoft.com clodo.ru
coderstate.com club-samodelkin.ru
colehaanoutlet.store clubfashionista.com
comissionka.net compiko.info
compliance-alex.xyz compliance-alex.top
compliance-andrew.top compliance-alexa.top
compliance-brian.top compliance-elena.top
compliance-irvin.top compliance-fred.top
compliance-julianna.top compliance-jane.top
compliance-willy.top copblock.org
copenergo.ru crystalslot.com
cottageofgrace.com cubook.supernew.org
cougfan.info
credit-online.ws
cscwtalkto.site
culad.com
culmimg.pw culmimg.pw
customergrowthsystems.com customchocolate.business-for-home.com
czat.wp.pl
d0t.ru
daptravel.com
darkages.info
darodar.com darodar.com
data-mining.tk data-mining.tk
daymusam.com daydream-studio.ru
ddlmega.net dayibiao.com
decoratinghomes.ga dealitright.click
defenderxtactical.com degerlund.net
demian.kz
dentfidemountpreach.tk dentfidemountpreach.tk
detki-opt.ru deplim.com
dieswaene.com dimaka.info
diminishedvalueoforegon.com dimfour.com
dirtpics.pw djstools.com
divatraffic.com dktr.ru
divku.ru docsportal.net
doeco.ru dominateforex.ml
dogsrun.net donvito.unas.cz
domini.cat dtm-spain.com
dostavka-v-krym.com dumuelave.xyz
downloaddy.net edge.sharethis.com
downloader12.ru educontest.net
downloadme.life edudocs.net
downvids.net elektrischezigarettekaufen2.cowblog.fr
drillsaw.ru elektrischezigaretten2.beeplog.com
dstroy.su elektroniksigaraankara.info
dtnlyss.com elektrozigaretten1.postbit.com
easyshoppermac.com elite-sex-finders.com
econom.co elitesportsadvisor.com
edelstahlschornstein-123.de elmifarhangi.com
eldiariodeguadalajara.com en.home-task.com
elektronischezigarettekaufen1.myblog.de erotikchat-24.com
elvenmachine.com erotiktreff24.info
englishgamer.com essay-edu.biz
epicbrogaming.com essayservicewriting.org
esnm.ru event-tracking.com
essaypro.com ewebarticle.info
este-line.com.ua excitacion.info
expertblog.info exdocsfiles.com
expolicenciaslatam.co expolicenciaslatam.co
extlabs.io extstat.com
extractorandburner.com fast-wordpress-start.com
ezigarettekaufen1.hpage.com felizporno.com
ezigarettenkaufen1.deviantart.com fifa-coins.online
famix.xyz fighrofacciufreesig.ga
fandlr.com filmbokep69.com
fashion-mk.net filmci.pro
favornews.com findpik.com
fdzone.org firesub.pl
fefo.gdn fishtauto.ru
filmania-x.ru fitfloponline.store
filmidivx.com fortwosmartcar.pw
financehint.eu forum20.smailik.org
finteks.ru freakycheats.com
fiverr.com free-share-buttons.com
flirt4free.com freefoto.ca
forex.osobye.ru freeseedsonline.com
foxtechfpv.com freeuploader.ml
free-share-buttons.top fres-news.com
freecamdollars.com
freenom.link
freevpn.space
freshwallpapers.info freshwallpapers.info
fuckingawesome.com frivgame250.com
fuckmill.com fx-brokers-review.com
fxtips.ru
fyl.com.ru fyl.com.ru
fym.com.ru g7m.pl
galleryawesome.com gdebestkupit.ru
gallictures.com gemara.com
gaytube.com genericlowlatencyasiodriverhq.aircus.com
gdzkurokam.ru get-your-social-buttons.info
geile-lelly.eu gg.zzyjxs.com
generalporn.org gigixo.com
get-free-social-traffic.com gilbertbanda.net
get-free-traffic-now.com girlporn.ru
getmiro.com glcomputers.ru
given2.com gojiberriess.apishops.ru
globalscam.ga goldandcard.ru
goo.ne.jp gopixdatabase.com
gorodservis.ru gov.yanao.ru
gosreg.amchs.ru gratis-sexkontakte.com
gotomontenegro.net greatgrace.ru
greatfind-a.akamaihd.net greatzip.com
greendream.com.ua greenshop.su
gsmtlf.ru gsasearchengineranker.site
guod.me gsasearchenginerankerdiscount.com
gwebtools.com.br gsou.cf
gwhwpxbw.bloger.index.hr guge.io
h2monline.com hamacapty.com
hanwei.us handicapvansarea.com
hdseriale.pl hatedriveapart.com
headpharmacy.com hawaiielectriclight.com
hd-film.pl
hdimagelib.com
healing-dysplasia.ru
hem.passagen.se hem.passagen.se
hermesbirkinhandbagoutlets.com hentai-manga.porn
herokuapp.com hidefiles.org
highland-homes.com host-tracker.com
histats.com
hitmuzik.ru
hkdiiohi.skyrock.com
hkladys.com
hoholikik.club
holidaypics.org
houdom.net houdom.net
hyiphunter.org hoverboard360.nl
i4track.net huhn.altervista.org
iflycapetown.co.za hully.altervista.org
infobabki.ru humanelydrew.com
insider.pro idegenvezeto.eu
iklysha.ml
ilmexico.com
imagerydatabase.com
images-graphics-pics.com
images.gyffu.com
impisr.edunsk.ru
in-tandem.co
incomekey.net
instakink.com
insurple.com insurple.com
intelhdgraphicsgtdrive6w.metroblog.com
intervsem.ru intervsem.ru
istizanidineanarcoticdrug.blogspot.com intimshop-fantasy.ru
ivanstroi.ru investyb.com
ixora.pro istizanidinelikexanax.blogspot.com
jav-fetish.com iwantedmoney.com
jerseychinabizwholesale.com iyasimasennka.com
jerseyswholesalechinalimited.com izhevsk.xrus.org
jikoman.info jasonpartington.com
johannesburgsingles.co.za javitas.info
johnrobertsoninc.com jennyfire.ru
jrpmakati.com jerseywholesalebizchina.com
kadashihotel.com jewelryandfiligree.com
kareliatobacco.ru kaliningrad.zrus.org
karpun-iris.ru katushka.net
kemerovo.zrus.org kerwinandcariza.com
keywordglobal.co.za kit-opt.ru
khafre.us kmd-pto.ru
kichenaid.ru kollesa.ru
kino-ecran.ru
kinotorka.ru
knogg.net
koral.se koral.se
kosmetyki.tm.pl krasnodar.ru
kozhniebolezni.com kustanay.kz
krasivoe-hd.com lacloop.info
kung-fu-ru.com lalalove.ru
kupit-adenu.ru lanadelreyfans.us
landinez.co laptop-4-less.com
lida-ru.com lawyers.cafe
lisque.batcave.net layola.biz.tc
liumimgx.pw legalrc.biz
lonely-mature.com letto.by
lovi-moment.com.ua lexaprogeneric.link
mac-shield.com libertybilisim.com
lifebyleese.com
live-sexchat.ru
lombia.co
lotto6888.com
luciddiagnostics.in
lyrics.home-task.com
m-google.xyz
macdamaged.tech
machicon-akihabara.info machicon-akihabara.info
macotool.com magnetpress.sk
magicplayer-s.acestream.net manifestation.betteroffers.review
matsdale.com manimpotence.com
mebel-ekb.com marmotstore.online
mediawhirl.net matematikus.info
medicine-4u.org maxxtor.eu
members.ghanaweb.com mazda-roadsters.com
menetie.ru mcar.in.ua
metabar.ru medmajor.ru
meteostate.com megahdporno.net
mielec.pl megaindex.ru
mil-stak.com megavolt.net.ua
miniature.io minecraft-neo.ru
missclub.info mir-betting.ru
mojowhois.com misterjtbarbers.com
mompussy.net mitrasound.ru
moncleroutletonline.win mlvc4zzw.space
montazhnic.ru mobstarr.com
mrbojikobi4.biz mokrayakiska.com
mriyadh.com moneyteam24.com
msk.afora.ru moneytop.ru
musicpro.monster moomi-daeri.com
muztops.ru moscow-region.ru
mydeathspace.com mp3films.ru
mydownloadengine.com mp3ritm.top
myiptest.com mulberryoutletonlineeu.com
myonlinepayday.co musezone.su
mysexpics.ru muz-tracker.net
myshopmatemac.com mygameplus.com
nagdak.ru mylesosibirsk.ru
narkologiya-voronezh.ru myonigroup.com
narutonaruto.ru mysex21.com
netvouz.com naj-filmy24.pl
nfvsz.com newsperuse.com
ntic.fr nikitabuch.com
nzfilecloud.weebly.com nikkiewart.ru
obiavo.in ningessaybe.me
obiavo.kz noclegonline.info
normalegal.ru
oballergiya.ru
obiavo.ru obiavo.ru
officedocuments.net offers.bycontext.com
ohmyrings.com office-windows.ru
okuos.com offtime.ru
old-rock.com omgtnc.com
olympescort.com omsk.xrus.org
onlinemeetingnow.com
onlyporno.ru onlyporno.ru
open-odyssey.org optom-deshevo.ru
openmultipleurl.com ostroike.org
orhonit.com ostrovtaxi.ru
ownshop.cf owohho.com
oyster-green.com p-business.ru
paccohichetoti.ml paceform.com
paclitor.com pagesense.com
parajumpersoutlet.online paighambot.com
park.above.com painting-planet.com
pdns.cz paradontozanet.ru
perfection-pleasure.ru patol01.pw
perfectplanned.com peekyou.com
perfectpracticeweb.com pepperstyle.ru
picturesfrom.com perm.xrus.org
picurams.pw pictures-and-images.net
pierrehardysale.online pierrehardysale.online
pillscheap24h.com pizdeishn.com
pirateday.ru plaff-go.ru
piter.xrus.org platinumdeals.gr
piulatte.cz playmsn.com
plastgranar.nu pokemon-go-play.online
plastjulgranar.se poligon.com
playfortuna-play.ru polimga.pw
playtap.us
ploenjitmedia.azurewebsites.net
pokerniydom.ru
pony-business.com
popander.mobi
popmarker.com
poppen-nw.net
popugauka.ru
poquoson.org poquoson.org
porndl.org porno.simple-image.com.ua
pornofiljmi.com
pornogratisdiario.com pornogratisdiario.com
pornosmola.info portside.cc
pornowarp.info pr-ten.de
pornoxxx.com.mx prizeestates.cricket
porodasobak.net
pozdravleniya-c.ru
predmety.in.ua
presleycollectibles.com
pretty-mart.com
preventheadacheguide.info
princeadvantagesales.com
privatefx-in.ru
profitwithalex.info profitwithalex.info
psoriasis-file.trade prokotov.com
psvita.ru promoheads.com
pufip.com prosperent.com
pukaporn.com prostitutki-belgoroda.org
qitt.ru proxyelite.biz
rangjued.com pulseonclick.com
radiodigital.co
ranking2017.ga
rankingchart.de rankingchart.de
ranksignals.com rating-casino2021.ru
rarbg.to recinziireale.com
rczhan.com regdefense.com
relatodelpresente.com.ar registrysweeper.com
relayblog.com replica-watch.ru
resant.ru
rfid-locker.co rfid-locker.co
rimedia.org rome2rio.com
robertefuller.com russia-today-video.ru
royalcar-ufa.ru samo-soznanie.ru
rusprostitute.com saneitconsulting.com
rybalka-opt.ru sanjosestartups.com
s1z.ru sbf441.com
scmor.ilxc.cc scanner-elena.top
screpy.com scanner-jack.top
scrapy.org
scrnet.biz.ua scrnet.biz.ua
search-goo.com sdelai-prosto.ru
seksotur.ru sdi-pme.com
sell-fb-group-here.com search-error.com
seo-platform.com search.1and1.com
seogadget.ru searchtooknow-a.akamaihd.net
seorank.info segol.tv
serialsway.ucoz.ru seksvideoonlain.com
sexad.net seocdvig.ru
sexgalleries.top service.adtech.us
shijian.ac.cn sex-pr.net
shinikiev.com.ua sexcamamateurchat.com
sexkontakte-seite.com
sexpartygirls.net
sexsearch.com
sexuria.net
sexy-pings.com
shemale-sex.net
shemalegalls.blogporn.in
shiksabd.com
ship-marvel.co.ua ship-marvel.co.ua
shopperifymac.com shlyahten.ru
site-analyzer.com simplesite.com
sitechecker.pro simpoed.ufop.br
sinestesia.host.sk
sizeplus.work sizeplus.work
smailik.org skypasss.com
small-games.biz slavic-magic.ru
smartadserver.com slim.sellany.ru
smartbalanceworld.com slkrm.ru
snjack.info smallseotools.com
snjatie-geroinovoy-lomki.ru snip.tw
snomer1.ru sockshare.net
snts.shell-pmr.ru softonicads.com
social-s-iii.xyz solitaire-game.ru
softtor.com songplanet.ru
solmarket.by soserfis.com
spb-plitka.ru
special-porn.com
spin2016.cf spin2016.cf
stair.registrydomainservices.com spl63.fr
stmassage.ru stardevine.com
stroiminsk.org startwp.org
sudexpert66.ru stretchmate.net
sundrugstore.com stroy-portal22.ru
superfish.com suhanpacktech.com
superinterstitial.com
support.nopeas.sk
swimpool.ca swimpool.ca
swinon.site swinger-mobil.net
szamponrevita.pl swiped.su
taaaak.com sygraem.com
tasteidea.com t-bygg.com
teenfuck.tv taihouse.ru
telemetryverification.net tamada69.com
template-kid.com taranerymagesswa.blogspot.com
thecoolimages.net tattomedia.com
thefarmergame.com teletype.in
thegolfclub.info thaisamkok.com
thejournal.ru
thenetinfo.com
thetoiletpaper.com
thexart.club
thfox.com thfox.com
thiegs.reco.ws tiandeural.ru
timecrimea.ru timdreby.com
timetorelax.biz titan-ads.life
tizanidinedruginteractions.blogspot.com tizanidinecapsules.blogspot.com
tizanidinestreetvalue.blogspot.com tizanidinehcl4mgtablet.blogspot.com
tmearegion26.com topbestgames.com
todohr.com topsy.com
tonivedu.it tracfone.com
tooplay.com trafficmania.com
top-study.work traffpartners.com
topclickguru.com ttsq.fr
torrnada.ru unrealcommander.org
track-rankings.online uptime-alpha.net
tracklead.net uptime-gamma.net
trafficgenius.xyz urccvfmc.bloger.index.hr
trafficstars.com urll.eu
tramadolandtizanidine.blogspot.com usacasino.com
truck-land.life ustion.ru
tubeline.biz varikozdok.ru
turbabitload.weebly.com vchulkax.com
twinderbella.com
uchetunet.su
uchil.net
udsgame.online
uni.me
unitygame3d.com
urengoy.pro
url2image.com
vasileostrovsky-rayon.ru
vbikse.com
vduplo.ru vduplo.ru
vesnatehno.com vegascosmetics.ru
vesnatehno.ru videosbox.ru
via-energy-acquistare.com videtubs.pl
via-energy-cumpara.com virtualbb.com
viddyoze.com visa-pasport.ru
videokrik.net vod.com.ua
vikistars.com vodaodessa.com
vinsit.ru voloomoney.com
vip2ch.com w3javascript.com
vipps.com.my wallpaperstock.net
virus-schutzmasken.de website-speed-check.site
viveresaniesnelli.it wedding-salon.net
vladimir.xrus.org weddingdresses.xyz
vladimir.zrus.org westen-z.life
voprosotvet24.ru
voronezh.xrus.org
vpnmouse.com
w-journal.ru
wdfdocando.com
we-are-gamers.com
webjam.com
weboptimizes.com
wholesalejerseychinaoutlet.com
whos.amung.us whos.amung.us
whosonmyserver.com
word-vorlagen.xyz word-vorlagen.xyz
worldhistory.biz wovis.site
worldinternetauthority.com wttavern.com
www.bookmaker-bets.com www.wohnkabinen-shop.de
www.timer4web.com
wygraj-teraz.com wygraj-teraz.com
x-diesel.org x-diesel.org
xclicks.net xbaboon.com
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai xev.ru
xn--80abgj3a5acid6ghs.top xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn--b1addnj3cah.xn--p1ai xn--80aanaardaperhcem4a6i.com
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai
xn--80ajbshivpvn2i.xn--p1ai
xpresscare.ru xpresscare.ru
xtrafficplus.com xvideosbay.com
yaoguangdj.com youbrainboost.asia
ynymnwbm.bloger.index.hr yougame.biz
yogamatsexpert.com yougotanewdomain.com
yottos.com youjizz.vc
yougetsignal.com yoursite.com
youtubologia.it yubikk.info
ypmuseum.ru yur-p.ru
yuarra.pluto.ro zajm-zalog-krasnodar.ru
zahvat.ru
zaidia.xhost.ro
zakonobosago.ru zakonobosago.ru
zapiszto.pl zaobao.com.sg
zazagames.org zarenica.net
zebramart.ru zarepta.com
zetmaster.ru zeroredirect2.com
zootoplist.com zirondelli.it
zrelaya.pw zrizvtrnpale.tk
zryydi.com zvuker.net
zvetki.ru
zvooq.eu

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00go.com 100searchengines.com
0lovespells0.blogspot.com 123cha.com
1bet.com 1bet.com
273-fz.ru 20pascals.nl
4webmasters.org 24x7-server-support.site
5i2.net 2drittel.de
6hopping.com 2pxg8bcf.top
7xc4n.com 72-news.com
7zap.com 8lufu.com
8jn.dating-app.ru 98oi.ru
actices.com abbanreddy.com
addbin.men activecampaign.dreamhosters.com
adrunnr.com adprotect.net
ampower.me adtology.com
antons-transporte.de all-streaming-media.com
app-ready.xyz allpdfmags.net
appsecurityr.com ameblo.jp
atelielembrancaqueencanta.com.br anonymousfox.co
avirasecureserver.com apessay.com
avtolombard-krasnodar.com apibetweenlinesn-a.akamaihd.net
azazu.ru apiluckyleapnet-a.akamaihd.net
balkanfarma.org approvedlocal.co.za
bankiem.pl arenanews.com.ua
beachdriveblog.com artefakct.com
berrymall.ru asia-forum.ru
best-coupon-offer.com asscenihotosidea.blogspot.com
best-seo-offer.com atmovs.com
besttorrentknifta.weebly.com autochoixspinelli.com
ayakino.net
b2b-lounge.com
baditri.com
bashtime.ru
battleforupdating.site
bbsoldes.fr
bestofferswalkmydogouteveryday.gq
betonka.pro betonka.pro
bioca.org betshuckclean.com
bioinnovate.co blockety.co
bizzliving.com blog.koreadaily.com
bkns.vn
blackbabesporn.com
bluesman.nu
bombla.org bombla.org
budpost.com.ua bonvillan.com
buigas.00it.com bristolhotel.com.ua
buntube.net buyessayonline19.blogspot.ru
buyparajumpers.online buzzurl.jp
buytizanidineonline.blogspot.com bwlx.prepedu.cn
buzz-porno.info
cacheimages.com cacheimages.com
cactussoft.biz cartierbracelet.xyz
camdenmemorials.com cayado.snn.gr
carstrends2015.com cdncash.org
chinese-amezon.com centrdebut.ru
cialisndbrx.com chatroulette.si
cindymatches.com cheapjerseysa.com
clash-clans.ru cinemaenergy-hd.ru
cleanmypc.com clicksor.net
clickzzs.nl
clipartnew.com clipartnew.com
coaching-netz.info coin-hive.com
compliance-donald.xyz compliance-brian.top
compucelunlock.net compliance-ivan.top
cornerstone-countertops.com connectionstrenth.com
crazyboost.pro covid-schutzmasken.de
creditnation.ru cupday.com
crirussian.ru czat.wp.pl
crystalslot.com daooda.com
customsua.com.ua darmebel.com.ua
demian.kz deadlinkchecker.com
destinationrealestate.com dealwifi.com
deutschehobbyhuren.net dogsrun.net
dieswaene.com earn-from-articles.com
dividendo.ru easyukraine.com
djonwatch.ru elektrischezigaretten1.blogse.nl
donna7753191.ru elektrozigaretten1.webs.com
dood.live empirepoker.com
dvr.biz.ua enpolis.ru
earncash.com.ua eredijovon.com
eets.net excitacion.info
eloconcream.blogspot.com ezigarettekaufen2.yolasite.com
en.altezza.travel f-online.de
excitacao.com fanpagerobot.com
ezrvrentals.com fba-mexico.com
fanoboi.com fifa-coins.online
filmfanatic.com folowsite.com
findsexguide.com forpostlock.ru
fireads.men fortevidyoze.net
fishingwholesale.us foxinsocks.ru
formularz-konkurs.tk freakycheats.com
free411games.com free-floating-buttons.com
freefoto.ca freecamdollars.com
ftns.ru funnypica.com
gameplexcity.com games.kolossale.ru
ge0ip.com gemgrab-a.akamaihd.net
glavprofit.ru gktt.ru
glavtral.ru glasof.es
global-ics.co.za glastecfilms.com.my
goforexvps.com goforexvps.com
good-mummy.ru gogalleryawesome.com
googlepositions.com grand-chlen.ru
graphicwe.org gribokstop.com
hd-film.pl gsasearchengineranker.space
hermesbelts.xyz gymi.name
hermesbracelets.xyz heatpower.ru
hesteel.pl help.tpu.ru
heygidday.biz hentaiheroes.com
hkladys.com hoholikik.club
hoterika.com hoporno.com
ilo134uloh.com hotblog.top
imgarit.pw houston-vikings.com
incep.imagine-ex.co hoverboard360.es
infogame.name huimin764128.com
instasexyblog.com iccornacircri.cf
intellego.info iklysha.ml
it-max.com.ua imabase.com
ivearchenceinflu.cf incanto.in.ua
iwanttodeliver.com instabid.tech
janavibekken.no iqbazar.ru
janettabridal.com junglenet-a.akamaihd.net
jbl-charge.info kamagragelusa.net
jerseyswholesalechinalimited.com keenoutlet.online
jwss.cc kemerovo.zrus.org
kaliningrad.zrus.org kiwe-analytics.com
kellyonline.xyz konkursowo-24.pl
keywordsdoctor.com krasnodar.ru
kinofak.net krynica.info
kinohall.ru ku6.com
kinozapas.com kuzinsp.ru
kletkimehan.ru laudit.ru
kosynka-games.ru layola.biz.tc
kovrenok.ru leadwayau.com
krasivoe-hd.net lflash.ru
lanadelreyfans.us liky.co.ua
laptoper.net link.ac
lerporn.info linksharingt.com
letslowbefast.today littlecity.ch
lollypopgaming.com lmrauction.com
lombia.com localflirtbuddies.com
love-baby.cz locimge.pw
lowephotos.info lsex.xyz
mafa-free.com magda-gadalka.ru
manipulyator-peterburg.ru magento-crew.net
maranbrinfo.com.br magicalfind-a.akamaihd.net
marketingtechniques.info mapquestz.us
meds-online24.com med-zdorovie.com.ua
meet-flirt-dating.com mesbuta.info
megagrabber.ru metallosajding.ru
meget.co.za mitrasound.ru
mexicosleevegastrectomy.com mockupui.com
mir-business-24.ru
mixed-wrestling.ru
mnogabukaff.net
moneyzzz.ru moneyzzz.ru
motherboard.vice.com msfsaar.de
mrpornogratis.xxx msk.afora.ru
mttwtrack.com muizre.ru
myfreetutorials.com musicdaddy.net
na-telefon.biz musirc.com
newsperuse.com my-floor.in.ua
nfljerseyscheapbiz.us mygameplus.ru
nonameread45.live myiptest.com
noumeda.com mypornfree.ru
ntic.fr nailsimg.com
o00.in nina.az
offers.bycontext.com nudo.ca
openfrost.net office-windows.ru
origin-my.ru onclkads.com
oxotl.com openstat.com
paintingplanet.ru pacificair.com
panicatack.com partnerline.men
paparazzistudios.com.au pastaleads.com
partner-high.men picsforkeywordsuggestion.com
pensplan4u.com pojdelo.weebly.com
pflexads.com poppen-nw.net
photokitchendesign.com pricheski-video.com
picturesfrom.com privatbank46.ru
pinup-casino1.ru profitfx.online
pornodojd.ru prohoster.info
pornosee.info quit-smoking.ga
pornosemki.info qwertty.net
porto.abuilder.net razvratnoe.org
printdirectforless.com refererx.com
prizesbook.online rentalcarnavi.info
propranolol40mg.blogspot.com rn-to-bsn.com
prosperent.com royalvegascasino.com
proxyelite.biz runstocks.com
puzzleweb.ru sabizonline.com
qor360.com saltspray.ru
ragecash.com seminarygeorgia59.ga
redirectme.net seocdvig.ru
reelheroes.net servicecenter.co.ua
relax.ru serw.clicksor.com
remybutler.fr sethrollins.net
roma-kukareku.livejournal.com sexyali.com
rumamba.com shijian.ac.cn
runtnc.net shitting.pro
rvi.biz sildenafilcitratemed.com
saneitconsulting.com silverage.ru
sarm.tk smartadserver.com
saveindex.xyz social-s-ggg.xyz
scanner-walt.top socialsignals24.com
seadragonherbery.com spacash.com
semaltmedia.com speechfoodie.com
seoexperimenty.ru spywarespy.com
seorankinglinks.com stair.registrydomainservices.com
sex-tracker.com stard.shop
sexblog.pw stroymonolit.su
shodanhq.com students-cheapskate.ml
sildenafil-tadalafil.info summerlinhomes411.info
simplepooltips.com supers.com.ua
site5.com tattooreligion.ru
slavic-magic.ru tecnoteakviareggio.it
slujbauborki.ru terrafootwear.us
soblaznu.net threecolumnblogger.com
soc-proof.su tizanidinehclsideeffects.blogspot.com
socialmediasuggest.com tizanidineinfo.blogspot.com
soft1.ru tizanidinewithdrawal.blogspot.com
softomix.net touchmods.fr
steamoff.net track.deriv.com
stroydetali.ru tradgardspartner.se
svarbit.com trafficmp.com
tackletarts.co truebeauty.cc
tam-gde-more.ru tsan.net
tastyfoodideas.com tube8.com
the-trader.net tx41tclega.ru
thenews-today.info typer.one
thexart.club uchil.net
tizanidine4mgstreetvalue.blogspot.com ucoz.ru
tizanidineandgabapentin.blogspot.com
topvidos.ru
traderzplanet.in
tvand.ru
ufa.xrus.org ufa.xrus.org
ukrtvory.in.ua uptime-gamma.net
urlopener.blogspot.com.au verymes.xyz
usiad.net viagra-soft.ru
vbikse.com vip-parfumeria.ru
vektorpress.ru voronezh.xrus.org
vkontarkte.com web-analytics.date
vzlomfb.com website-analyzer.info
wanker.us websitevaluebot.com
webinstantservice.com wikes.20fr.com
website-speed-check.site withstandingheartwarming.com
website-stealer.nufaq.com wjgony.com
westum.se wleuaprpxuvr.ga
whatzmyip.net womensterritory.ru
wholesalejerseychinaoutlet.com worldtraveler.world
wholesalejerseyscheapjerseys.us.com wosik-dach.service-for-web.de
x-diesel.com writersgroup580.web.fc2.com
xaijo.com x-diesel.biz
yubikk.info xchat26.myfreecams.com
zeroredirect9.com xportvusbdriver8i.snack.ws
znakomstva-piter78.ru xtraffic.plus
zocaparj.kz yottos.com
zojirushi-products.ru youdao.com
zoodrawings.com zacreditom.ru
zynax.ua zakon-ob-obrazovanii.ru
zarepta.com
zhuravlev.info
zoominfo.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.24 0; 10.1.0.176 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2537 ### Version: V4.2021.08.2538
### Updated: Thu Aug 19 22:01:29 UTC 2021 ### Updated: Fri Aug 20 10:03:10 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2537 ### Version: V4.2021.08.2538
### Updated: Thu Aug 19 22:01:29 UTC 2021 ### Updated: Fri Aug 20 10:03:10 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##