vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 18:50:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.09.2600 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-09-19 10:10:01 +00:00
parent d3dfecdcee
commit 94eaa3e7a1
46 changed files with 82023 additions and 82023 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2599 #### Version: V4.2021.09.2600
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2599 #### Version: V4.2021.09.2600
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.09.2599 #### Version: V4.2021.09.2600
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10116
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.09.2599 #### Version: V4.2021.09.2600
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10120
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10120
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10120
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10120
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

10120
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

10120
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

10120
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2599 2600

10120
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10120
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.96.4.8 0; 13.90.27.186 0;

10120
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

10120
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

10120
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

10120
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.96.4.8 0; 13.90.27.186 0;

10120
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
23.96.4.8 0; 13.90.27.186 0;

10120
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 1; 10.1.0.76 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Boardreader Cliqzbot
InfoTekies Larbin
JamesBOT Masscan
NearSite Musobot
NetAnts POE-Component-Client-HTTP
Picsearch Panscient
SemrushBot Snake
Telesoft WebStripper
Turingos muhstik-scan
WebLeacher siteripz

310
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker ASPSeek
404enemy AfD-Verbotsverfahren
ALittle Client Anarchy99
Aboundexbot Anthill
Aipbot Aspiegel
Alligator
AlphaBot
Anarchie
Anarchy
Asterias Asterias
BDCbot Attach
BDFetch AwarioSmartBot
BLEXBot BLEXBot
BackDoorBot Backlink-Ceck
BackStreet BacklinkCrawler
Badass
Bandit Bandit
BetaBot
Bitacle
Black Hole Black Hole
BlowFish Blackboard
BotALot Boardreader
Brandprotect
Brandwatch Brandwatch
CATExplorador Buck
CCBot Buddy
BuiltBotTough
Bullseye
BunnySlippers
BuzzSumo
CODE87 CODE87
Cegbfeieh
CensysInspect CensysInspect
CherryPicker
ChinaClaw
Chlooe Chlooe
Claritybot
Cliqzbot Cliqzbot
Copier Cloud mapping
Copyscape Cocolyzebot
Cosmos Cosmos
Craftbot
Crawling at Home Project Crawling at Home Project
Crescent Crescent
CyotekWebCopy Curious
DatabaseDriverMysqli Custo
Deusu DIIbot
Digincore DTS Agent
Demon
Devil
DigitalPebble
Disco
Discobot Discobot
Dispatch Dispatch
DomCopBot DomainSigmaCrawler
DomainCrawler DomainStatsBot
Domains Project Domains Project
Dotbot Dotbot
Dragonfly Download Wonder
Drip
ECCP/1.0 ECCP/1.0
EMail Wolf
EasyDL EasyDL
Ecxi
Evil Evil
ExtLinksBot
Extractor Extractor
EyeNetIE
FDM
FHscan FHscan
Freeuploader Flunky
Fuzz Foobot
FyberSpider Fyrebot
G-i-g-a-b-o-t
GT::WWW GT::WWW
Gotit GalaxyBot
GrapeshotCrawler Genieo
GetWeb
Getintent
Go!Zilla
GrabNet
Grabber
HEADMasterSEO HEADMasterSEO
HMView HMView
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack Haansoft
HaosouSpider Havij
Hloader
HybridBot
IDBot
IRLbot
Iblog Iblog
Image Sucker
InfoNaviRobot
InfoTekies InfoTekies
Internet Ninja
Iria Iria
Iskanie
JOC Web Spider JOC Web Spider
JennyBot Jbrofuzz
Jorgee JetCar
JustView JikeSpider
Jyxobot
Kenjin Spider Kenjin Spider
Kinza
Kozmosbot
LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Larbin
Leap Leap
Lftp LeechGet
LieBaoFast LexiBot
Lightspeedsystems Lightspeedsystems
Likse
LinkWalker LinkWalker
Linkbot
Linkdexbot Linkdexbot
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinqiaMetadataDownloaderBot
Litemage_walker LinqiaRSSBot
MJ12bot Ltx71
MQQBrowser MFC_Tear_Sample
MIDown tool
MIIxpc
MSFrontPage MSFrontPage
MTRobot
Magnet Magnet
Mail.RU_Bot
Majestic SEO Majestic SEO
Mass Downloader Majestic12
Mata Hari MarkMonitor
Mb2345Browser MarkWatch
MeanPath Bot
MegaIndex.ru MegaIndex.ru
MicroMessenger Metauri
Mister PiX Microsoft Data Access
Microsoft URL Control
Moblie Safari
Mojolicious Mojolicious
Mozlila Morfeus Fucking Scanner
Musobot Musobot
NICErsPRO
NPbot NPbot
Name Intelligence Navroad
NearSite NearSite
Net Vampire Needle
NetMechanic Nessus
NetLyzer
NetZIP NetZIP
Netcraft Netcraft
Nibbler Nettrack
Niki-bot Nikto
NimbleCrawler
Nimbostratus
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
PHPCrawl Openfind
Page Analyzer OrangeBot
OutclicksBot
OutfoxBot
PageAnalyzer PageAnalyzer
PageScorer Petalbot
Pandalytics
Panscient
Papa Foto
Pavuk
PeoplePal
Picsearch Picsearch
PictureFinder
Piepmatz
Pimonster Pimonster
ProWebWalker PleaseCrawl
Probethenet Pockey
ProPowerBot
PxBroker
PyCurl
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot RankActive
RankFlex RankFlex
Rankivabot RankingBot2
RankurBot
RealDownload RealDownload
Reaper
Recorder
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SBIder SBIder
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler
SISTRIX SISTRIX
SMTBot SMTBot
SalesIntelligent SalesIntelligent
ScoutJet Searchestate
Scrapy Semrush
ScrepyBot Siphon
SearchmetricsBot SiteCheckerBotCrawler
SemrushBot SiteSucker
SeoSiteCheckup
Shodan
SiteLockSpider
SiteSnagger
Sitebeam Sitebeam
SlySearch Siteimprove
Snapbot SmartDownload
Snoopy Snake
SocialRankIOBot SpaceBison
Sociscraper SpankBot
Sosospider Spanner
Spinn3r Spinn3r
Sqlworm Sqlmap
Stripper
Sucuri Sucuri
SuperBot SuperBot
SuperHTTP Surfbot
SurveyBot
Suzuran
Swiftbot
Szukacz Szukacz
T0PHackTeam
T8Abot T8Abot
TeleportPro Teleport
Telesphoreo Telesphoreo
Thumbor Telesphorep
Toweyabot Trendictionbot
Trendiction True_Robot
Typhoeus Turnitin
URLy Warning URLy Warning
URLy.Warning
VB Project VB Project
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VoidEYE
Voil Voil
WASALive-Bot WBSearchBot
WEBDAV WEBDAV
WISENutbot WPScan
WWW-Collector-E WWW-Collector-E
WWW::Mechanize WallpapersHD
WWWOFFLE Web Fuck
Wallpapers Web Sucker
Wallpapers/3.0 WebEnhancer
Web Collage
Web Enhancer
Web Fetch
Web Pix
WebBandit
WebFetch WebFetch
WebFuck WebPix
WebReaper WebStripper
WebSauger
WebSucker WebSucker
WebWhacker
WebZIP
WebmasterWorldForumBot WebmasterWorldForumBot
Webshag
Website Quester Website Quester
WebsiteQuester
Webster Webster
Whack
Whatweb Whatweb
Widow WiseGuys Robot
YoudaoBot Wotbox
Xenu
Zade Zade
Zermelo Zermelo
Zitebot Zeus
ZyBorg ZmEu
archive.org_bot ZoomBot
arquivo-web-crawler ZoominfoBot
autoemailspider arquivo.pt
backlink-check
clark-crawler clark-crawler
coccocbot-web
com.plumanalytics com.plumanalytics
crawler.feedback
demandbase-bot
eCatch eCatch
heritrix gopher
instabid
internetVista monitor internetVista monitor
lwp-request ips-agent
magpie-crawler magpie-crawler
meanpathbot
mediawords mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler s1z.ru
ripz
satoristudio.net satoristudio.net
seobility seobility
seoscanners
seostar seostar
serpstatbot
sitechecker.pro
siteripz
sogouspider sogouspider
spyfu spyfu
tAkeOut tAkeOut
trendiction.de trendiction.com
voyagerx.com ubermetrics-technologies.com
x22Mozilla zgrab
xpymep1.exe

316
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
80legs 80legs
AIBOT ALittle Client
Aboundex Aboundex
AfD-Verbotsverfahren AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
Alexibot
AllSubmitter AllSubmitter
Alligator
AlphaBot AlphaBot
Anarchy99 Anthill
Apexoo
Aspiegel Aspiegel
BBBike Asterias
BDCbot AwarioRssBot
BLEXBot AwarioSmartBot
BDFetch
BackStreet
BackWeb BackWeb
BacklinkCrawler Backlink-Ceck
Badass Bandit
Barkrowler Barkrowler
BatchFTP Battleztar Bazinga
BlackWidow BetaBot
Black Hole
Blackboard Blackboard
Blow
BotALot BotALot
Brandprotect
Buck Buck
BuiltBotTough BuiltBotTough
Bullseye
CCBot CCBot
CSHttp CheeseBot
CazoodleBot Cloud mapping
Cegbfeieh
CensysInspect
CherryPicker
Chlooe
Cliqzbot
Cogentbot Cogentbot
CopyRightCheck Copier
Copyscape Copyscape
Cosmos
Craftbot Craftbot
Crawling at Home Project CrazyWebCrawler
Crescent Crescent
Custo CrunchBot
Curious
DBLBot
DIIbot
DSearch DSearch
DataCha0s DTS Agent
DatabaseDriverMysqli Demon
Deusu
Devil
Digincore
Disco
Discobot Discobot
Dispatch
DittoSpyder DittoSpyder
DnyzBot DnyzBot
DomainAppender DomCopBot
DomainCrawler
DomainStatsBot DomainStatsBot
Download Wonder Download Wonder
EMail Siphon EMail Siphon
EasyDL Ebingbong
Ecxi Ecxi
EroCrawler EroCrawler
Evil ExtLinksBot
Exabot Ezooms
Extreme Picture Finder FDM
FHscan FemtosearchBot
FlashGet Fimap
FrontPage Foobot
Fyrebot G-i-g-a-b-o-t
GT::WWW GermCrawler
GalaxyBot
Getintent Getintent
Gigabot Go-Ahead-Got-It
GoZilla GoZilla
Gotit
GrabNet GrabNet
Grabber GrapeFX
Grafula GrapeshotCrawler
HTMLparser GridBot
Haansoft HMView
HaosouSpider
Harvest Harvest
Heritrix Hloader
IDBot Humanlinks
Iblog HybridBot
IDBTE4M
Id-search Id-search
IlseBot IlseBot
Image Fetch IndeedBot
Indy Library Indy Library
InfoNaviRobot InfoNaviRobot
Intelliseek Intelliseek
InterGET InterGET
InternetSeer
Iria
Iskanie Iskanie
JOC Web Spider JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JennyBot JennyBot
JetCar JetCar
Joomla Jetty
Jorgee Kenjin Spider
JustView Larbin
LNSpiderguy LeechFTP
Lanshanbot LexiBot
LibWeb
LieBaoFast LieBaoFast
Likse
LinkScan
LinkWalker LinkWalker
Linkdexbot
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
MIIxpc Lipperhey
MQQBrowser Lipperhey Spider
Lmspider
MFC_Tear_Sample
MSIECrawler
MTRobot MTRobot
Mag-Net
Magnet Magnet
Majestic SEO Mail.RU_Bot
Majestic-SEO
Majestic12 Majestic12
MarkWatch MarkWatch
Masscan
Meanpathbot Meanpathbot
Microsoft Data Access Mister PiX
Mojeek Moblie Safari
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3
Msrabot
Musobot
NICErsPRO NICErsPRO
Nameprotect
NearSite NearSite
Needle
Nessus Nessus
Net Vampire
NetAnts NetAnts
NetLyzer Netvibes
NetSpider
Netcraft
NextGenSearchBot NextGenSearchBot
Nibbler
Niki-bot Niki-bot
Nimbostratus Nimbostratus
Octopus Nuclei
Nutch
Offline Navigator
OnCrawl OnCrawl
OrangeSpider OpenVAS
Openvas
OutclicksBot OutclicksBot
PHPCrawl PECL::HTTP
Page Analyzer POE-Component-Client-HTTP
PageGrabber Pandalytics
PeoplePal Picscout
Pi-Monster Picsearch
PictureFinder Pimonster
ProPowerBot Pixray
Pockey
ProWebWalker
Probethenet Probethenet
Psbot
Pu_iN Pu_iN
PxBroker Pump
RSSingBot PyCurl
RankFlex QueryN Metasearch
RankActiveLinkBot
RankingBot
RankingBot2 RankingBot2
Rankivabot RankurBot
Reaper RebelMouse
RedesScrapy RepoMonkey
RocketCrawler
Rogerbot
SEOlyticsCrawler
SEOprofiler SEOprofiler
SMTBot SearchmetricsBot
SalesIntelligent Semrush
ScanAlert SentiBot
Scanbot
Scrapy
Screaming
Searchestate
Shodan
Siphon Siphon
SiteExplorer Site Sucker
SiteLockSpider SiteCheckerBotCrawler
Sitebeam SiteSnagger
Siteimprove Sogou web spider
Sitevigil
Snake
Snapbot
Snoopy
SocialRankIOBot
Sosospider Sosospider
Spammen
SpankBot SpankBot
SputnikBot Sqlmap
Sqlworm SuperBot
Sqworm SurveyBot
Steeler Swiftbot
Stripper T8Abot
Suzuran TeleportPro
Telesphoreo TheNomad
Toata Thumbor
Tracemyfile Trendiction
True_Robot Trendictionbot
TwengaBot Turingos
URLy.Warning Typhoeus
Upflow UnisterBot
V-BOT
VB Project VB Project
VCI
Vacuum Vacuum
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
Virusdie VidibleScraper
Voil Voil
WASALive-Bot WISENutbot
WEBDAV WWW-Collector-E
WWW::Mechanize WWWOFFLE
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD
WeSEE WeSEE
Web Enhancer
Web Fuck
Web Sauger
WebAuto WebAuto
WebFetch WebBandit
WebPix WebCollage
WebImageCollector
WebLeacher
WebSauger WebSauger
WebStripper Webshag
WebSucker
Webalta
WebsiteExtractor WebsiteExtractor
WebsiteQuester Whack
Whatweb Whatweb
Widow Who.is Bot
WinHTTrack Wotbox
Xaldon_WebSpider
Zade Zade
Zermelo Zermelo
Zeus ZmEu
ZoomBot ZoomBot
ZoominfoBot ZumBot
ZyBorg ZyBorg
arquivo-web-crawler autoemailspider
arquivo.pt backlink-check
crawl.sogou.com check1.exe
crawler.feedback clark-crawler
crawler4j com.plumanalytics
dataforseo.com
domainsproject.org
eCatch
evc-batch evc-batch
facebookscraper heritrix
internetVista monitor instabid
ips-agent ips-agent
isitwp.com lwp-request
meanpathbot lwp-trivial
mediawords mediawords
muhstik-scan netEstate NE Crawler
oBot page scorer
polaris version plumanalytics
ripz satoristudio.net
s1z.ru
scan.lol
seobility seobility
seostar seoscanners
sexsearcher
sitechecker.pro sitechecker.pro
siteripz
sogouspider
sp_auditbot sp_auditbot
spyfu
sysscan
tAkeOut tAkeOut
trendiction.com trendiction.com
trendiction.de ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
x22Mozilla webpros.com
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
2girls1cup-free.com bet-prognoz.com
c1.onedmp.com ezigarettekaufen2.mpbloggar.se
djekxa.ru igadgetsworld.com
ekaterinburg.xrus.org imagefinder.site
infogame.name kongruan.com
milblueprint.com mainhp.net
mirmedinfo.ru ringtonepartner.com
pijoto.net vroze.com
pornohub.me xblognetwork.com
ultramart.biz yopoint.in

932
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1000n1.ru 0n-line.tv
12masterov.com
13tabs.com
14b.info
1j7740kd.website 1j7740kd.website
1kinobig.ru 20pascals.nl
24-ak.ru 3rbseyes.com
2itech.ru 4webmasters.com
34.gs 4webmasters.org
3w24.com 7minuteworkout.com
404.mn 83net.jp
4pp13.com
4ureyesonly.com
888.com 888.com
8gold.com 8kisses.eu
a96527gi.beget.tech 9med.net
aa8780bb28a1de4eb5bff33c28a218a930.com a1security.com.ua
aadroid.net aa08daf7e13b6345e09e92f771507fa5f4.com
aasoldes.fr aa8b68101d388c446389283820863176e7.com
adbetclickin.pink abbanreddy.com
adclickservice.com abtasty.com
adloads.com addblueoff.com.ua
adexprts.com
adidas.frwebs.fr
adpremium.org
adserver-e7.com
adsland.men
adultwebhosting.info adultwebhosting.info
afmuseum.com affiliate-fr.com
all4invest.ru albumsuper.info
alldubai.biz allforminecraft.ru
allesohnegirls.net aloofly.com
allfinweb.com alpinism.ru
allnews24.in ameblo.top
allproblog.com anapa-inns.ru
allwidewallpapers.com animal-drawings.com
alphavisions.net anticrawler.org
amazon-adsystem.com antons-transporte.de
amehdaily.com apelsinnik.website
amigobulls.com apidiamondatanet-a.akamaihd.net
animali.deagostinipassion.it apifasterlightin-a.akamaihd.net
anti-virus-removal.info apilinkswiftco-a.akamaihd.net
apiluckyleapnet-a.akamaihd.net apimossnetinfo-a.akamaihd.net
apioasisspacenet-a.akamaihd.net apishops.ru
arabseyes.com appiq.mobi
asiavirtualsolutions.com applicationg29.com
asrv-a.akamoihd.net arcteryxsale.online
asseenontv.ru as5000.com
asacopaco.tk
astrochicks.com astrochicks.com
atley.eu.pn aussie-prizes.com
autogrep.ru autotrends.today
avtolombard-krasnodar.com babyfactory.fr
azte.ch bagsonsale.online
b00kmarks.com baladur.ru
babyguns.ru bangla-chat-uk-paralud.ga
backlinkwatch.com bannerconnect.net
balkanfarma.org
balkanfarma.ru
bang-hotties.com
bankofthewext.com
barbourjackets.online
barnfurnituremart.com
battleforupdating.site
bazaronline24.ru bazaronline24.ru
beachdriveblog.com beachfix.co
best-seo-offer.com beachpics.com
bestcalovebracelet.cn beauty-b0x.pl
bestdraws.com belinvestdom.by
bestwebsiteawards.com beremenyashka.com
bigpenisguide.com berrymall.ru
bio-japan.net bestprofits-there.com
biznesschat.net bestwaystogetpaid.us
blog.koorg.ru bharatdefencekavach.com
blogig.org bizlist.com.de
bluesman.nu bjetjt.com
brains2.biz blackcurranthumidifiers.site
brainsandeggs.blogspot.com blog-hits.com
briomotor.co blogporn.in
bmw.afora.ru
boc.kz
bochemit.com.ua
bonanza-fortune.men
bonky.biz
bonzbuddy.com
bonzibuddi.com
boom-celebs.com
brandimensions.com
brus-vsem.ru brus-vsem.ru
btvn.ru burkesales.com
bum.com.ru burn-fat.ga
butstrap.space buypillsorderonline.com
buy-meds24.com buzzurl.jp
buzz-porno.info calc-for-credit.ru
byme.se
callawaygolfoutlet.online
callcafe.info callcafe.info
callmd5map.com cartierreplica.win
cartierjusteunclou.xyz cbox.ws
cartierreplica.xyz cbozhe.com
celejihad.info cdncash.org
cgi2.nintendo.co.jp cementaresearch.se
cheapbarbour.online chastnoeporno.com
chee-by.biz chlooe.com
chinese-amezon.com
choosecuisine.com
cialischsrx.com
cialisndbrx.com
cleanmypc.com cleanmypc.com
cocyq.inwtrade.com clevernt.com
collegerentals.com clickcash.com
cometorussia.net clickpapa.com
compliance-willy.top clippingphotoindia.com
cntravelre.com
commoncrawl.org
compliance-jess.top
comsysnet.com
contextualyield.com contextualyield.com
cookie-law-enforcement-ee.xyz country-chic.ru
cool-wedding.net courtshipgift.com
cottageofgrace.com crest-poloski.ru
crynet.cc crynet.cc
customcollegeessays.net cubook.supernew.org
d0t.ru customerpromos-a.akamaihd.net
dandiyabeats.in czeshop.info
darkages.info dalavia.ru
datadepths.com danschawbel.com
dealwifi.com date-withme.com
dfiles.me dcj-nn.ru
dfwu1019.info demian.kz
directrev.com deplim.com
discovertreasurenow.com design-ideas.info
diusyawiga.tk deutschland123.de
dktr.ru diesel-parts28.ru
dokfilms.net diminishedvalueoforegon.com
dolohen.com dms-sw.ru
dosug-lux.ru docsportal.net
dosugrostov.site doctissimo.fr
downloaddy.net dogbreedspicture.net
doxyporno.com dominterior.org
drugstoreforyou.com donvito.unas.cz
easyshoppermac.com doska-vsem.ru
ednorxmed.com doublepimp.com
eduserver.net downvids.net
ekto.ee doyouknowtheword-flummox.ml
elektronischezigarettekaufen1.myblog.de drpornogratisx.xxx
dverimegapolis.ru
e-poker-2005.com
eachdayisagift.review
eas-seo.com
elektrischezigarette1.onsugar.com
elektrischezigaretten2.beeplog.com
elite-sex-finders.com elite-sex-finders.com
emediate.eu elkacentr.ru
energy-ua.com elmacho.xyz
enginebay.ru elvenar.com
epngo.bz en.altezza.travel
erotik0049.com eugenevaultstorage.com
eshop.md exchangeit.gq
etotupo.ru exchanges-bet.com
eupornstar.info exdocsfiles.com
european-torches.ru extlinks.com
euroskat.ru extrabot.com
evehemming.blogspot.com.au ezigarettekaufen2.yolasite.com
f-loaded.de familyphysician.ru
f5mtrack.com fapgon.com
fetlifeblog.com feargames.ru
fickkontaktehobbyhuren.com femdom.twiclub.in
fifa-coins.online filmgo.ru
filmfanatic.com financehint.eu
fishtauto.ru find1friend.com
flex4launch.ru fireads.men
floridamobilebillboards.com fishingwholesale.us
forexgb.ru floridamhca.org
formulaf1results.blogspot.com footballfarrago.com
fordsonmajbor.cf
forpackningsutveckling.se
forum.doctissimo.fr forum.doctissimo.fr
forums.d2jsp.org forums.d2jsp.org
forzeronly.com foxweber.com
free-share-buttons.top free-laptop-reward.com
free-stock-illustration.com freegamesplay.online
freenom.link freenom.link
freewebs.com freewlan.info
freshsuperbloop.com front.ru
freshwallpapers.info fsakhalin.ru
fuckbuddybestgilf.info fxgallery.com
fuckmill.com fxund.us
fusoradio.info fym.com.ru
g33.org g7m.pl
gael-s.ru gael-s.ru
gamebackyard.com gameonasia.com
gardene.ru gay-site.store
genericlowlatencyasiodriverhq.aircus.com getresponse.com
germes-trans.com glall.ru
getyourimage.club goldpanningtools.com
girlgamerdaily.com golmau.host.sk
girlsatgames.ru googglet.com
glcomputers.ru gosmeb.ru
glicol.kz gov.yanao.ru
gloverid.site gpirate.com
gojiberriess.apishops.ru gratuitxblcodes.com
golden-praga.ru greatzip.com
gombita.info groupmoney.ru
gomusix.com
goodhumor24.com
goodprotein.ru
gorodservis.ru
gox.com.ua
gpms.org.my
grtyi.com
gsasearchengineranker.pw
gsasearchengineranker.space gsasearchengineranker.space
gsmtlf.ru gsasearchenginerankerdiscount.com
guge.io guidefs.ru
gwhwpxbw.bloger.index.hr hdfreeporno.net
hatdc.org hentai-manga.porn
hawaiisurf.com hit-men.men
hem.passagen.se homelygarden.com
herehloadibs.cf horoshieokna.com
hostsshop.ru hotblognetwork.com
hotloans.ru hothot.ru
housekuba.org hotkeys.com
hoverboard360.es hoverboard360.de
hoverboard360.se howtowhitenteethfast.xyz
howmuchdoestizanidinecost.blogspot.com htmlcorner.com
howtoclean.club huimin764128.com
i-hobot.ru id-forex.com
ico.re idealtits.net
ideibiznesa2015.ru igrovyeavtomaty777.ru
iflycapetown.co.za ilikevitaly.com
igru-xbox.net
ilo134uloh.com ilo134uloh.com
iminent.com imperiafilm.ru
imk.com.ua importchinacoach-teach.com
impisr.edunsk.ru infobabki.ru
import-sales.com infobanks.ru
instabid.tech internetproviderstucson.com
intervsem.ru io9.com
investmac.com iqbazar.ru
iqoption.com irkutsk.zrus.org
istizanidinelikexanax.blogspot.com isitwp.com
istmira.ru
itag.pw itag.pw
izoll.ru itsdp3.com
jagg.info ix20.ru
jerseychinabizwholesale.us javxxx18.com
jo24news.com jennyfire.ru
journeydownthescale.info jovencitas.gratis
jrpmakati.com kakablog.net
jurajskie.info kasino-money.pw
justucalling32211123456789.tk
jyvopys.com
kaliningrad.zrus.org
kellyonline.xyz kellyonline.xyz
keywordteam.net kihi.gdn
klin3952.ru klejonka.info
kollekcioner.ru klumba55.ru
komp-pomosch.ru koopilka.com
ktotut.net krafte.ru
kutikomi.net kung-fu-ru.com
laapp.com landmania.ru
lancheck.net landscapebackgrounds.blogspot.com
laxdrills.com languagecode.com
lesbian.xyz leadn.pl
letsrepair.in lecbter-relationships.ga
liumimgx.pw ledis.top
liveshoppersmac.com legalrc.biz
lmrauction.com lego4x4.ru
lol-smurfs.com letmacwork.world
lombardfinder.ru levaquin750mg.blogspot.com
louboutinbooties.xyz lida-ru.com
low-format.ru liky.co.ua
luckyday.world link.ac
locatellicorretor.com.br
loftdigital.eu
longgreen.info
losangeles-ads.com
luckyshop.net.ua luckyshop.net.ua
lulea-auktionsverk.se luxmagazine.cf
mabdoola.blogspot.com m1media.net
machicon-ueno.info macrotek.ru
maju.bluesalt.co mafcards.ru
makis.nu mahnwachen-helfen.info
marmitaco.cat mailemedicinals.com
matematikus.info maladot.com
maxthon.com mantramusic.ru
mazda-roadsters.com manve.info
manyresultshub-a.akamaihd.net
mararoom.ru
marinetraffic.com
martlinker.com
master-muznachas.ru
masterseek.com
mcadamssupplyco.com mcadamssupplyco.com
mere.host.sk mebel-alait.ru
meteostate.com meccadumps.net
methodsmarketing.com medicinacom.ru
microstatic.pl mega-bony-2017.pl
mielec.pl megahdporno.net
mir-limuzinov.ru meteocast.net
mirmedinfo.ru micasainvest.com
michaelkorsoutletstore.net
midst.eu
mir-betting.ru
misandesign.se misandesign.se
moncleroutletonline.xyz mister-shop.com
mountainstream.ms mmgq.ru
mrlmedia.net mmofreegames.online
mturkcontent.com moetomnenie.com
mydirtystuff.com mojowhois.com
moomi-daeri.com
morepoweronmac.trade
moscow-region.ru
mrpornogratis.xxx
muycerdas.xxx
mxgetcode.com
mydownlodablefiles.com mydownlodablefiles.com
myfreecams.com myftpupload.com
myhitmp3.club myhealthcare.com
myonigroup.com myphotopipe.com
mysex21.com naj-filmy24.pl
naperehresti.info nakozhe.com
nardulan.com nalogovyykodeks.ru
nastroyke.net nero-us.com
nerudlogistik.ru newsperuse.com
new-post.tk nrjmobile.fr
newfilmsonline.ru nucia.biz.ly
news-speaker.com nzfilecloud.weebly.com
next-dentists.tk o.light.d0t.ru
nodding-passion.tk oddamzadarmo.eu
nodup.ru ofanda.com
nootrino.com oneshotdate.com
nosecret.com.ua ontargetseo.us
notasprensa.info openfrost.com
ntic.fr oralsexfilme.net
o-o-11-o-o.com
oakridgemo.com
obiavo.ru
odesproperty.com
omsk.xrus.org
online247.ml
onlinedomains.ru
onlineku.com
ooo-gotovie.ru
openstat.com
opinionreelle.com
oranga.host.sk oranga.host.sk
osagonline.ru ossmalta.com
osoznanie-narkotikam.net ourville.info
outpersonals.com oyster-green.com
page2rss.com paleohub.info
palvira.com.ua papasdelivery.ru
pcads.ru partner-print.men
penisvergrotendepillennl.ovh pdns.cz
petitions.whitehouse.gov pekori.to
pewit.pw pelfind.me
phormchina.com physfunc.ru
phuketscreen.com piccshare.com
picquery.com picturesfrom.com
picscout.com pierrehardysale.online
pintattoos.com pinstake.com
pinup-casino1.ru pkr1hand.com
pio.polytopesexempt.com pokemongooo.ml
pirateday.ru poligon.com
pivka.xyz pontiacsolstice.info
pk-services.ru porno-play.net
playtap.us pornogig.com
poiskzakona.ru pornozhara.com
porn-w.org pourvous.info
pornoelita.info pozdrawleniya.ru
pornokorol.com privat-girl.net
pornoplen.com privatefx-in.ru
pornpost.in prizestohandle.club
postclass.com promoheads.com
powc.r.ca.d.sendibm2.com pssucai.info
prodess.ru pukaporn.com
proftests.net pussyfleet.com
promoforum.ru quelle.ru
pron.pro quickbuck.com
pronorm.fr quickloanbank.com
prostitutki-rostova.org quizzitch.net
purchasepillsnorx.com r-control.ru
pushdata.sendpulse.com razorweb-a.akamaihd.net
puteshestvennik.com rebelmouse.com
qor360.com registrysweeper.com
rangjued.com rennlist.com
rank-checker.online rentalcarnavi.info
ranksonic.com responsive-test.net
rapidsites.pro rialp.getenjoyment.net
realnye-otzyvy.info ritlweb.com
realresultslist.com rockingclicks.com
remontbiz.ru royal-investments.net
reversing.cc russia-today-video.ru
rfd-split.hr rusvideos.su
rfserial.net safe-app.net
ringporno.com
rollercoin.com
roofers.org.uk
rosbalt.com.ua
roznica.com.ua
sampleletters.net sampleletters.net
sandhillsonline.com saveindex.xyz
sanyuprojects.com saveriopiazza.it
sardinie.us scanmarine.info
scat.porn scanner-mark.top
screpy.com search.1and1.com
sdelatmebel.ru searchimage.co
searchmywindow-a.akamaihd.net security60-e.com
secret.xn--oogle-wmc.com see-your-website-here.com
seoheap.com semaltmedia.com
serving.adbetclickin.pink sendearnings.com
servingnotice.com serialsway.ucoz.ru
sexystrippe.info service.adtech.us
share-buttons-for-free.com sexs-foto.com
shikiso.info sexuria.net
shop.acim.org sexvporno.ru
sindragosa.comxa.com sexy-pings.com
sexy.babes.frontend-stack.top
shitting.pro
shopcheermakeup.info
shopvilleroyboch.com.ua
shtaketniki.kz
sibecoprom.ru
sibvitr.ru
sientalyric.co
simple-image.com.ua
simply.net
sinel.info
site5.com site5.com
siteheart.net
sitevaluation.com
skuteczna-dieta.co.pl skuteczna-dieta.co.pl
skylta.com skutecznetabletkinaporostwlosow.pl
slavkokacunko.de skytraf.xyz
slim.sellany.ru smartadserver.com
slkrm.ru smokewithrabbits.com
slowmac.tech smstraf.ru
sluganarodu.ru snegozaderzhatel.ru
sneakyboy.com socas.pluto.ro
snjatie-geroinovoy-lomki.ru social-button.xyz
soblaznu.net social-search.me
soft-program.com songplanet.ru
solinf.co soviet-portal.do.am
soundfrost.org space-worry.ml
spitfiremusic.com spammen.de
sport-video-obzor.ru stair.registrydomainservices.com
spywarebegone.com stjamesschool.info
statustroll.com
staynplay.net
steamoff.net
stoki.ru
streetfooduncovered.com
stretchmate.net
stroilka.info
stroiminsk.com
strongsignal-a.akamaihd.net
students-cheapskate.ml
studiokamyk.com.pl
sukarame.net sukarame.net
superiends.org superstats.com
support.nopeas.sk susanholtphotography.com
sureone.pro sweepstakes.rewardit.com
surgut.zrus.org swinon.site
svolze.com tedxrj.com
swinger-mobil.net telsis.com
swiped.su teslathemes.com
swsociety.se themeforest.net
szamponrevita.pl themestotal.com
tam-gde-more.ru thepornsex.org
taranerymagesswa.blogspot.com
tecspb.ru
telefonsex-ohne0900.net
terraclicks.com
tetracsaudi.com
the-usa-games.blogspot.com
thebestweightlosspills.ovh
thecoral.com.br
thedownloadfreeonlinegames.blogspot.com
thefotosgratis.eu
thesmartsearch.net thesmartsearch.net
tilido.com titan-cloud.life
timeallnews.ru tizanidine4mgstreetprice.blogspot.com
topdownloads.ru tizanidine4mguses.blogspot.com
topkarkas.com tizanidinedruginteractions.blogspot.com
tradedeals.biz tizanidinedrugtest.blogspot.com
tri-slona.org tizanidineduringpregnancy.blogspot.com
trk-4.net tizanidinepills.blogspot.com
tryrating.com tizanidinerestlesslegsyndrome.blogspot.com
tuberkuleznik.ru top10-way.com
turizm.bz topads.men
tytoona.com torture.ml
uchebavchehii.ru toys.erolove.in
unrealcommander.org tracklead.net
uptime-as.net trafficshaper.com
usiad.net trustaffs.com
vacances-voyages.info tula.mdverey.ru
vacuumscleaner.com tvand.ru
validccseller.com tx41tclega.ru
valmetrundan.se ublaze.ru
ukrobstep.com
uni.me
univerfiles.com
urlcut.ru
urlopener.com
ussearche.cf
utrolive.ru
valsalud.com valsalud.com
varikoz24.com vasileostrovsky-rayon.ru
vchulkah.net vbtracker.net
vekzdorov.ru
vellings.info vellings.info
venta-prom.ru video-hollywood.ru
vereo.eu videochat.bz
viagraneggrx.com vinsit.ru
viandpet.com viven.host.sk
videochat.mx vizitki.net
vip-dom.in vkonche.com
vitoriacabos.com vkontarkte.com
vodaodessa.com vzubah.com
voditeltrezviy.ru
w-journal.ru
wahicbefa31.soup.io wahicbefa31.soup.io
walpaperlist.com waterefficiency.co
web.cvut.cz watracker.net
webadvance.club
webcamdevochka.com
webinstantservice.com
webnode.me webnode.me
weburok.com website-datenbank.de
wejdz-tu.pl website-stealer.nufaq.com
wildcattube.com wholinkstome.com
wineitudes.wordpress.com whos.amung.us
witclub.info womama.ru
www.jbetting.com woman-h.ru
www.pinnacle-bets.com wosik-dach.service-for-web.de
www.wohnkabinen-shop.de woweb.com.ua
wygraj-skiny.win wsgames.ru
xfluro.com ww1943.ru
xgftnlrt.bloger.index.hr www.bookmaker-bets.com
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf www.get-free-traffic-now.com
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai www.timer4web.com
xn--h1aakne2ba.xn--p1ai x-diesel.biz
x-diesel.info
x-lime.net
xdoza.com
xev.ru
xn--80aafb2a.xn--p1ai
xn--80ahvj9e.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
xn--80ak6aa92e.com
xn--b1ag5cfn.xn--p1ai
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xnxx699.com xvideospornoru.com
xtraffic.plus xxlargepop.com
xxxhdvideo.site xxxasianporn.net
yeartwit.com xxxguitars.com
yebocasino.com y8games-free.com
yoluxuryevents.com yellowads.men
yoopsie.com yogamatsexpert.com
yougetsignal.com yubikk.info
youradulthosting.com zacreditom.ru
yourdesires.ru zaidia.xhost.ro
zaim-pod-zalog-krasnodar.ru zakon-ob-obrazovanii.ru
zakazvzloma.com zambini.ru
zaloro.com zazagames.org
zastenchivosti.net zhuravlev.info
zatjmuzu.info zigzog.ru
zawyna.ua zojirushi-products.ru
zeroredirect7.com zona-aqua.ru
zetgie.com.pl zrelaya.pw
zoodrawings.com zygophyceous.womanstars.site
zoogdiesney.com zynax.ua
zoogdinsney.com
zoominfo.com
zreloeporno.tv

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1adult.com 000free.us
23kw.ru 2kata.ru
2x2fan.ru 3w24.com
activepr.ru aa81bf391151884adfa3dd677e41f94be1.com
adpremium.org absolutelyanalog.com
adrenalin-stk.ru academiacsmendoza.org
adzpower.com adultnet.in
ahhjf.com adventureparkcostarica.com
alert.scansafe.net agysya.ru
alfa-img.com akvamaster.dp.ua
allproblog.com all4invest.info
amehdaily.com amospalla.es
animal36.com amung.us
anmysite.com anlimebel.kiev.ua
apioutoboxnet-a.akamaihd.net anticrawler.org
appsaurus.com apccargo.com
autovideobroadcast.com apiappenableinfo-a.akamaihd.net
aversis.net aproposde.com
avramstroy.ru autotop.com.ua
azartclub.org babs.com.ua
baltgem.com bagcionderlab.com
battle.net bamo.xsl.pt
beetpics.pw bankofthewext.com
beget.tech bannertesting.com
bestempresas.es best-coupon-offer.com
bestwrinklecreamnow.com bestofferswalkmydogouteveryday.gq
bezsporno.ru bestplacetobuyeriacta.jw.lt
bigcities.org beyan.host.sk
billiard-classic.com.ua
binomo.com
binomo.kz binomo.kz
bitnote.co
bizlist.com.de bizlist.com.de
bnt-team.com blogos.kz
bronzeaid-a.akamaihd.net bluesalt.co
buypanicdisorderpill.com brainboosting.club
candlehandmade.com brewdom.ru
canrioloadm.gq brianhenry.co
cartiertrinity.xyz buybest1.biz
cdn.walkme.com buyviagraa.blogspot.com
cdnnetwok.xyz cannibalcheerleader.com
cejewelry.xyz cartierreplica.xyz
chorus.terakeet.com cenoval.ru
christianlouboutinreplica.win chatroulette.si
clean-virus-mac.com cheapmoncler.xyz
checktext.org
cl.s7.exct.net
cleanallvirus.com cleanallvirus.com
clickaider.com clickprotects.com
club-lukojl.ru clickso.com
comeondog.info cocolyze.com
compliance-barak.xyz comissionka.net
compliance-viktor.top compliance-josh.top
contextualyield.com coolyarddecorations.com
cookielawblog.wordpress.com craftburg.ru
cougfan.info csgo4.win
crystalslot.com cubook.supernew.org
culmimg.pw customcollegeessays.net
d0t.ru dame-ns.kz
danschawbel.com data-mining.tk
daretodonate.co datract.com
ddsoldes.fr de.zapmeta.com
dktr.ru deanmoore.ie
doktoronline.no dfiles.me
download-walpaperhd.blogspot.com dichvuvesinhhanoi.com
e-collantes.com diegolopezcastan.com
easycommerce.cf dildofotzen.net
edelstahlschornstein-123.de dimkino.ru
edudocs.net directivepub.com
dnepr-avtospar.com.ua
documentsite.net
easytuningshop.ru
ebonyporn.site
egovaleo.it egovaleo.it
eropho.com eldiariodeguadalajara.com
erotische-geschichten-xxl.com emerson-rus.ru
extlinks.com euroskat.ru
fapgon.com exhibitionplus.eu
farm26.ru exlarseva.webblog.es
fashion-mk.net f-loaded.de
favorcosmetics.com fashionindeed.ml
feargames.ru fefo.gdn
filesmonster.porn forodvd.com
filmci.pro foto-weinberger.at
filmidivx.com freesitetest.com
finansov.info
findpik.com
flash4fun.com
forex-procto.ru
formulaf1results.blogspot.com
forum-engineering.ru
freakycheats.com
free-deals.faith
freshdz.com
fuckingawesome.com fuckingawesome.com
gamerextra.com fullgirl.ru
garciniaxt.us gameonasia.com
generic-pills-online.com ge0ip.com
gnuetella.com genetworx.com
golfresa.lucania.se getlaid-xxxhookupdirect.com
gonextmedia.com getyourimage.club
guarrasdelporno.xxx gheus.altervista.org
haikuware.com girlsatgames.ru
healmytrauma.info glall.ru
hopeonthestreet.co.uk gogalleryawesome.com
houston-vikings.com gratuitxblcodes.com
howopen.ru greatidea.marketing
imagerydatabase.com guildebzh.info
insider.pro hard-porn.mobi
iphantom.com hatedriveapart.com
izhevsk.xrus.org high-speed1.net
izhevsk.zrus.org himazin.info
jackpotchances.com hkdiiohi.skyrock.com
javxxx18.com hmywwogw.bloger.index.hr
jillepille.com hostsshop.ru
job.icivil.ir housemilan.ru
johannesburgsingles.co.za houtings.xyz
jwss.cc ido3.com
kazan.xrus.org imperia31.ru
keywordbasket.com increasewwwtraffic.info
kinostorm.net inmate-locator.us
koversite.info instabid.tech
krasnodar.ru invivo.hu
krassh.ru iwantedmoney.com
labelwater.se jav-idol.com
lanasshop.ru jerseyschinabizwholesale.us
lecbter-relationships.ga karting196.ru
letslowbefast.today keenoutlet.online
loadopia.com keywords-monitoring-success.com
longlifelomilomi.info keywordsuggests.com
lyrster.com kupit-adenu.ru
mainlinehobby.net landinez.co
manipulyator-peterburg.ru landscapebackgrounds.blogspot.com
maxthon.com laxdrills.com
midst.eu laxob.com
mistr-x.org m.b00kmarks.com
mnogolok.info magicalfind-a.akamaihd.net
mobile-appster.ru mansiondelrio.co
mobilemedia.md marwer.info
mobioffertrck.com masterseek.com
mobplayer.net mcar.in.ua
moonci.ru medanestesia.ru
mrsdalloways.com meet-flirt-dating.com
mykings.pw mega-bony-2017.pl
national-today-winning-winner.club meget.co.za
newfilmsonline.ru minegam.com
nfljerseyscheapbiz.us mmstat.com
nikhilbahl.com moesexy.com
novgorod.xrus.org moomi-daeri.com
nucia.biz.ly morf.snn.gr
nufaq.com mrwhite.biz
oakridgemo.com msk-diplomat.com
osb.se11.ru mydirtyhobby.com
oyster-green.com mylovelibrabry.com
paclitor.com myplaycity.com
par-fallen.ga myseoconsultant.com
penisvergrotendepillennl.ovh mzdish.site
perso.wanadoo.es nlfjjunb5.ru
php-market.ru no-fuel.org
pic2fly.com normalegal.ru
piter.xrus.org oksrv.com
placid-rounded-coast.glitch.me one.net.in
playfortuna-play.ru online-sbank.ru
porngalleries.top paceform.com
pornogad.com phelissota.xyz
pornoreino.com picmoonco.pw
preparevideosafesystem4unow.site pics-group.com
preventheadacheguide.info poffet.net
pst2017.onlinewebshop.net pornosmola.info
qitt.ru poshiv-chehol.ru
quickcashlimited.com pr0fit-b0x.com
r-e-f-e-r-e-r.com preparevideosafesystem4unow.space
ragecash.com privat-girl.net
rcpmda.ikan1080.xyz productarium.com
reklamuss.ru professionaldieselcare.com
rentehno.ru prostitutki-kiev.org
rep-am.com putitin.me
resant.ru qor360.com
restaurantlescampi.com rank-checker.online
runofilms.ru real-time-analytics.com
sbprabooks.com registry-cleaner.net
scanner-nelson.top royal-investments.net
screentoolkit.com rusexy.xyz
se.bnt-team.com s8-nowy-wygraj.comli.com
searchinterneat-a.akamaihd.net sabizonline.com
seksotur.ru saltspray.ru
semalt.com scanner-donald.top
senger.atspace.co.uk searchaddis.com
seorankinglinks.us searchwebknow-a.akamaihd.net
sex-znakomstva.online seasonvar.ru
sexiporno.net seoheap.com
serpstat.com
service.adtech.fr
sex-sex-sex5.com
sexy-screen-savers.com sexy-screen-savers.com
shillyourcoins.com siha.de
shopperifymac.com silktide.com
signoredom.com simplepooltips.com
skypasss.com sledstvie-veli.net
sluganarodu.ru smartshoppymac.com
sonnikforme.ru sobecjvuwa.com.ru
soochi.co socialbutton.xyz
spl63.fr sports-supplements.us
spravka130.ru stats-collector.org
stpicks.com stevemonsen.com
streamin.to stretchingabuckblog.com
stromerrealty.com studworks.org
strongsignal-a.akamaihd.net sunflowerdrawingpaintings.blogspot.com
stuff-about-money.com susanholtphotography.com
sumo.com taxi-v-eisk.ru
svnuppsalaorebro.se teenforporn.com
td-33.ru tgtclick.com
tds-advert002.info thepantonpractice.co.uk
tdsing.ru thruport.com
telefonsexkostenlos.tk timecrimea.ru
template-kid.com tizanidineforopiatewithdrawal.blogspot.com
tfxiq.com tizanidinewithdrawal.blogspot.com
thelottosecrets.com tonerbox.kz
timdreby.com top10-online-games.com
tizanidinevsvalium.blogspot.com torrentred.games
toonfamilies.net tourcroatia.co.uk
tootoo.to
topmira.com
toproadrunner5.info
track-rankings.online track-rankings.online
track112.site turbo-suslik.org
tracksurf.daooda.com
turbodsp.com
tv-spoty.info tv-spoty.info
udayavani.com uchebavchehii.ru
uptime-gamma.net uptime-gamma.net
url2image.com v24s.net
urlopener.blogspot.com.au vabasa.inwtrade.com
utrolive.ru varbergsvind.se
vadimkravtcov.ru viktoria-center.ru
valmetrundan.se
versaut.xxx-cam.webcam
viandpet.com
villakohlanta.nu
vinsit.ru vinsit.ru
vse-pesni.com vip2ch.com
watch-movies.ru voloslove.ru
web.cvut.cz vzube.com
webpromotion.ae web-analytics.date
webshoppermac.com westsextube.com
wfb.hatedriveapart.com whatistizanidine2mg.blogspot.com
wheelchairliftsarea.com whatsappbot.flyland.ru
while.cheapwebsitehoster.com wholesalejerseysgaa.com
wowcasinoonline.ooo wingsoffury2.com
wufak.com wormix-cheats.ru
www.pinnacle-bets.com www.bookmaker-bets.com
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf www.event-tracking.com
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai x-rates.ru
xn--hxazdsfy.blogspot.com x-stars.ru
xxx-cam.webcam xboxster.ru
youporn-forum.ga xchat26.myfreecams.com
yubikk.info xgames-04.com
zaim-pod-zalog-krasnodar.ru xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai
znakom.sibtest.ru xn--24-glceagatoq7c2a6ioc.xn--p1ai
zog.link xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
zzlgxh.com yougetsignal.com
youradexchange.com
yurgorod.ru
zhacker.net
zigzog.ru
zlatnajesen.com
znakomstva-piter78.ru
zophim.me
zynax.ua

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.1.47 0; 10.1.0.76 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2599 ### Version: V4.2021.09.2600
### Updated: Sat Sep 18 22:01:17 UTC 2021 ### Updated: Sun Sep 19 10:04:19 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2599 ### Version: V4.2021.09.2600
### Updated: Sat Sep 18 22:01:17 UTC 2021 ### Updated: Sun Sep 19 10:04:19 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##