vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-02 10:40:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.08.2504 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-08-05 22:07:01 +00:00
parent ee64eabee7
commit 4cb1c9445d
46 changed files with 85655 additions and 85655 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2503 #### Version: V4.2021.08.2504
#### Bad Referrers Blocked: [7074](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7074](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.08.2503 #### Version: V4.2021.08.2504
#### Bad Referrers Blocked: [7074](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7074](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.08.2503 #### Version: V4.2021.08.2504
#### Bad Referrers Blocked: [7074](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7074](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10572
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.08.2503 #### Version: V4.2021.08.2504
#### Bad Referrers Blocked: [7074](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7074](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [618](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10576
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10576
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10576
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10576
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

10576
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

10576
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

10576
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2503 2504

10576
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10576
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.250.118.86 0; 52.151.13.207 0;

10576
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

10576
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

10576
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

10576
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.250.118.86 0; 52.151.13.207 0;

10576
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.250.118.86 0; 52.151.13.207 0;

10576
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BuiltWith AiHitBot
HybridBot Buddy
LinksManager Digincore
PageGrabber Dirbuster
Quick-Crawler Libwhisker
Sitebeam Pimonster
WebEnhancer SiteCheckerBotCrawler
sitechecker.pro TwengaBot
x09Mozilla Wonderbot
x22Mozilla ips-agent

298
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 01h4x.com
ALittle Client 404checker
AfD-Verbotsverfahren AIBOT
AlphaBot Aboundex
Anarchie AhrefsBot
Aipbot
Alexibot
Anarchy99 Anarchy99
Apexoo Ankit
Asterias AwarioRssBot
AwarioSmartBot BDFetch
BBBike BackDoorBot
BLEXBot
BackStreet
BackWeb BackWeb
BatchFTP Backlink-Ceck
BacklinkCrawler
Battleztar Bazinga Battleztar Bazinga
BetaBot BetaBot
Bigfoot Blow
BlackWidow BlowFish
BotALot Boardreader
Brandwatch Brandprotect
Buddy BuiltWith
BunnySlippers
CCBot
CSHttp CSHttp
Calculon Calculon
CensysInspect CheTeam
CheeseBot CheeseBot
CherryPicker CherryPicker
Chlooe ChinaClaw
Claritybot Cloud mapping
Copier Cocolyzebot
CopyRightCheck CopyRightCheck
Copyscape
Craftbot Craftbot
Custo CrazyWebCrawler
CyotekWebCopy Curious
DIIbot
DTS Agent DTS Agent
Deusu Demon
Digincore
DigitalPebble DigitalPebble
Discobot Discoverybot
Domains Project DittoSpyder
DomainAppender
DomainCrawler
DomainStatsBot
Download Wonder
Drip
ECCP/1.0 ECCP/1.0
EMail Wolf
EasyDL EasyDL
EirGrabber
EroCrawler EroCrawler
Evil
ExtLinksBot ExtLinksBot
ExtractorPro Extractor
Extreme Picture Finder EyeNetIE
Ezooms
FDM
FemtosearchBot FemtosearchBot
Fimap
Firefox/7.0
FlashGet FlashGet
Foobot FrontPage
Freeuploader
Fuzz Fuzz
Fyrebot GT::WWW
Genieo
GermCrawler GermCrawler
GetRight GetRight
GetWeb
Gigablast Gigablast
Go!Zilla Go!Zilla
Gotit
GrabNet GrabNet
Grafula Grabber
GrapeshotCrawler
GridBot GridBot
HMView HMView
HaosouSpider HTMLparser
Harvest HTTrack
Hloader Haansoft
HybridBot
IDBot
IRLbot IRLbot
Image Sucker Id-search
IlseBot
Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies Iskanie
InterGET
Internet Ninja
JOC Web Spider
JamesBOT
Jbrofuzz Jbrofuzz
JennyBot JennyBot
JetCar
Jetty Jetty
Jyxobot JikeSpider
Kenjin Spider Jorgee
Keyword Density
Kinza Kinza
LNSpiderguy LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Leap Leap
LexiBot LeechFTP
Lftp
LibWeb LibWeb
Libwhisker
Likse
LinkScan LinkScan
LinkWalker
Linkbot
Linkdexbot Linkdexbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinksManager
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
Ltx71 Lmspider
MFC_Tear_Sample MIIxpc
MIDown tool
MJ12bot MJ12bot
MSFrontPage MQQBrowser
MSIECrawler MSIECrawler
Mag-Net
Magnet Magnet
Mail.RU_Bot
Majestic SEO
Majestic12 Majestic12
MarkMonitor
MarkWatch MarkWatch
Mass Downloader Mata Hari
Masscan
MeanPath Bot
Meanpathbot Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
MicroMessenger MicroMessenger
Microsoft Data Access Microsoft Data Access
Moblie Safari Mister PiX
Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
NICErsPRO Mozlila
NPbot Msrabot
Nameprotect
Nessus
Net Vampire
NetAnts
NetLyzer NetLyzer
NetZIP NetMechanic
Netcraft
Nettrack
NextGenSearchBot NextGenSearchBot
Nibbler
Nikto Nikto
NimbleCrawler
Nuclei Nuclei
Nutch
Octopus Octopus
Offline Explorer Offline Explorer
OnCrawl OnCrawl
OpenLinkProfiler Openfind
OutclicksBot Openvas
OrangeSpider
PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageGrabber PageGrabber
Pandalytics PageThing.com
Papa Foto Panscient
PeoplePal Pavuk
Petalbot
Pi-Monster
Picscout Picscout
PictureFinder
Piepmatz Piepmatz
PleaseCrawl
Pockey
ProPowerBot ProPowerBot
ProWebWalker
Psbot
Pu_iN
QueryN Metasearch
RankActive RankActive
Rankivabot RankingBot
RankingBot2
RankurBot RankurBot
Re-re
ReGet
RealDownload
Reaper Reaper
RebelMouse RebelMouse
Recorder
RepoMonkey RepoMonkey
RocketCrawler Rogerbot
SBIder SEOprofiler
SEOkicks-Robot SEOstats
SalesIntelligent SISTRIX
ScanAlert Scrapy
Scanbot Screaming
ScrepyBot Semrush
SemrushBot SentiBot
SeoSiteCheckup
Seomoz Seomoz
Shodan Shodan
Siphon SiteCheckerBotCrawler
Site Sucker
SiteExplorer SiteExplorer
SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam Sitebeam
Sitevigil Snake
SlySearch Snoopy
SocialRankIOBot SocialRankIOBot
SpaceBison Sociscraper
SpankBot Sogou web spider
Spanner Sosospider
Spbot Sottopop
Sqlworm
Sqworm
SuperBot SuperBot
Surfbot Surfbot
SurveyBot SurveyBot
Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
T8Abot Teleport
Thumbor Telesphoreo
Toata Telesphorep
The Intraformant
TheNomad
TightTwatBot
Toweyabot
Tracemyfile
Trendictionbot
True_Robot
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Twice Twice
Typhoeus
V-BOT V-BOT
VelenPublicWebCrawler VB Project
Virusdie Vacuum
Voltron Voil
WASALive-Bot WBSearchBot
WEBDAV WEBDAV
WPScan WPScan
WWW-Mechanize WWW-Mechanize
Wallpapers/3.0 WWWOFFLE
Web Fetch WeSEE
Web Fuck Web Auto
Web Sucker Web Collage
WebAuto Web Sauger
WebBandit
WebCollage
WebCopier
WebFuck
WebGo IS WebGo IS
WebPix
WebReaper WebReaper
WebSauger
WebStripper
WebSucker WebSucker
Webalta
Website Quester Website Quester
Whacker
Who.is Bot
Widow Widow
WinHTTrack
WiseGuys Robot
Wonderbot
Woobot
Wotbox
Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Zauba
Zeus Zeus
archive.org_bot arquivo.pt
crawler.feedback check1.exe
crawler4j coccocbot-web
crawl.sogou.com
demandbase-bot demandbase-bot
eCatch domainsproject.org
evc-batch
gopher gopher
muhstik-scan internetVista monitor
lwp-request
oBot oBot
page scorer page scorer
satoristudio.net plumanalytics
scalaj-http polaris version
sysscan probe-image-size
ripz
s1z.ru
sexsearcher
sitechecker.pro
tAkeOut tAkeOut
voyagerx.com
x09Mozilla x09Mozilla
x22Mozilla x22Mozilla
xpymep1.exe
zauba.io zauba.io
zgrab

294
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
ADmantX 360Spider
404checker
AIBOT AIBOT
Aboundex ASPSeek
Abonti
Acunetix Acunetix
AfD-Verbotsverfahren
Aipbot
Alexibot Alexibot
Alligator AllSubmitter
Anarchie Anthill
Anarchy Aspiegel
Anarchy99 AwarioSmartBot
Apexoo
Asterias
BBBike
BLEXBot
BackDoorBot
BackStreet
BackWeb BackWeb
Backlink-Ceck
Badass
Bandit Bandit
Barkrowler Barkrowler
BatchFTP
BetaBot BetaBot
Bitacle Bitacle
Black Hole
BlackWidow
Bolt Bolt
BotALot
Brandprotect Brandprotect
Brandwatch
Buck
Buddy Buddy
CATExplorador BuiltWith
CCBot CCBot
CSHttp CSHttp
Calculon
Cegbfeieh Cegbfeieh
CensysInspect CensysInspect
Collector CheeseBot
Copier CherryPicker
Cocolyzebot
Copyscape
Cosmos
CrazyWebCrawler CrazyWebCrawler
Crescent CrunchBot
Curious Curious
Custo CyotekWebCopy
DBLBot DBLBot
DIIbot DIIbot
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Digincore Deusu
Dirbuster
Disco Disco
Discoverybot Discoverybot
Dispatch Dispatch
DittoSpyder DomCopBot
DomainAppender DomainAppender
DomainStatsBot
Domains Project Domains Project
Download Wonder Download Wonder
Dragonfly Dragonfly
EMail Wolf Drip
EasyDL ECCP/1.0
Ebingbong
EirGrabber EirGrabber
EroCrawler
Evil Evil
Exabot
Express WebPictures Express WebPictures
ExtractorPro Extractor
Extreme Picture Finder Extreme Picture Finder
EyeNetIE FemtosearchBot
FDM Fuzz
Foobot Fyrebot
Freeuploader
FyberSpider
G-i-g-a-b-o-t
GT::WWW GT::WWW
GalaxyBot
GermCrawler
GetRight GetRight
Getintent Gigablast
Gigabot
GoZilla GoZilla
Gotit
GrapeFX GrapeFX
GridBot
HEADMasterSEO HEADMasterSEO
HMView HMView
HTMLparser HTMLparser
HTTP::Lite
Haansoft
Havij Havij
Heritrix
Humanlinks
HybridBot HybridBot
IlseBot IDBTE4M
IDBot
IRLbot
Image Fetch Image Fetch
Image Sucker Image Sucker
InfoNaviRobot InternetSeer
InfoTekies
InterGET
Internet Ninja
Iria Iria
JOC Web Spider JOC Web Spider
Jetty JennyBot
JikeSpider Jorgee
Joomla
JustView JustView
Kenjin Spider Jyxobot
Kinza Kozmosbot
LNSpiderguy LNSpiderguy
LWP::Simple LWP::Simple
Leap LeechGet
Lftp LexiBot
Lightspeedsystems Libwhisker
LinkScan Linkbot
LinkWalker LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaScrapeBot Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
MIIxpc Lmspider
Ltx71
MFC_Tear_Sample
MJ12bot
MSFrontPage MSFrontPage
MSIECrawler MTRobot
Majestic-SEO Mag-Net
Magnet
Mail.RU_Bot
MarkWatch
Mass Downloader Mass Downloader
Masscan Masscan
Mb2345Browser
Meanpathbot Meanpathbot
Metauri Mediatoolkitbot
MicroMessenger Microsoft Data Access
Moblie Safari
Mojeek
Mojolicious Mojolicious
Morfeus Fucking Scanner Morfeus Fucking Scanner
Mozlila Mozlila
Msrabot
NPbot
Name Intelligence Name Intelligence
Nameprotect
Navroad
NearSite NearSite
Nessus Nessus
NetAnts NetAnts
NetLyzer NetSpider
Netcraft NetZIP
Nettrack Nettrack
Netvibes
Nibbler Nibbler
Niki-bot
NimbleCrawler NimbleCrawler
Nutch Nmap
Offline Explorer Offline Navigator
OpenVAS OnCrawl
Openfind OpenLinkProfiler
OutfoxBot OrangeBot
PageAnalyzer PECL::HTTP
PageScorer POE-Component-Client-HTTP
PageThing.com
Panscient
Papa Foto
Pavuk Pavuk
Pi-Monster PeoplePal
Picscout Picscout
Picsearch Picsearch
Pixray PictureFinder
Pimonster
PleaseCrawl PleaseCrawl
ProWebWalker
Pu_iN Pu_iN
PxBroker Pump
RSSingBot PyCurl
RankFlex Quick-Crawler
RankingBot RankActiveLinkBot
RankingBot2
Rankivabot
Recorder
RepoMonkey
Ripper
RocketCrawler
Rogerbot
SBIder SBIder
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SEOprofiler
SEOstats SMTBot
ScanAlert Scanbot
ScoutJet Scrapy
Screaming
ScrepyBot ScrepyBot
Semrush SeoSiteCheckup
SentiBot Seomoz
Siphon
Site Sucker Site Sucker
SiteExplorer Sitevigil
SiteSnagger SmartDownload
Sitebeam Snapbot
SlySearch Sosospider
Snoopy SpaceBison
SocialRankIOBot
Sociscraper
Spammen
SpankBot SpankBot
Spanner Sucker
Spinn3r SuperHTTP
Stripper
Surfbot Surfbot
SurveyBot SurveyBot
Suzuran Suzuran
Teleport Swiftbot
Telesphorep Szukacz
T0PHackTeam
Thumbor
Toweyabot
Tracemyfile Tracemyfile
Turnitin Trendictionbot
TwengaBot True_Robot
Twice Turingos
Typhoeus Upflow
URLy Warning
URLy.Warning
V-BOT V-BOT
Vacuum
Vagabondo
VeriCiteCrawler
VidibleScraper VidibleScraper
WASALive-Bot VoidEYE
WBSearchBot Voltron
WEBDAV
WISENutbot WISENutbot
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWWOFFLE
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WallpapersHD Web Auto
WeSEE
Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Fetch
Web Fuck Web Sucker
Web Sauger WebGo IS
WebFuck WebImageCollector
WebReaper WebReaper
WebSucker WebSauger
WebWhacker WebWhacker
WebZIP Webshag
WebsiteExtractor WebsiteExtractor
WebsiteQuester
Webster Webster
Who.is Bot Who.is Bot
Wotbox WinHTTrack
Xaldon WebSpider
Xenu Xenu
YoudaoBot
Zade Zade
Zermelo Zeus
ZmEu
ZoominfoBot
ZyBorg ZyBorg
archive.org_bot
arquivo.pt arquivo.pt
backlink-check autoemailspider
check1.exe coccocbot-web
cognitiveseo com.plumanalytics
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j eCatch
demandbase-bot evc-batch
domainsproject.org gopher
heritrix heritrix
isitwp.com instabid
lwp-trivial lwp-request
magpie-crawler mediawords
muhstik-scan page scorer
oBot
plumanalytics plumanalytics
polaris version ripz
probe-image-size satoristudio.net
scan.lol
serpstatbot serpstatbot
sitechecker.pro sitechecker.pro
sogouspider sogouspider
sp_auditbot spyfu
sysscan
tAkeOut
trendiction.com
trendiction.de trendiction.de
voyagerx.com
x09Mozilla
zauba.io

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
design-lands.ru balans.shahterworld.org
diminishedvalueoforegon.com fxgallery.com
dyshagi.ru gsasearchengineranker.top
extremal-blog.com pictures-and-images.com
footbalive.org saddiechoua.com
mp3films.ru templates.franklinfire.co
ntic.fr timecrimea.ru
quit-smoking.ga voloo.ru
talant-factory.ru wineitudes.wordpress.com
tengohydar.tk x-rates.ru

940
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
0n-line.tv 00author.com
1001watch.com.ua 00webcams.com
100dollars-seo.com 08800.top
123cha.com 12masterov.com
12u.info 214jaluwobapef.cf
300richmond.co.nz 23kw.ru
3th.co.in 273-fz.ru
45en.ru 4kepics.com
4senses.co 9icmzvn6.website
4webmasters.com aad1f4acb0a373420d9b0c4202d38d94fa.com
5u.com aarbur.com
7minuteworkout.com
7search.com
8558l.jobs.net
991mostfm.co.id
aac500b7a15b2646968f6bd8c6305869d7.com
abcdefh.xyz abcdefh.xyz
abcdeg.xyz above.com
academia-nsk.org actionnooz.com
acunetix-referrer.com adhome.biz
ads-cool.pro adobereader-free.ru
advokateg.ru adsref.men
agapovdg.ru adtech.us
adtology.com
advokateg.xyz
ahhjf.com ahhjf.com
aibolita.com aimayangzhi.com
akita.kz albuteroli.com
aktivator-windows10.blogspot.com alfa9.com
alert-hgd.xyz alfabot.xyz
algarveglobal.com algerianembassy.co.in
andpolice.com alldownload.pw
angigreene.com allforminecraft.ru
allsilver925.co.il
amateurmatch.com
anabolics.shop
anal-acrobats.hol.es
analnoeporno.tv
animal36.com animal36.com
animalrank.com animenime.ru
app5.letmacworkfaster.world anjalika.co.in
appearance-cool.com anonymous-redirect.com
apple.com-webbrowsing-security.science apiholdingmypage-a.akamaihd.net
arendovalka.xyz approvedlocal.co.za
articlesdirectoryme.info artdestshop.eu
astana.xxxkaz.org artpicso.com
autogrep.ru asociatia-tipografilor-transilvania.ro
autonew.biz asrvrep-a.akamaihd.net
autowebmarket.com.ua autodo.info
aviav.ru.com b-styles.xyz
avkzarabotok.com b2b-lounge.com
avtoarenda.by baksman.com
avtolombard-krasnodar.com baltgem.com
azbaseimages.net bannerswap.com
babieca.com bard-real.com.ua
backlink4u.net battleforupdating.site
bad-stars.net
bausparen.bz.it bausparen.bz.it
beauty-bracelet.com bbsoldes.fr
beclean-nn.ru belstaffstore.online
berrymall.ru
bestbuy.ca bestbuy.ca
bestsexyblog.com bestcurs.org
besttorrentknifta.weebly.com bestmobilityscooterstoday.com
betterhdporn.com bestplacetobuyeriacta.jw.lt
bidbuy.co.kr betonka.pro
bigcities.org bezsporno.ru
bikini-image.com beztuberkuleza.ru
bin-brokers.com bif-ru.info
bitcoinremote.com bitnote.co
bjgugu.net.ua blog.koorg.ru
blog-hits.com bloggers.nl
blog.f00kclan.de
bloglag.com
blogos.kz blogos.kz
bluesman.nu blogsfunky672.weebly.com
boltalko.xyz bluejays-jerseys.us
bonux.nextview.ru bnt-team.com
bosman.pluto.ro bodybuilding-shop.biz
brainboosting.club bonky.biz
bonzbuddy.com
bracketsmackdown.com
bradleylive.xyz
brandimensions.com
break-the-chains.com
browsepulse-a.akamaihd.net browsepulse-a.akamaihd.net
browserprotecter.com btnativenav.com
btvn.ru businesn.men
buehne-fuer-menschenrechte.de bvlgariring.xyz
bugshoot.cn bycontext.com
businesxxl.com bytimedance.ru
cand.jp c1.onedmp.com
cakemediahq.com.au
camdolls.com
candlehandmade.com candlehandmade.com
carabela.com.do carfax.com.ua
carloans.com
cartechnic.ru cartechnic.ru
cartierreplica.top cartiertrinity.xyz
cayado.snn.gr cartujano-pre.de
censys.io centrumcoachingu.com
chatroulette.video cheatcode-lita12.rhcloud.com
cheapbarbour.online checkpagerank.net
cheapjerseysa.com chee-by.biz
cheapsergiorossi.online children-learningreading.info
checkhit.com chlooe.com
citetick.com chocolateslim-u-srbiji.com
cityadspix.com ci.ua
cleanersoft.com cinemaenergy-hd.ru
clicksotrk.com claytransformations.info
clubfashionista.com clicksor.net
compliance-george.xyz colehaanoutlet.store
compliance-alexa.xyz
compliance-brian.xyz
compliance-ivan.xyz compliance-ivan.xyz
compliance-jack.top compliance-jane.top
compliance-mary.top compliance-john.top
compucelunlock.net confib.ifmo.ru
computernetworksonline.com cordstrap.cc
conboy.us counciltally.com
contentsexpress.com cymbaltaandtizanidine.blogspot.com
coolgramgoods.com damedingel.ya.ru
copenergo.ru daretodonate.co
countercrazy.com data-mining.tk
cranly.net ddpills.com
day-news.info deadlinkchecker.com
dentalpearls.com.au demenageur.com
dev-seo.blog demian.kz
dev.citetick.com demon-tweeks.com
dirtpics.pw dentfidemountpreach.tk
dispo.de devochki.top
docs4all.com dignitasdata.se
docstoc.com diminishedvalueoforegon.com
domain-submit.info diy-handmade-ideas.com
domaincheck.io dnsrsearch.com
dogbreedspicture.net
dominterior.org
dostavka-v-krym.com dostavka-v-krym.com
dotmass.top doxyporno.com
download-walpaperhd.blogspot.com drugstoreforyou.com
dpihatinh.gov.vn druzhbany.ru
dptaughtme.com dumpsmania24.com
drev.biz eachdayisagift.review
drivotracker.com econom.co
dyshagi.ru education-cz.ru
dyt.net eduinfosite.com
e-collantes.com eduserver.net
ecookna.com.ua elektrischezigarette2.onsugar.com
elektir.ru empathica.com
elektroniksigaraankara.info
elexies.info
emailaccountlogin.co
en.altezza.travel en.altezza.travel
energy-ua.com enews.tech
eredijovon.com engines-usa.com
ero-advertising.com errorfixing.space
extlabs.io es5.com
evening-dating-club.info
executehosting.com
exmasters.com
expertnaya-ocenka.ru
extractorandburner.com extractorandburner.com
ezigarettekaufen2.blox.pl extremez.net
ezigaretteshop.webs.com eyes-on-you.ga
facecup.top ezigarettekaufen.myblog.de
feargames.ru f-loaded.de
felizporno.com f012.de
fetroshok.ru faracontrol.ir
fighrofacciufreesig.ga farmingworm.com
fastfixing.tech
favorcosmetics.com
feeriaclub.ru
feorina.ru
fickluder69.com
filesdatabase.net filesdatabase.net
filesmonster.porn filkhbr.com
filmi-v.online flashlarevista.com
financehint.eu flirt4free.com
financepoints.eu fm-upgrade.ru
find1friend.com forpackningsutveckling.se
fishtauto.ru fortwosmartcar.pw
flowwwers.com fotzen-ficken.com
forum69.info foxinsocks.ru
forzeronly.com foxweber.com
frankofficial.ru franecki.net
free-traffic.xyz front.ru
freesitetest.com fxund.us
freevpn.space fyxabomiw.ru
freewebs.com game-top.su
frequiry.com gamedayhouse.com
freshmac.space gamesprite.me
friendflnder.com gamezblox.com
fuckingawesome.com getmiro.com
fullfileaccess.com getoutofdebtfree.org
fxtips.ru gigablast.com
galleryawesome.com gilbertbanda.net
game-mmorpg.net gk-atlant.info
gaytube.com glall.ru
gazporno.com glavtral.ru
gdcentre.ru gobongo.info
gemgrab-a.akamaihd.net gribokstop.com
get-your-social-buttons.info gsmlab.pl
getdot.ru gstatey.net
gilsonchiro.xyz guarrasdelporno.xxx
globalsurfari.com guildebzh.info
goldandcard.ru hahashka.ru
goodwinmetals.co hamilton.ca
google-liar.ru handicapbathtubarea.com
googlsucks.com hasshe.com
gopro-online.info hatedriveapart.com
gosreg.amchs.ru hchha.com
gratuitxblcodes.com healthcarestore.info
greatdealshop.com hetmanship.xyz
greenshop.su hkladys.com
gsasearchenginerankerhelp.com hostsshop.ru
gsou.cf howtowhitenteethfast.xyz
gta-top.ru
gwebtools.com
haikuware.com
hanink.biz.ly
hdapp1008-a.akamaihd.net
hdimagelib.com
heatpower.ru
hermesbracelets.xyz
hesteel.pl
hildinghr.se
hiwibyh.bugs3.com
homedo.fabpage.com
homelygarden.com
hoverboard360.at
hoverboard360.es
howlongdoestizanidinestayinyoursystem.blogspot.com
howmuchdoestizanidinecost.blogspot.com
hoztorg-opt.ru hoztorg-opt.ru
hully.altervista.org ibmdatamanagement.co
hybrid.ru icaseclub.ru
i-service.kz idegenvezeto.eu
ido3.com ifmo.ru
igrovyeavtomaty777.ru
ihc2015.info
ihtec2019.org
iinstalll-fii1leis.jus0wil.pp.ua iinstalll-fii1leis.jus0wil.pp.ua
imagez.co ilbe.club
impisr.ru immobiliaremassaro.com
incep.imagine-ex.co import-sales.com
intermesh.net incest-ru.com
internet-apteka.ru infoupdate.org
investingclub.ru innodgfdriverhm.aircus.com
iopeninghours.co.uk instasexyblog.com
ipchicken.com intimshop-fantasy.ru
iqs.biz.ua isistaylorporn.info
iradiology.ru istanbulit.com
istizanidineanopiate.blogspot.com istripper.com
istock-mebel.ru javlibrary.cc
itis4you.com johnnyhaley.top
janavibekken.no jrcigars.com
jav-fetish.com
javspace.net
javxxx18.com
journeydownthescale.info
joyceblog.top
jwss.cc
kamen-e.ru
kartiniresto.com kartiniresto.com
kiwe-analytics.com kidskunst.info
knogg.net kinoplen.ru
knowsitall.info kletkimehan.ru
kosmetyki.tm.pl kmgamex.cf
krafte.ru kochanelli.com
krasivoe-hd.net koleso24.com.ua
krynica.info kometars.xyz
kvartira-sutochno.com krasivoe-hd.com
lacasamorett.com krasivye-devushki.net
laptoper.net kurwa.win
live-cam6.info kvartiry-remont.ucoz.ru
locustdesign.co lanadelreyfans.us
lodki-pvh.dp.ua lanasshop.ru
longadventure.com landreferat.ru
mafcards.ru lankarns.com
magicalfind-a.akamaihd.net law-check-two.xyz
magnetpress.sk lawyersinfo.org
mahnwachen-helfen.info laxdrills.com
mansparskats.com leftofcentrist.com
marinetraffic.com lerporn.info
masterseek.com lexiangwan.com
matpre.top lexixxx.com
mature.free-websites.com lida-ru.com
mcnamaratech.com life-instyle.com
michaelkorssaleoutletonline.net liffa.ru
mikrobiologies.ru limtu.ifmo.ru
missclub.info linerdrilling.com
mobioffertrck.com liran-locks.com
mobplayer.net livefixer.com
mobplayer.ru livejasmin.com
mokrayakiska.com liver-chintai.org
monclerboots.xyz loadingpages.me
moneymaster.ru loadopia.com
montazhnic.ru lovi-moment.com.ua
moskva.nodup.ru lyrster.com
magnetic-bracelets.ru
manuscript.su
maranbrinfo.com.br
mararoom.ru
marcogrup.com
masserect.com
meble-bogart.info
meendo-free-traffic.ga
megagrabber.ru
members.ghanaweb.com
menhealed.net
mexicosleevegastrectomy.com
midst.eu
minet.club
mirmedinfo.ru
misswell.net
mnimmigrantrights.net
motherboard.vice.com motherboard.vice.com
mountainstream.ms movies-in-theaters.net
movpod.in
mp3films.ru mp3films.ru
mp3s.club msk.afora.ru
mrinsidesales.com muschisexbilder.com
na-telefon.biz music7s.me
nacap.ru musicstock.me
narosty.com muznachas-service.ru
nastroyke.net my-bc.ru
nationalbreakdown.com myfreecams.com
netanalytics.xyz nbsproject.ru
nhl17coinsforps3.gratisblog.biz needtosellmyhousefast.com
nucia.biz.ly neodownload.webcam
obesidadealgarve.com nextbackgroundcheck.gq
obiavo.su ngps1.ru
ofanda.com nippon-bearings.ru
officedocuments.net novodigs.com
olvanto.ru nrv.co.za
omoikiri-japan.ru nuit-artisanale.com
online247.ml o.light.d0t.ru
odesproperty.com
offer.wpsecurity.website
office2web.com
old-rock.com
olympescort.com
oneclickfiles.com
online-x.ru
onlinemegax.com onlinemegax.com
ooomeru.ru ooo-olni.ru
osb.se11.ru open-odyssey.org
osnova3.ru ops.picscout.com
panouri-solare-acoperis.com optikremont.ru
partner-high.men orhonit.com
photorepair.ru ossmalta.com
piccshare.com outrageousdeal-a.akamaihd.net
picsforkeywordsuggestion.com owathemes.com
picturesboss.com oxford-book.com.ua
pizza-imperia.com paleohub.info
pobeiranie.pl paperwritingservice17.blogspot.ru
poem-paying.gq parajumpersoutlet.online
polytopesexempt.com parlament.biz
popup-jdh.xyz perfection-pleasure.ru
porn555.com perm.xrus.org
porndl.org photokitchendesign.com
porndroids.com picphotos.net
porno.simple-image.com.ua pigrafix.at
pornofeuer.com pinkduck.ga
pornofiljmi.com pixshark.com
pornoklad.net playboyfiles.xblog.in
pornolook.net playtap.us
pozdrawleniya.com pony-business.com
pr0fit-b0x.com portal-eu.ru
pregnant.guru powitania.pl
privacylocationforloc.com pozdrawleniya.ru
privatov-zapisi.ru praisong.net
prizestohandle.club promover.org
profitkode.com prostitutki-odessa.org
proposal-engine.com pskcijdc.bloger.index.hr
prosperent.com purchasepillsnorx.com
prostitutki-almata.org push-ad.com
putitin.me pushdata.sendpulse.com
pyramidlitho.webs.com pussyfleet.com
quit-smoking.ga pxhdwsm.com
quizzitch.net qexyfu.bugs3.com
rankia.com quality-traffic.com
rapidgator-porn.ga quickbuck.com
qwesa.ru
razleton.com
razorweb-a.akamaihd.net razorweb-a.akamaihd.net
redhotfreebies.co.uk recinziireale.com
referencemoi.com relax.ru
refererx.com remontgruzovik.ru
reklamuss.ru remybutler.fr
remont-ustanovka-tehniki.ru rewardit.com
rfserial.net rockprogblog.com
rocis.site roleforum.ru
rocketchange.ru ros-ctm.ru
rossanasaavedra.net royal-betting.net
royalvegascasino.com roznica.com.ua
rp9.ru rusoft-zone.ru
rubanners.com rutor.group
rukino.org s8-nowy-wygraj.comli.com
russintv.fr samo-soznanie.ru
rusvideos.su samolet.fr
s1z.ru scanner-ivan.top
sack.net scanner-viktor.top
sajatvelemeny.com scopich.com
sayyoethe.blogspot.co.za screentoolkit.com
sbprabooks.com sdsjweb.com
scanner-walt.top searchmywindow-a.akamaihd.net
scanner-willy.top seminarygeorgia59.ga
schlampen-treffen.com seojokes.net
sdrescher.net sex-watch.com
secret.xn--oogle-wmc.com sexsearch.com
securesmrt-dt.com sharebutton.to
sell-fb-group-here.com
semprofile.com
semrush.com
seoexperimenty.ru
seoheap.com
seoprofiler.com
seorank.info
servethis.com
serw.clicksor.com
sex-dating.co
sex-znakomstva.online
sexkontakte-seite.com
sexy-screen-savers.com
sexyebonyteen.com
shakhtar-doneck.ru
shisha-swag.de
shodanhq.com
shoplvlv.us shoplvlv.us
shopwme.ru si-unique.com
sicfor.bcu.cc signal03.ru
sientalyric.co sim-service.net
sigmund-freud.co.uk sitechecker.pro
sildenafil-tadalafil.info skachat-besplatno-obrazcy.ru
simply.net skyway24.ru
site-speed-checker.site slavkokacunko.de
site5.com smailik.org
skatestick.bid snegozaderzhatel.ru
sky-mine.ru socialmediasuggest.com
slavia.info
smart-scripts.com
sneakyboy.com
snomer1.ru
snworks.com
social-vestnik.ru
socialsignals24.com
soft-program.com
softlinesolutions.me
solmarket.by solmarket.by
sotechco.co souvenir.cc
sotkal.lark.ru spb.ru
spidtest.space spbchampionat.ru
spravka130.ru specialfinanceoffers.com
stard.shop spinnerco.ca
stjamesschool.info splendorsearch-a.akamaihd.net
sukirgenk.dvrlists.com sstroy44.ru
supers.com.ua startwp.org
svensk-poesi.com sterva.cc
svolze.com streetfire.net
svtrd.com strongsignal-a.akamaihd.net
swsociety.se stroymonolit.su
taraz.xkaz.org sudexpert66.ru
technopellet.gr superinterstitial.com
telemetryverification.net superoboi.com.ua
terraclicks.com surflinksmedical.com
texbaza.by surgut.zrus.org
svnuppsalaorebro.se
swingerseiten.com
tackletarts.co
takethatad.com
tamada69.com
taxi-v-eisk.ru
tds-advert005.info
techart24.com
templates.franklinfire.co
teslathemes.com
testingads.pro
thegioixekhach.com thegioixekhach.com
thepornsex.org thenetinfo.com
titslove.yopoint.in tiens2010.ru
tizanidineinteractions.blogspot.com time-japan.ru
tizanidineonline.blogspot.com timecrimea.ru
tizanidinesideeffectsweightloss.blogspot.com tizanidine4mgstreetvalue.blogspot.com
tizanidinevssoma.blogspot.com tizanidinecapsules.blogspot.com
top-l2.com tizanidinehcl2mg.blogspot.com
top250movies.ru tizanidinerecreationaluse.blogspot.com
topsiteminecraft.com tizanidinewithdrawal.blogspot.com
token-lab.org
tongkatmadura.info
top10-online-games.com
torrent-newgames.com
tour-line.net
tourcroatia.co.uk tourcroatia.co.uk
tozup.com tpu.ru
track2.shop track-rankings.online
trafficmonetize.org trustedmaccleaner.com
truck-addzilla.life tsyndicate.com
truck-rece.life tubo360.com
trustedhealthtips.com turbabitload.weebly.com
tuberkulezanet.ru tvteleport.ru
tuberkuleznik.ru ugguk.online
tupper-posuda.ru
turist-strani.ru
turizm.bz
turizmus.us
twodollarshows.com
uasb.ru
ucoz.ru
ukkelberg.no ukkelberg.no
ultimatesetnewfreeallsoftupgradesystems.pw ukrtvory.in.ua
unmaroll.ya.ru ultramart.biz
unpredictable.ga unrealcommander.com
upstore.me unrealcommander.org
uptime-gamma.net
url-extractor.xyz
url-img.link
urlcut.ru urlcut.ru
urlopener.com uzporno.mobi
uroffer.link vbabule.net
vesnatehno.ru vchulkah.net
viagra-soft.ru vegascosmetics.ru
viandpet.com vekzdorov.ru
videochat.tv.br verymes.xyz
videochat.world via-gra.webstarts.com
videokrik.net virus-respirators.com
vietimgy.pw visitcambridge.org
vilingstore.net viven.host.sk
vip-dom.in
vkak.ru
vkgaleria.com
vkontaktemusic.ru vkontaktemusic.ru
vod.com.ua vkontarkte.com
vpnhowto.info vksex.ru
vsesubwaysurfers.com vostoktrade.info
vtcdns.com vozbujdenie.com
wahicbefa31.soup.io vuclip.com
wallpaperstock.net vvpg.ru
waterpurifier.club watchmyfb.pl
webcamtalk.net web-analytics.date
webjam.com webnode.me
weblo.com webstatsdomain.org
webmasterhome.cn wesharepics.info
weburok.com wetgames.ru
weclipart.com whatzmyip.net
wleuaprpxuvr.ga white-truck.life
wonderfulflowers.biz wingsofrefuge.net
www.ehscloud.cn wiosenny-bon-1500.pl
www.souvenirua.com wladimirpayen.com
www.wohnkabinen-shop.de worldoffiles.ru
x69ty.ru wowas31.ucoz.ru
xchangetrak.com wurr.voila.net
xmronta.com www.solartek.ru
www.timer4web.com
x-lime.com
x-mix.info
xbaboon.com
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
xn--80ab4aa2g.xn--p1ai xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai
xn--b1ag5cfn.xn--p1ai xn--h1ahbi.com.ua
xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai xxxhdvideo.site
xnxxandxvideos.com
xtube.com
xwatt.ru
xxxsiterips.xyz
xxxtubesafari.com
xz618.com
yellocloud.be
yellowads.men
youdao.com youdao.com
za-music.mymobiplanet.com youresponsive.com
zaloadi.ru youtuhe.com
zebradudka.com yun56.co
zeroredirect8.com zaimhelp.ru
zetmaster.ru zarepta.com
zigarettenonl.canalblog.com zoogdiesney.com
zigarettenonlinekaufen2.bloog.pl zootoplist.com
zigzog.ru zrizvtrnpale.tk
zoolubimets.ru zzbroya.com.ua
zrus.org
zvezdagedon.ru

474
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1stat.ru 1-99seo.com
101billion.com
123any.com
20pascals.nl 20pascals.nl
2ads.co.uk 24videos.tv
300richmond.co.nz
4pp13.com 4pp13.com
4senses.co 4webmasters.com
51unlim.ru 7wind.ru
98oi.ru 91abcw.com
a.pr-cy.ru aadroid.net
aanapa.ru aanapa.ru
acc.eu.org aaronabel.com
adobereader-free.ru admeasures.com
adsfresh.men adultfullhd.com
afora.ru adzerg.com
agroeconom.kz
akama.com akama.com
alif-ba-ta.com alfa-img.com
allsilver925.co.il algerianembassy.co.in
amazon-adsystem.com allboard.xobor.de
apiholdingmypage-a.akamaihd.net alldubai.biz
apptester.tk aludecor.info
asdfz.pro ample-awards-today.us
asrv-a.akamoihd.net asacopaco.tk
avtocredit-legko.ru audiofree.ru
benchmarkcommunications.co.uk automatic-seo.com
besthomemadepornsites.com autotop.com.ua
bestpornuha.com avon-novosib.ru
bimmerpost.com b-styles.xyz
bin-brokers.com barbourjackets.online
blockworld.ru bbtec.net
bouda.kvalitne.cz bedandbreakfast.com
brg8.com berrymall.ru
bitcoinremote.com
bjorkbacken.nu
bkns.vn
bmwhighperformers.com
bristolhostel.com
buckspar.gq buckspar.gq
bukleteg.xyz bum.com.ru
buyk.host.sk cakesplus.com.au
cablecar.us calstaterealty.us
camdenmemorials.com capacitacionyaprendizaje.com
candiceloves.us cartechnic.ru
candy-glam-hp.com caveavins.fr
ccjp.eu cheapcigarettesc.info
chatroulette.si cheapjerseysap.com
cheapbarbour.online chinacheapelitejerseys.com
chelnytruck.ru chocolateslim-u-srbiji.com
citetick.com clickpapa.com
com-supportcenter.website club-lukojl.ru
compliance-john.top codq.info
copro.pw compliance-george.top
creditmoney.com.ua copypaste.traderzplanet.in
culad.com corta.co
custom-product-labels.com customboxes4less.com
dating-time-now.com customsua.com.ua
datsun-do.com cxpromote.com
decorationspcs.com datract.com
deda-moroza-zakaz.ru davebestdeals.com
den-noch24.ru detoxmed24.ru
dnepr-avtospar.com.ua docs4all.com
download-wallpaper.net dominateforex.ml
download-walpaperhd.blogspot.com drillsaw.ru
doxyporno.com duplicashapp.com
e-poker-2005.com easync.io
edge.sharethis.com ege-russian.ru
education-cz.ru electrik-avenue.com
elektrischezigarettekaufen2.cowblog.fr elektrischezigaretten2.beeplog.com
elektroniksigaraankara.info elitedollars.com
erotikchat-24.com enginebay.ru
eugenevaultstorage.com ewebarticle.info
familyphysician.ru exct.net
fantasticpixcool.com eywords-monitoring-your-success.com
fermersovet.ru fillmewithhappiness.com
fishtauto.ru filmania-x.ru
flipper.top filmgo.ru
foto-sisek.porngalleries.top freshsuperbloop.com
frameimage.org
free-social-buttons.com
freevpn.space
freewebs.com
front.to front.to
fungamelands.com funnypica.com
gamerscorps.com gambarkatabaru.com
gamevalue7.weebly.com get-your-social-buttons.info
garciniaxt.us getridofstretchmarks.org
gogps.me golfresa.lucania.se
greatfind-a.akamaihd.net gratis-sexkontakte.com
greenidesign.co hamacapty.com
gsasearchengineranker.space hao123.com
guitar-master.org hatdc.org
happy.new.yeartwit.com hawaiielectriclight.com
hdseriale.pl hotsocialz.com
hetmanship.xyz ifmo.ru
heygidday.biz insomniagamingfestival.com
hitcpm.com ipchicken.com
hitmuzik.ru iphantom.com
hobbyhuren-datenbank.com iradiology.ru
homeart.space itsdp3.com
hotgirlhdwallpaper.com janavibekken.no
houston-vikings.com kaidalibor.de
hoverboard360.at keki.info
hoverboard360.de kellyonline.xyz
hscsscotland.com kerwinandcariza.com
icaseclub.ru keywordbasket.com
igithab.com kinofak.net
imagerydatabase.com kiwe-analytics.com
innodgfdriverhm.aircus.com kovesszucs.atw.hu
interesnie-faktu.ru krasivye-devushki.net
inzn.ru kuzinsp.ru
izismile.com kvartira-sutochno.com
japfm.com lawyersinfo.org
karadene.com limtu.ifmo.ru
kino-filmi.com lookingglassemb.com
kiprinform.com louboutinbooties.xyz
klin3952.ru m292.info
kopihijau.info macfix.life
leadwayau.com mahnwachen-helfen.info
lezbiyanki.net map028.com
libertybilisim.com mature.free-websites.com
limads.men max-eclat.men
locksmith.jp mebel-ekb.com
lonerangergames.com medi-fitt.hu
lubetube.com menstennisforums.com
magiadinamibia.blogspot.com metaxalonevstizanidine.blogspot.com
manualterap.roleforum.ru minecraft-rus.org
mebeldekor.com.ua mir-business-24.ru
medizinreisen.de mirzonru.net
meds-online24.com moncleroutletonline.pw
medtherapy.ru musezone.su
mesbuta.info my-big-family.com
message-warning.net myhealthcare.com
mhtr.be mypets.by
michaelkorssaleoutletonline.net mysexpics.ru
mixed-wrestling.ru naughtyconnect.com
mobile.ok.ru nerudlogistik.ru
moneyteam24.com news-speaker.com
moviezbonkerssk.cf
mrcsa.com.au
muabancantho.info
myfreecams.com
nbsproject.ru
newstaffadsshop.club
nextlnk12.com nextlnk12.com
nippon-bearings.ru nikitabuch.com
ntic.fr nomuos.it
nuit-artisanale.com nowtorrents.com
oneminutesite.it nrjmobile.fr
onlywoman.org oballergiya.ru
orel-reshka.net onclkads.com
origin-my.ru onemantrip.com
online7777.com
otbelivanie-zubov.com
ourville.info
ownshop.cf
paphoselectricianandplumber.com paphoselectricianandplumber.com
partner-print.men pathwhelp.org
partnerworkroom.men pc-test.net
paydayonlinecom.com pcboa.se
pinup-casino1.ru peekyou.com
plugingeorgia.com perfectpracticeweb.com
pomoc-drogowa.cba.pl pizdopletka.club
popander.mobi platinumdeals.gr
pornoelita.info pod-muzyku.club
pornofiljmi.com postclass.com
pornoinn.com printingpeach.com
pornoplen.com prosperent.com
pornozhara.com qor360.com
powenlite24.ru quickloanbank.com
pufip.com relayblog.com
qwesa.ru remedyotc.com
rankia.com rniaeba.ga
razvratnoe.org robertefuller.com
rcb101.ru rostov.xrus.org
reklama1.ru royalads.net
resant.ru samolet.fr
rezeptiblud.ru saneyes.com
rfd-split.hr savefrom.com
saitevpatorii.com
saveriopiazza.it saveriopiazza.it
scanmyphones.com sbetodiodnye-lampy.ru
se-welding.ru sbwealthsolutions.ca
search-goo.com scanner-jessica.top
seogadget.ru scenicmissouri.us
seonetwizard.com scripted.com
sex-tracker.com sdi-pme.com
sex-watch.com se.bnt-team.com
sexfreepornoxxx.com semrush.com
sexkontakteao.info sexcamamateurchat.com
sitebeam.net sharebutton.org
skutecznetabletkinaporostwlosow.pl shareyards.com
slonechka.ru sharpchallenge.com
smailik.org shop.acim.org
smartbalanceworld.com sisiynas.ru
spammen.de site-analyzer.com
spravka-medosmotr.ru sitevaluation.org
stanthonyscatholicchurch.org social-search.me
stroicol.net socialbookmarksubmission.org
stroiminsk.org solitaire-game.ru
svensk-poesi.com spb.afora.ru
tacbibirfa.tk stefanbakosab.se
technika-remont.ru surflinksmedical.com
testbotprocessor44.com sweepstakes.rewardit.com
texbaza.by t-bygg.com
thepokertimer.com tattooha.com
theprofitsmaker.net tds-advert005.info
thetoiletpaper.com tecspb.ru
thin.me.pn teenporn18.net
tizanidine6mg.blogspot.com telemetryverification.net
tizanidineorflexeril.blogspot.com templates.radiodigital.co
tizanidineusedfor.blogspot.com thegolfclub.info
top-deal.com.pl tizanidinehydrochloride2mg.blogspot.com
top250movies.ru tizanidinehydrochloride4mgstreetvalue.blogspot.com
topcasinoratings.ru tizanidinestreetprice.blogspot.com
topsy.com tnctrx.com
traffic100.com topclickguru.com
trafficjunky.com topsiteminecraft.com
tubo360.com topvidos.ru
tupper-posuda.ru tracklead.net
turbodsp.com truck-rece.life
turizm.bz twodollarshows.com
typer.one ugogo.info
uac.net.au unitygame3d.com
umaseh.com
unitexindia.com
unlimitdocs.net unlimitdocs.net
urlopener.com unpredictable.ga
unrealcommander.biz
uptime.com
urll.eu
v24s.net v24s.net
vekzdorov.ru vavilone.com
vellings.info vduplo.ru
verymes.xyz veselokloun.ru
videochat.ph via-gra.webstarts.com
videosbox.ru video-chat.in
vipromoffers.com videochat.cafe
virus-respirators.com virus-respirators.com
vkmusics.ru vksaver-all.ru
vkonche.com vksex.ru
vvon.co.uk vod.com.ua
vysigy.su wallpaperaccess.com
webcamdevochka.com warezaccess.com
webjam.com weightatraining.com
website-datenbank.de welck.octopis.com
whatsappdownload10.com westsextube.com
wholesalejerseychinashop.com white-truck.life
witherrom55.eklablog.fr word-vorlagen.xyz
womensplay.net wordpress-crew.net
writingservices17.blogspot.ru works.if.ua
ww1943.ru worldinternetauthority.com
www.ehscloud.cn worldmusicfests.com
wwwadultcheck.com
x5market.ru x5market.ru
xchangetrak.com xjrul.com
xgames-04.com xn--hxazdsfy.blogspot.com
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xvideosporn.biz
xn--80aanaardaperhcem4a6i.com xxxmania.top
xn--80aodinpgi.xn--p1ai xxxtube69.com
xsion.net youjizz.com
xvideospornoru.com yubikk.info
zambini.ru zigzog.ru
znakom.sibtest.ru zryydi.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.8 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2503 ### Version: V4.2021.08.2504
### Updated: Thu Aug 5 10:03:32 UTC 2021 ### Updated: Thu Aug 5 22:01:18 UTC 2021
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.08.2503 ### Version: V4.2021.08.2504
### Updated: Thu Aug 5 10:03:32 UTC 2021 ### Updated: Thu Aug 5 22:01:18 UTC 2021
### Bad Bot Count: 618 ### Bad Bot Count: 618
################################################### ###################################################
### Version Information ## ### Version Information ##