vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-05 12:09:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.09.3371 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-09-25 10:09:03 +00:00
parent fb05aee86d
commit 92c43ce6e6
46 changed files with 96405 additions and 96405 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.09.3370 #### Version: V4.2022.09.3371
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.09.3370 #### Version: V4.2022.09.3371
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.09.3370 #### Version: V4.2022.09.3371
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11914
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.09.3370 #### Version: V4.2022.09.3371
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11918
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11918
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11918
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11918
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

11918
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

11918
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

11918
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3370 3371

11918
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11918
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.165.56.35 0; 20.12.6.215 0;

11918
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

11918
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

11918
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

11918
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.165.56.35 0; 20.12.6.215 0;

11918
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.165.56.35 0; 20.12.6.215 0;

11918
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 1; 10.1.0.10 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Bigfoot Boardreader
EyeNetIE G-i-g-a-b-o-t
Meanpathbot GrapeFX
Needle HMView
Netvibes Keyword Density
OpenLinkProfiler Libwhisker
Petalbot Nuclei
Suzuran OutclicksBot
WebCopier Spammen
internetVista monitor meanpathbot

308
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
ADmantX 360Spider
Abonti 404checker
AfD-Verbotsverfahren Aboundex
AhrefsBot Acunetix
Anarchy AiHitBot
Atomseobot Aipbot
Attach Alligator
AwarioSmartBot Ankit
Apexoo
BDCbot BDCbot
BackDoorBot
BackStreet BackStreet
Backlink-Ceck BackWeb
Bandit Badass
BatchFTP
BetaBot
Bigfoot Bigfoot
BlackWidow
Blackboard Blackboard
Boardreader Blow
Brandwatch BlowFish
BuiltBotTough Bolt
BuiltWith BotALot
Buck
Buddy
BunnySlippers BunnySlippers
CATExplorador
CCBot CCBot
CazoodleBot CODE87
Cegbfeieh Calculon
CheTeam
CherryPicker CherryPicker
Chlooe Chlooe
Cloud mapping Claritybot
Cocolyzebot Cliqzbot
Cogentbot Copier
CrazyWebCrawler CopyRightCheck
Crescent Crawling at Home Project
CrunchBot
Curious Curious
Custo
DIIbot
DSearch DSearch
DTS Agent DTS Agent
DataCha0s DataCha0s
Deusu
Digincore Digincore
DigitalPebble
Disco Disco
Dispatch DittoSpyder
DnyzBot DnyzBot
DomainCrawler Domains Project
Dotbot EasyDL
Download Wonder Ebingbong
Dragonfly
EMail Wolf
Ecxi Ecxi
EroCrawler EroCrawler
Evil Evil
Exabot ExtractorPro
Express WebPictures FemtosearchBot
Extractor Flunky
EyeNetIE Fuzz
FHscan FyberSpider
Fyrebot Fyrebot
G-i-g-a-b-o-t
GT::WWW
GalaxyBot GalaxyBot
GetRight Genieo
GetWeb
Getintent
Gigabot Gigabot
Go!Zilla Go!Zilla
GoZilla
Gotit
GrabNet
Grabber Grabber
Grafula
GrapeFX GrapeFX
HEADMasterSEO GrapeshotCrawler
HTTP::Lite
Haansoft
HaosouSpider HaosouSpider
Heritrix Heritrix
Humanlinks Hloader
IRLbot HonoluluBot
Image Sucker HybridBot
IndeedBot IlseBot
Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies
Intelliseek
Internet Ninja Internet Ninja
InternetSeer
Iskanie
IstellaBot IstellaBot
JOC Web Spider
JennyBot
JetCar JetCar
JikeSpider Jetty
Kenjin Spider Joomla
Jorgee
JustView
Jyxobot
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
LWP::Simple Keyword Density
Lanshanbot Lanshanbot
Larbin
LeechFTP LeechFTP
LeechGet LibWeb
LieBaoFast Libwhisker
LinkScan Lightspeedsystems
LinkWalker LinkWalker
Linkbot Linkbot
LinkextractorPro LinkextractorPro
LinkpadBot
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
Lipperhey Lipperhey
Litemage_walker
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc MIIxpc
MQQBrowser MJ12bot
MSFrontPage
Mag-Net
Magnet
Majestic SEO
Majestic12
MarkWatch
Mass Downloader Mass Downloader
Masscan Mata Hari
MauiBot Meanpathbot
Mb2345Browser Microsoft URL Control
MicroMessenger Mister PiX
Microsoft Data Access Moblie Safari
Minefield Mojeek
Mojolicious Mojolicious
Mr.4x3 MolokaiBot
NICErsPRO Mozlila
NearSite Musobot
NPbot
Name Intelligence
Needle
Net Vampire Net Vampire
NetLyzer NetSpider
NetZIP Netcraft
Niki-bot NimbleCrawler
Nikto
Nuclei Nuclei
Octopus Offline Navigator
Offline Explorer
Openfind Openfind
OrangeBot OrangeBot
OrangeSpider OutclicksBot
OutfoxBot POE-Component-Client-HTTP
PECL::HTTP PageAnalyzer
Page Analyzer
PageScorer
PageThing.com
Pandalytics Pandalytics
PeoplePal Pavuk
Petalbot Pi-Monster
PictureFinder Picsearch
Piepmatz Piepmatz
Pimonster
ProPowerBot ProPowerBot
Probethenet ProWebWalker
Psbot Psbot
Pu_iN Pu_iN
PxBroker PxBroker
PyCurl
QueryN Metasearch
Quick-Crawler Quick-Crawler
RSSingBot
RankActive
RankActiveLinkBot
RankFlex
RankingBot2 RankingBot2
RankurBot RankurBot
ReGet ReGet
RebelMouse
Recorder Recorder
RedesScrapy
Ripper Ripper
RocketCrawler
Rogerbot Rogerbot
SBIder
SEOkicks SEOkicks
SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats SISTRIX
ScreenerBot Scrapy
SentiBot Screaming
Shodan Searchestate
Site Sucker Semrush
SeoSiteCheckup
SeobilityBot
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker
Sitebeam Sitebeam
Siteimprove
Sitevigil Sitevigil
Snapbot Snake
SocialRankIOBot Sogou web spider
Sociscraper
Sosospider
SpaceBison SpaceBison
Spammen Spinn3r
Spanner Sqlmap
Sqlworm Steeler
Stripper
Sucuri
SuperHTTP
SurveyBot
Suzuran Suzuran
Swiftbot
T0PHackTeam
Teleport Teleport
TeleportPro
Telesoft
Telesphoreo
Telesphorep
The Intraformant
TheNomad TheNomad
Titan Toata
Tracemyfile Tracemyfile
Trendiction
True_Robot True_Robot
Typhoeus Turnitin
URLy Warning Twice
URLy.Warning
Upflow
V-BOT
VB Project VB Project
VCI VCI
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
Virusdie
VoidEYE VoidEYE
Voil
Voltron
WASALive-Bot WASALive-Bot
WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize
WWWOFFLE
Wallpapers
Wallpapers/3.0
WeSEE WeSEE
Web Collage
Web Enhancer Web Enhancer
Web Fetch Web Fetch
Web Pix
Web Sucker
WebEnhancer WebEnhancer
WebImageCollector WebGo IS
WebPix WebSauger
WebStripper
WebSucker WebSucker
WebWhacker
Webalta
WebmasterWorldForumBot
Webshag
Webster
Whack Whack
WinHTTrack Whacker
Whatweb
WiseGuys Robot
Wonderbot Wonderbot
Woobot Woobot
Wotbox Wotbox
Wprecon Xaldon WebSpider
ZoomBot YoudaoBot
Zade
ZoominfoBot ZoominfoBot
ZyBorg ZyBorg
adscanner adscanner
archive.org_bot archive.org_bot
arquivo.pt check1.exe
backlink-check
cah.io.community
coccocbot coccocbot
crawler4j com.plumanalytics
dataforseo.com crawler.feedback
gopher demandbase-bot
ips-agent domainsproject.org
eCatch
isitwp.com isitwp.com
linkdexbot mediawords
lwp-trivial
meanpathbot
muhstik-scan
netEstate NE Crawler
oBot oBot
page scorer pcBrowser
probe-image-size
s1z.ru s1z.ru
scan.lol seoscanners
seocompany.store
seostar seostar
sp_auditbot sexsearcher
tAkeOut sitechecker.pro
trendiction.com sogouspider
voyagerx.com x22Mozilla
webgains-bot
webmeup-crawler
xpymep1.exe

302
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
80legs
ADmantX
ASPSeek ASPSeek
Abonti Abonti
Aboundex
Aboundexbot Aboundexbot
Acunetix
AfD-Verbotsverfahren
AhrefsBot AhrefsBot
AiHitBot
AllSubmitter AllSubmitter
AlphaBot
Anarchie Anarchie
Anthill Anthill
Asterias Apexoo
Aspiegel
BDCbot BDCbot
BDFetch BDFetch
Badass BLEXBot
BatchFTP BackStreet
Battleztar Bazinga Backlink-Ceck
Barkrowler
Bigfoot Bigfoot
Bitacle Bitacle
Black Hole Black Hole
BlackWidow Blow
Boardreader Boardreader
Bolt
Brandprotect Brandprotect
Brandwatch Brandwatch
Buck BuiltBotTough
Buddy BuiltWith
BunnySlippers
BuzzSumo BuzzSumo
CCBot CATExplorador
CODE87
CSHttp CSHttp
Calculon Calculon
CazoodleBot CheeseBot
Cegbfeieh ChinaClaw
CensysInspect
CherryPicker
Chlooe Chlooe
Claritybot Cliqzbot
Cogentbot
Collector
CopyRightCheck
Cosmos
CrazyWebCrawler CrazyWebCrawler
Curious Crescent
Custo CrunchBot
DBLBot
DIIbot DIIbot
DSearch DSearch
DTS Agent DTS Agent
Deusu DataCha0s
Devil Devil
Digincore
DigitalPebble DigitalPebble
Dirbuster
Disco Disco
Discobot DnBCrawler-Analytics
DnyzBot DnyzBot
DomCopBot DomainSigmaCrawler
DomainCrawler Domains Project
Download Wonder Dotbot
Drip Drip
ECCP/1.0
EMail Wolf EMail Wolf
EasyDL
EirGrabber
Evil Evil
Exabot
Express WebPictures
Extractor Extractor
ExtractorPro EyeNetIE
Extreme Picture Finder Foobot
Firefox/7.0 Fuzz
Freeuploader Fyrebot
FrontPage
GT::WWW GT::WWW
GetWeb Genieo
Getintent GetRight
Gigabot Go!Zilla
GrabNet
Grabber Grabber
HEADMasterSEO GrapeFX
GridBot
HTTrack HTTrack
HaosouSpider
Havij Havij
HonoluluBot HybridBot
IDBTE4M IDBTE4M
IRLbot Image Fetch
Iblog Image Sucker
Indy Library Indy Library
Iskanie InfoTekies
IstellaBot InterGET
JOC Web Spider JamesBOT
Jetty Joomla
Jyxobot JustView
Keyword Density Kozmosbot
LWP::Simple Larbin
LeechGet LeechGet
LexiBot LexiBot
LibWeb Lftp
LinkextractorPro LieBaoFast
LinqiaRSSBot Likse
Lipperhey Linkbot
Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MQQBrowser MIDown tool
MSFrontPage MSFrontPage
MSIECrawler
MTRobot
Mag-Net Mag-Net
Magnet Majestic SEO
Mass Downloader Mass Downloader
Masscan MauiBot
Mata Hari Metauri
Mb2345Browser
MeanPath Bot
Meanpathbot
Microsoft URL Control Microsoft URL Control
Mojeek Mojeek
Musobot Mojolicious
NICErsPRO MolokaiBot
Nameprotect
Navroad
NearSite NearSite
Needle
Nessus Nessus
Net Vampire Net Vampire
NetAnts NetAnts
NetMechanic
NetSpider NetSpider
NetZIP NetZIP
Nettrack Nettrack
Netvibes Netvibes
NextGenSearchBot
Nibbler
Niki-bot Niki-bot
Nikto
Ninja Ninja
Nmap Nutch
Octopus
OnCrawl OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS OrangeBot
OrangeSpider
OutclicksBot OutclicksBot
PECL::HTTP
PHPCrawl
Page Analyzer Page Analyzer
PageThing.com
Panscient
Papa Foto Papa Foto
Picscout PeoplePal
Pi-Monster
PictureFinder PictureFinder
Piepmatz Piepmatz
Pixray PleaseCrawl
ProPowerBot Pockey
Pu_iN Pu_iN
PxBroker Quick-Crawler
PyCurl
QueryN Metasearch
RSSingBot
RankActive RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot
RankingBot2 RankingBot2
Rankivabot
Re-re
ReGet
RealDownload RealDownload
Reaper RedesScrapy
Ripper
RocketCrawler RocketCrawler
Rogerbot Rogerbot
SBIder
SEOkicks SEOkicks
SEOprofiler
SISTRIX
SalesIntelligent
Scanbot Scanbot
Scrapy ScreenerBot
SearchmetricsBot SearchmetricsBot
Seekport Seekport
SemanticJuice
Semrush
SeoSiteCheckup SeoSiteCheckup
SeobilityBot
Shodan Shodan
Siphon Site Sucker
SiteLockSpider Sitevigil
SiteSucker Snake
Siteimprove Sottopop
SmartDownload Spammen
Snapbot SpankBot
Sogou web spider SputnikBot
Spanner Sqlmap
Spinn3r Sqlworm
Steeler Sqworm
Stripper
Sucker
Sucuri
Surfbot
Suzuran Suzuran
T0PHackTeam Szukacz
T8Abot T8Abot
Teleport TeleportPro
TheNomad Telesoft
Thumbor The Intraformant
TightTwatBot Titan
Tracemyfile Toata
Trendiction Toweyabot
True_Robot Trendictionbot
Turingos
Turnitin
TurnitinBot TurnitinBot
TwengaBot UnisterBot
VCI
Vagabondo
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie
WASALive-Bot WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV WISENutbot
WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WallpapersHD
WWWOFFLE Web Auto
Wallpapers Web Collage
WeSEE
Web Enhancer Web Enhancer
Web Fetch Web Fuck
Web Sauger Web Pix
WebBandit Web Sucker
WebEnhancer WebCollage
WebFetch WebGo IS
WebImageCollector
WebLeacher WebLeacher
WebSauger WebPix
WebStripper WebStripper
WebWhacker Webshag
WebZIP
WebmasterWorldForumBot
Webster Webster
Whack Whack
Whacker Whacker
Widow
WinHTTrack WinHTTrack
Wonderbot WiseGuys Robot
Woobot
Wotbox
Wprecon Wprecon
Xaldon WebSpider
Xaldon_WebSpider Xaldon_WebSpider
Zauba Zauba
ZumBot Zitebot
ZmEu
ZoomBot
adscanner adscanner
archive.org_bot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt backlink-check
autoemailspider cah.io.community
coccocbot
cognitiveseo
com.plumanalytics com.plumanalytics
eCatch crawler4j
demandbase-bot
heritrix
instabid instabid
isitwp.com internetVista monitor
ips-agent
iubenda-radar
linkdexbot linkdexbot
meanpathbot lwp-request
netEstate NE Crawler pcBrowser
page scorer plumanalytics
polaris version ripz
probe-image-size s1z.ru
scalaj-http
scan.lol
seocompany.store
seostar seostar
sysscan serpstatbot
webgains-bot sitechecker.pro
webprosbot sp_auditbot
x09Mozilla ubermetrics-technologies.com
xpymep1.exe webmeup-crawler
zauba.io webpros.com
x22Mozilla
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
bio-optomarket.ru 6hopping.com
buy-meds24.com coolwallpapers4k.info
buytizanidineonline.blogspot.com googlemare.com
cubs-jerseys.us medosmotr-ufa.ru
domain.webkeyit.com offer.wpsecurity.website
labplus.ru pic-re.blogspot.com
muycerdas.xxx pricheskaonline.ru
picswe.com r-control.ru
tagil.zrus.org ros-ctm.ru
youandcredit.ru tivolibasket.it

940
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com 0lovespells0.blogspot.com
00it.com 1001watch.com.ua
0daymusic.org
101flag.ru 101flag.ru
24videos.tv 13tabs.com
24-ak.ru
2nt.ru 2nt.ru
3dracergames.com 2rich4bitches.com
5forex.ru 3-letter-domains.net
98oi.ru 4rent.ru
aa08daf7e13b6345e09e92f771507fa5f4.com 51.la
aa81bf391151884adfa3dd677e41f94be1.com 72-news.com
adloads.net 8lufu.com
adobereader-free.ru aa8780bb28a1de4eb5bff33c28a218a930.com
adtech.de aac500b7a15b2646968f6bd8c6305869d7.com
abiente.ru
academiacsmendoza.org
acortarurl.es
acunetix-referrer.com
adloads.com
admatic.com.tr
adultgalls.com
adultmeetonline.info
advertex.info
adviceforum.info adviceforum.info
afftrack001.com ahhjf.com
afslankpillen2017nl.eu albamargroup.com
agahinameh.com allcredits.su
aihelen.net allwomen.info
alif-ba-ta.com ameblo.top
allblogroll.com amt-k.ru
aloofly.com
am15.net
anabolics.shop anabolics.shop
ananumous.ru andpolice.com
animaltoplist.com anjalika.co.in
anonymousfox.co antons-transporte.de
api.stathat.com apartamentwroclaw.eu
apimountainbikei-a.akamaihd.net apibrowsesmartne-a.akamaihd.net
apioasisspacenet-a.akamaihd.net apifasterlightin-a.akamaihd.net
apioutoboxnet-a.akamaihd.net apptester.tk
apisecretsaucebi-a.akamaihd.net aquarium-pro.ru
apishops.ru artel116.ru
apiwebwebgetcom-a.akamaihd.net asseenontv.ru
app-ready.xyz auto-moto-elektronika.cz
apple.com-webbrowsing-security.science autovideobroadcast.com
architecturebest.com baixar-musicas-gratis.com
arendas.net balkanfarma.ru
artavenuegardenstudios.com banksrf.ru
artdestshop.eu bannerspace.com
asacopaco.tk bannertesting.com
atelielembrancaqueencanta.com.br bard-real.com.ua
auto-zapchasti.org battleforupdating.site
autoblogger24.info bavsac.com
autolombard-krasnodar.ru bayareaaeroclub.org
aviav.ru beatifullandscape.co
avkzarabotok.com beauty-things.com
avon-novosib.ru bedcapdealers.com
avon-ukraine.com begalka.xyz
azbukadiets.ru bestcoin.cc
babespcs.com bestkfiledxd.cf
baladur.ru
bankiem.pl
bbtec.net
bestcalovebracelet.cn
bestmarriages.com
bestwaystogetpaid.us
bestwrinklecreamnow.com
bet365.com bet365.com
bfz.biz betshuckclean.com
bitporno.sx bibys.com
bizlist.com.de bizzliving.com
bkns.vn bjorkbacken.nu
blackhatworth.com
blacktwhite.com
bloggerads.net
blogsfunky672.weebly.com
bluerobot.info bluerobot.info
bluesalt.co bouda.kvalitne.cz
bobba.dzaba.com
bocoarchives.org
bosefux.esy.es
bosman.pluto.ro
bpro1.top
brandedleadgeneration.com
break-the-chains.com break-the-chains.com
buckspar.gq bristolhotel.com.ua
buehne-fuer-menschenrechte.de brokergid.ru
bugof.gq bronzeaid-a.akamaihd.net
bumble.cheapwebsitehoster.com budpost.com.ua
buqayy0.livejournal.com bukleteg.xyz
burger-imperia.com buytizanidineonlinenoprescription.blogspot.com
buy-cheap-pills-order-online.com canacopegdl.com
buyantiviralwp.com candelluxsklep.pl
buyfriend.ru candy-glam-hp.com
buzz-porno.info cartierlovestore.com
call-of-duty.info cartiertrinity.xyz
callcafe.info casablancamanor.co.za
canoncdriverq3.pen.io ccjp.eu
canrole.com centrumcoachingu.com
carapics.com ciceros.co
cartierlove.xyz cleanmypc.com
cartierreplica.win clevernt.com
cartierring.xyz clickbank.net
cezartabac.ro clicksotrk.com
changde.58.com
chatroulette.online
checkpagerank.net
chocolateslim-en-france.com
choosecuisine.com
ciproandtizanidine.blogspot.com
classicakuhni.ru
cleanersoft.com
clickprotects.com
clksite.com
coaching-netz.info
cocolyze.com cocolyze.com
codysbbq.com cocyq.inwtrade.com
cognitiveseo.com compliance-andrew.xyz
coldfilm.ru compliance-brian.top
collegerentals.com compliance-irvin.xyz
companies-catalog.com compliance-josh.top
compiko.info compliance-julianna.top
compliance-ivan.top compliance-mary.top
compliance-margo.top conciergegroup.org
conocer-sanabria.com contentsexpress.com
converse.ddsoldes.fr
cool-wedding.net cool-wedding.net
coolingoods.com cordstrap.cc
coop-gamers.ru corta.co
coslab.club countbertwistdisp26.soup.io
cpabegins.ru csgo4.win
crafthubs.com customboxes4less.com
culmimg.pw customergrowthsystems.com
czat.wp.pl customerguru.in
darkages.info cymbaltaandtizanidine.blogspot.com
dataloading.net cyse.tk
ddpills.com daisye.top
ddsoldes.fr dealighted.com
dealitright.click dealitright.click
dear-diary.ru
delfin-aqua.com.ua delfin-aqua.com.ua
detskie-konstruktory.ru dentalpearls.com.au
dickssportinggoods.com deutschland123.de
dieswaene.com dev78.dioniqlabb.se
digilander.libero.it devochki.top
dimaka.info dieta-personalna.pl
dipstar.org dinosaurus.site
discountliv.com disruptingdinnerparties.com
discovertreasurenow.com dojki365.online
domashneeruporno.com dokumentalkino.net
domination.ml dollartree.info
dostavimvdom.ru domainsphoto.com
download-walpaperhd.blogspot.com donvito.unas.cz
downloadme.life doska-vsem.ru
dpihatinh.gov.vn downloaddy.net
drpornogratisx.xxx earn-from-articles.com
dscaas.website earthmagic.info
dtnlyss.com educatemetv.com
dverimegapolis.ru ege-russian.ru
e705.net ekto.ee
educontest.net election.interferencer.ru
ekobata.ru elektrischezigarette2.devhub.com
electrik-avenue.com elektrozigarette1.dreamwidth.org
eleimgo.pw elektrozigarettekaufen2.blogse.nl
elektrischezi.canalblog.com encodable.com
elektrischeziga.livejournal.com erotikchat-24.com
elektrozigarette2.wordpressy.pl errorfixing.space
enbersoft.com eu-cookie-law.info
englishgamer.com excitacion.info
eonpal.com expertnaya-ocenka.ru
ero-advertising.com ezigaretteshop2.mywapblog.com
erotik-kostenlos.net f07.de
esopini.com faithe.top
espaceinventoristes.com fandlr.com
etur.ru faracontrol.ir
europages.com.ru fickenprivat.info
eurosamodelki.ru fickkontaktehobbyhuren.com
everflora.ru filkhbr.com
excitacao.com filmbokep69.com
exlarseva.webblog.es financetip.eu
exportshop.us finejewelryshop.ru
extremez.net finuse.com
ezigarettenkaufen2.dreamwidth.org fix-website-errors.com
facialporntube.com
falopicm.pw
felizporno.com
fetlifeblog.com
fetroshok.ru
filesdatabase.net
filmidivx.com
find1friend.com
firesub.pl
flash4fun.com flash4fun.com
folowsite.com flavors.me
forensicpsychiatry.ru floridahuntingfishingadventures.com
forumprofi.de flowersbazar.com
foxinsocks.ru flowwwers.com
freza-sverlo.ru food.dtu.dk
funnel.co.za foodgid.net
fxund.us forex.osobye.ru
gameplexcity.com forexunion.net
gaming-journal.com formulaantiuban.com
gemgrab-a.akamaihd.net franch.info
ghazel.ru franklinfire.co
goodprotein.ru free-floating-buttons.com
googglet.com freevpn.space
google-liar.ru freewlan.info
gratis-sexkontakte.com freshwallpapers.info
gtopstats.com frivgame250.com
front.to
fuck-paid-share-buttons.xyz
fuckbuddybestgilf.info
fuel-gas.com
gallerily.com
game-top.su
gaz-voshod.ru
gazporno.com
getlamborghini.ga
girls-ufa.ru
glogow.pl
goforexvps.com
golfresa.lucania.se
goosefishpost.bid
gq-catalog.gq
graphicwe.org
greenidesign.co
grtyi.com
grupografico-pilar.com.ar
gsbs.com.ua
gunsvicceadadebt.tk
gz2.bbsoldes.fr
hannasolution.ru
happy.new.yeartwit.com
health-medical-portal.info health-medical-portal.info
hesteel.pl heartofbeijing.blogspot.com
hidemyass.com heliko.no
histats.com hem.passagen.se
hit-men.men hexpilot.com
hkdiiohi.skyrock.com highspeed5.net
hobbyhuren-datenbank.com hitmuzik.ru
hobbyhuren24.net hmmm.cz
honyaku.yahoofs.jp hopeonthestreet.co.uk
hotcore.info host-protection.com
hotshoppymac.com host-tracker.com
housekuba.org hostnow.men
hubbble.com hotblognetwork.com
huimin764128.com hotelcrocenzi.sm
igithab.com hully.altervista.org
igru-xbox.net iamsport.org
ikearugs.xyz iblogpress.xyz
impisr.ru ideawheel.com
intervsem.ru ie.57883.net
investpamm.ru imk.com.ua
iphantom.com interesnie-faktu.ru
iplogger.org intimshop-fantasy.ru
ispaniya-costa-blanca.ru invest-pamm.ru
iusstf.org investmac.com
jabimgo.pw irunfar.com
jennyfire.ru islandminingsupply.wordpress.com
jerseychinabizwholesale.us isotoner.com
jwss.cc jamiembrown.com
karachev-city.ru janavibekken.no
kenaba.su japfm.com
keywordsking.com jewelryandfiligree.com
kfon.eu karting196.ru
kidd.reunionwatch.com ketrzyn.pl
kimcurlrvsms.com keywords-monitoring-success.com
kiwe-analytics.com keywords-monitoring-your-success.com
klikbonus.com kichenaid.ru
kollekcioner.ru kino-rf.ru
komp-pomosch.ru kinohall.ru
konica.kz kinotorka.ru
lafourmiliaire.com kit-opt.ru
lafriore.ru kladrus.ru
landoftracking.com kndxbkdx.bloger.index.hr
lankarns.com kollesa.ru
letslowbefast.today kostenloser-sex.com
lion.conboy.us kupit-adenu.ru
livefixer.com kwzf.net
ljusihus.se ladsup.com
lol-smurfs.com laexotic.com
lombia.com laxdrills.com
lsex.xyz letmacwork.world
lubetube.com lietaer.com
lyngdalhudterapi.no link.ac
mactechinfo.info liveinternet.ro
mamasuper.prom.ua loadingpages.me
marmotstore.online lodki-pvh.dp.ua
masthopehomes.com loginduepunti.it
matb3aa.com lolnex.us
max-eclat.men lookingglassemb.com
medicovi.com luxmagazine.cf
megakino.net m-google.xyz
machicon-ueno.info
mafcards.ru
magicalfind-a.akamaihd.net
makenahartlin.com
makis.nu
managerpak204.weebly.com
mastershef.club
maxxtor.eu
mebeldekor.com.ua
mecash.ru
medmajor.ru
member-quiz.com member-quiz.com
middlerush-a.akamaihd.net metallosajding.ru
migente.com microsearch.ru
missvietnam.org misandesign.se
mobplayer.net mixed-wrestling.ru
mobicover.com.ua
mobstarr.com
mokrayakiska.com mokrayakiska.com
mompussy.net monarchfind-a.akamaihd.net
monclercheap.xyz monclerboots.xyz
moncleronline.xyz
mosdverka.ru
moskva.nodup.ru moskva.nodup.ru
mostantikor.ru mosrif.ru
mp3films.ru mrinsidesales.com
mrvideospornogratis.xxx mydearest.co
multgo.ru myfreemp3.eu
mundoaberrante.com myliveblog.ru
musflashtv.com na-telefon.biz
music7s.me napalm51.nut.cc
myghillie.info nardulan.com
myhealthcare.com natali-forex.com
myhitmp3.club
mypets.by
mysexpics.ru
mzdish.site
nailsimg.com
namenectar.com
narkologiya-peterburg.ru
naval.jislaaik.com naval.jislaaik.com
nero-us.com nibbler.silktide.com
newtechspb.ru ningessaybe.me
nfljerseysforsalewholesaler.com novatech.vn
nicefloor.co.uk nudepatch.net
novodigs.com nuker.com
nuit-artisanale.com o-o-6-o-o.ru
o-o-11-o-o.com obiavo.ru
o-o-8-o-o.ru odoratus.net
oballergiya.ru offer.camp
oconto.ru offside2.5v.pl
offers.bycontext.com oil-td.ru
offf.info okeinfo.online
officedocuments.net oklogistic.ru
okroshki.ru oneshotdate.com
olgacvetmet.com online-zaymy.ru
onclkads.com onlinewritingjobs17.blogspot.ru
online-sbank.ru onload.pw
onlinefilmz.net onlyporno.ru
openfrost.com ordernorxx.com
origin-my.ru ostroike.org
osnova3.ru outshop.ru
ossmalta.com paintingplanet.ru
ostrovtaxi.ru parajumpersjakkesalgnorge.info
oynat.info partnerworkroom.men
panchro.co.uk
partner-pop.men
partnersafe.men
patol01.pw
pavlodar.xkaz.org pavlodar.xkaz.org
pb-dv.ru perm-profnastil.ru
pelfind.me picturesmania.com
pertlocogasilk.tk pijoto.net
php-market.ru pinkduck.ga
picswe.com piratecams.com
pigrafix.at pix24x7.com
pinapchik.com pixshark.com
pipki.r.acdnpro.com pk-pomosch.ru
pivka.xyz pl.aasoldes.fr
pix-hd.com plugingeorgia.com
pizdeishn.com poiskzakona.ru
pochtovyi-index.ru polska-poezja.com
popcash.net poptool.net
popunder.ru popunder.net
poquoson.org porndroids.com
pornogratisdiario.com porno-video-chati.ru
pornohd1080.online porodasobak.net
pornoklad.net pourvous.info
pornolook.net preconnubial.usuby.site
praisong.net preventheadacheguide.info
priora-2.com privatbank46.ru
privacyassistant.net pro-okis.ru
producm.ru professionalwritingservices15.blogspot.ru
prohoster.info profitwithalex.info
promover.org ptr.ruvds.com
pronorm.fr purchasepillsnorx.com
prosperent.com push-ad.com
qexyfu.bugs3.com qcstrtvt.bloger.index.hr
r-control.ru ragecash.com
ranksays.com rankingchart.de
rapidsites.pro reelheroes.net
razvratnoe.org registrysweeper.com
realizmobi.com reimageplus.com
realmonte.net relatodelpresente.com.ar
registry-cleaner.net remontgruzovik.ru
rentalcarnavi.info rialp.getenjoyment.net
responsive-test.net ringtonepartner.com
rfd-split.hr rique.host.sk
richinvestmonitor.com robot-forex.biz
rocis.site
rocketchange.ru rocketchange.ru
roofers.org.uk roll123.com
rootandroid.org rossanasaavedra.net
ryetaw.com royalads.net
s8-nowy-wygraj.comli.com rubanners.com
saletool.ru rucrypt.com
santechnik.jimdo.com runstocks.com
russintv.fr
sady-urala.ru
sape.top
sarafangel.ru sarafangel.ru
savingsslider-a.akamaihd.net sarahmilne.top
sashagreyblog.ga
satellite.maps.ilovevitaly.com
sayyoethe.blogspot.co.za sayyoethe.blogspot.co.za
schlampen-treffen.com scanner-andrew.top
sdsjweb.com scanner-julianna.top
search.alot.com scanner-mark.top
secretscook.ru scanner-walter.top
seeingmeerkat.com securityallianceservices.com
sel-hoz.com seocdvig.ru
seo-platform.com sex-tracker.com
seoanalyses.com sex-watch.com
seokicks.de sexcamamateurchat.com
sex-tracker.de
sexad.net
sexkrasivo.net sexkrasivo.net
sexpornotales.com sexobzor.info
sexy-screen-savers.com sexvideo-sex.com
shareyards.com sharpchallenge.com
shop.xz618.com shikiso.info
shoplvlv.us shop.acim.org
shopwme.ru silktide.com
siha.de sim-service.net
sinel.info simplesite.com
sitebeam.net site-speed-checker.site
siteimprove.com siteexpress.co.il
sizeplus.work sizeplus.work
skutecznetabletkinaporostwlosow.pl skytraf.xyz
slayerlife.com slayerlife.com
slonechka.ru slkrm.ru
slutloadlive.com slutloadlive.com
sneakyboy.com
snworks.com snworks.com
soblaznu.net soc-proof.su
social-s-iii.xyz socialbookmarksubmission.org
soft-program.com soft1.ru
softomix.com sosdepotdebilan.com
solution4u.com spacash.com
sotechco.co spamnuker.com
sotkal.lark.ru
souvenir.cc
staff.prairiesouth.ca
starpages.net
starwars.wikia.com
stats-collector.org stats-collector.org
staynplay.net stillmiracle.com
steebook.com strigkaomsk.ru
stefanbakosab.se strongholdsb.ru
stjamesschool.info studiofaca.com
stoki.ru studworks.org
stop-zavisimost.com summerlinhomes411.info
stretchmate.net sumo.com
stromerrealty.com suture.co
strv.se swinger-mobil.net
sukarame.net sygraem.com
sunflowerdrawingpaintings.blogspot.com symbaloo.com
supernew.org szamponrevita.pl
surflinksmedical.com tattoo33.ru
syndicate.fun testingads.pro
tastyfoodideas.com theendivechronicles.com
tattomedia.com tizanidineinteractions.blogspot.com
tecspb.ru tizanidinesleepaid.blogspot.com
tedy.su tizanidineusedfor.blogspot.com
tehngr.ru tizanidinevssoma.blogspot.com
the-usa-games.blogspot.com
theautoprofit.ml
theguardlan.com
thepornsex.org
tizanidinecost.blogspot.com
tizanidinehcl2mg.blogspot.com
tjkckpytpnje.com tjkckpytpnje.com
tonerbox.kz topclickguru.com
top10-online-games.com torrentdownloadhub.com
top10-way.com totu.us
topads.men toyota.7zap.com
tpu.ru traffic-club.info
track112.site traffic2cash.org
trackmedia101.com trafficmp.com
tradgardspartner.se tube8.com
trainoffend.ml tubo360.com
tri-slona.org tuckpointingmasonrysystems.com
truck-rece.life tversvet.ru
tryrating.com tvorozhnaja-zapekanka-recept.ru
ttrraacckkrr.com twelvevisionspartyofcolorado.com
tubeline.biz twiclub.in
u17795.netangels.ru u-cheats.ru
uhod-za-sobakoj.ru uggbootsoutletsale.us
unisexjewelry.org uhdtv.website
unlimitdocs.net ukrtvir.com.ua
unrealcommander.org undergroundcityphoto.com
uogonline.com uptime-as.net
url-extractor.xyz uptime.com
usswrite.com urccvfmc.bloger.index.hr
validdomain.xyz us-america.ru
valoresito.com utrolive.ru
vandrie-ict.nl uytmaster.ru
vclicks.net vgoloveboli.net
vektorpress.ru video--production.com
veopornogratis.xxx vksaver-all.ru
via-gra.webstarts.com volgograd.xrus.org
videochat.world vrotike.ru
voloo.ru
vsdshnik.com vsdshnik.com
vucms.com vseigru.one
w-journal.ru w3data.co
w7s.ru warningzscaler.heraeus.com
webalan.ru watchmygf.net
webshoppermac.com webs.com
website-datenbank.de websitevaluebot.com
weclipart.com wedding0venues.tk
weekes.biz.tc wheelchairliftsarea.com
whatsappbot.flyland.ru whiteelephantwellington.com
where-toget.com wholinkstome.com
whereiskentoday.com williamrobsonproperty.com
whereverdesperate.gq wingsofrefuge.net
whipme.yopoint.in witclub.info
wholesalenfljerseys.us.com woman-h.ru
windowssearch-exp.com woman-orgasm.ru
withstandingheartwarming.com womensplay.net
womens-journal.net worldis.me
wordpresscore.com worldlovers.ru
world-mmo.com www.jbetting.com
wowas31.ucoz.ru x-true.info
wttavern.com xedserver.com
wufak.com xerox-douglas.cf
www.bookmaker-bets.com xfluro.com
www.ehscloud.cn xitjw.info
xclicks.net
xjlottery.com xjlottery.com
xn----itbkqkfiq.xn--p1ai xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn--1-8sbcpb0bdm8k6a.xn--p1ai xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xn--d1acah0c.xn--p1ai xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--l1aengat.xn--p1ai xn--80ahvj9e.xn--p1ai
xolodremont.ru xpresscare.ru
xrus.org y8games-free.com
xvideosbay.com yadro.ru
xvideosporn.biz yaminecraft.ru
xxlargepop.com yebocasino.com
yeartwit.com yhit.press
yes-com.com ynymnwbm.bloger.index.hr
youghbould.wordpress.com yourads.website
yubikk.info yourporngay.com
zahvat.ru yun56.co
zawyna.ua zagadki.in.ua
zeroredirect2.com zeroredirect12.com
zirondelli.it
znakomstva-piter78.ru
zolotoy-lis.ru
zrelaya.pw

484
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
123movies.love 0lovespells0.blogspot.com
2rich4bitches.com 0n-line.tv
55wmz.ru 123any.com
6hopping.com 2ads.co.uk
991mostfm.co.id 34.gs
aad1f4acb0a373420d9b0c4202d38d94fa.com 4rent.ru
aanapa.ru 72-news.com
abtasty.com 9med.net
acc.eu.org aa9bd78f328a6a41279d0fad0a88df1901.com
adidas.frwebs.fr adbetclickin.pink
adpremium.org adclickservice.com
afora.ru adobereader-free.ru
afslankpillen2017nl.eu adsfresh.men
allproblog.com adultfullhd.com
amehdaily.com adzerg.com
artdeko.info alessandraleone.com
aruplighting.com all4invest.ru
atofilms.com allsilver925.co.il
availit.weebly.com angel1777.kz
aviav.ru arendakvartir.kz
azartniy-bonus.com artparquet.ru
asmxsatadriverin.aircus.com
autobudpostach.club
babespcs.com
backgroundpictures.net
bad-stars.net
balla-boo.se
bank.uz bank.uz
beyan.host.sk bannerpower.com
birzha-truda.eu bayareaaeroclub.org
blogsfunky672.weebly.com berdasovivan.ru
bobinoz.com beremenyashka.com
brianhenry.co best-businessman.ru
businesxxl.com bet365.com
buy-cheap-online.info bio.trade-jp.net
cardsharp1.ru biznesrealnost.ru
cardul.ru bluejays-jerseys.us
carina-sy.de boazpower.com
carmuffler.net brimstonehillfortress.org
carsplate.com bristolhotel.com
casinos4dummies.co.uk buchananshardware.com
cheapestjerseysonwholesale.com budilneg.xyz
bumble.cheapwebsitehoster.com
buttons-for-website.com
c1.onedmp.com
cactussoft.biz
callcafe.info
candlewooddentalcentre.com.au
cartierreplica.xyz
cdnanalytics.xyz
christianlouboutinreplica.win
cinemaflix.website cinemaflix.website
citizenclsdriveri7.pen.io
clevernt.com
click2pawn.com
clickintext.net
clickso.com
clickzzs.nl
cognitiveseo.com
compliance-george.top
compliance-walter.top compliance-walter.top
contentsexpress.com cookie-law-enforcement-ff.xyz
counciltally.com cosmetic.donna7753191.ru
cranly.net craftinsta.ru
cutt.us dailyrank.net
cymbaltaandtizanidine.blogspot.com dairyindia.in
daooda.com darodar.com
dealwifi.com deadlinkchecker.com
deutschland123.de dignitasdata.se
dimaka.info discovertreasurenow.com
djonwatch.ru dispo.de
dneprsvet.com.ua dms-sw.ru
dogoimage.com doeco.ru
eandsgallery.com dollartree.info
earthmagic.info downloader12.ru
eduinfosite.com downloadkakaotalk.com
elvel.com.ua drupa.com
enge-fotzen.info electrik-avenue.com
escort-russian.com elektrozigarettekaufen2.blogse.nl
etm-consult.de elkacentr.ru
euronis-free.com erotag.com
everflora.ru erotik-kostenlos.net
evrotekhservis.ru essayassist.com
extremal-blog.com extremepornos.net
falcoware.com facebook-mobile.xyz
familienzahnaerzte.com facecup.top
fanoboi.com filmci.pro
fanpagerobot.com firstsiteguide.com
farmprofi.net flprog.com
fast-torrent.ru folowsite.com
favornews.com forum.poker4life.ru
forums.d2jsp.org forumprofi.de
frankofficial.ru foto-basa.com
free-share-buttons.com free-social-buttons.xyz
freejabs.com freelifetimefuckbook.com
freelinkbuilding.website.tk freeuploader.com
front.ru fugarif.ga
gambarkataku.co futbolkisales.ru
geoads.com galleryawesome.com
getlaid-xxxhookupdirect.com genericviagrasildenafiled.net
ggiaro.com gonextmedia.com
glavtral.ru gotorussia.com
googlefeud.com greenzaim.ru
gratuitxblcodes.com guarrasdelporno.xxx
green-tea.tv healthcarestore.info
grizzlysgrill.com hebr.myddns-flir.com
guardlink.com hermesbracelets.xyz
gwagka.com hosting-tracker.com
hdimagegallery.net hotel-mkad.ru
hdpixent.com house.sieraddns.com
herehloadibs.cf houston-vikings.com
highland-homes.com i-midias.net.br
honyaku.yahoofs.jp iboss.com
houdom.net igrovyeavtomaty777.ru
houseofrose.com ihc2015.info
howlongdoestizanidinestayinyoursystem.blogspot.com ikritikimou.gr
hybrid.ru images-free.net
idealtits.net impisr.ru
innoslicon.com incanto.in.ua
instabid.tech infostatsvc.com
itbc.kiev.ua irkutsk.online-podarki.com
izhevsk.zrus.org isoveti.ru
javspace.net joy-penguin.com
jerseywholesaleelitestore.com khafre.us
johnrobertsoninc.com kino-rating.ru
jumptap.com kliksaya.com
kenaba.su kopihijau.info
kerwinandcariza.com kostenlos-sexvideos.com
kinomaniatv.pl kupiproday.com.ua
kollekcioner.ru lasvegaslockandsafe.com
konkursov.net leftofcentrist.com
koronirealestate.gr lennatin.info
kretpicf.pw liumimgx.pw
kw21.org ljusihus.se
labelwater.se machicon-akihabara.info
ladsblue.com marmitaco.cat
lakomka.com.ua meet-flirt-dating.com
lapitec.eu meetlocalchicks.com
lashstudia.ru mesbuta.info
layola.biz.tc michaelkorsoutlet.store
loadingpages.me mikozstop.com
louboutinreplica.pw mobioffertrck.com
m292.info mobplayer.ru
magicalfind-a.akamaihd.net monarhs.info
megaapteka.ru moomi-daeri.com
michellblog.online moscow-clining.ru
mir-limuzinov.ru mrinsidesales.com
mistr-x.org music.utrolive.ru
mobifunapp.weebly.com mybackgroundlandscape.blogspot.com
mostantikor.ru mydearest.co
na-telefon.biz northfacestore.online
natali-forex.com ntic.fr
netallergy.ru nuup.info
novgorod.xrus.org offers.bycontext.com
obiavo.kz office-windows.ru
oecnhs.info omgtnc.com
oivcvx.website onlainbesplatno.ru
onko-24.com online-hit.info
onlineserialy.ru online-sbank.ru
open-odyssey.org onlineporno.site
openmediasoft.com osb.se11.ru
partnerads.men outpersonals.com
parvezmia.xyz
pathwhelp.org
pechikamini.ru
pekori.to pekori.to
picurams.pw pharmacyincity.com
pl-vouchers.com piccshare.com
pojdelo.weebly.com poisk-zakona.ru
porno-raskazy.ru pons-presse.com
pornohub.me popander.mobi
pornzone.tv porn-w.org
porodasobak.net porngalleries.top
pp-budpostach.com.ua presleycollectibles.com
printingpeach.com prizestohandle.club
privatamateure.com pronorm.fr
prom23.ru prostitutki-astana.org
pskcijdc.bloger.index.hr prostitutki-rostova.org
push-ad.com realresultslist.com
qitt.ru redhotfreebies.co.uk
r-e-f-e-r-e-r.com remont-komputerov-notebook.ru
rarbg.to remorcicomerciale.ro
rcb101.ru renewablewealth.com
remmling.de retreatia.com
rfd-split.hr rimedia.org
rocketchange.ru robot-forex.biz
rockingclicks.com
rockprogblog.com rockprogblog.com
royalvegascasino.com rucrypt.com
rusoft-zone.ru rutor.vip
saddiechoua.com sarf3omlat.com
sape.top scalerite.co.za
scanner-barak.top sdrescher.net
scanspyware.net searchaddis.com
scmor.ilxc.cc searchmywindow-a.akamaihd.net
seogadget.ru serdcenebolit.com
seojokes.net servethis.com
seozoom.it sexsaoy.com
serptehnika.ru share-buttons-for-free.com
seryeznie-znakomstva.ru shareyards.com
sex-foto.pw shoppingmiracles.co.uk
sexsearch.com siteexpress.co.il
sextracker.de skylta.com
shiksabd.com soc-proof.su
shtora66.ru spidtest.org
site-auditor.online
site3.free-share-buttons.com
skatestick.bid
skyway24.ru
slowmac.tech
slowmacfaster.trade
smokewithrabbits.com
snworks.com
soft-program.com
solitaire-game.ru
sotkal.lark.ru
spasswelt.net
sport-video-obzor.ru sport-video-obzor.ru
stanthonyscatholicchurch.org spy-sts.com
stroydetali.ru superlist.biz
supervesti.ru superoboi.com.ua
sygraem.com svensk-poesi.com
t-bygg.com swsociety.se
tako3.com syndicate.fun
texbaza.by tam-gde-more.ru
thaismartloan.com tasteidea.com
tizanidineorflexeril.blogspot.com taximytishi.ru
topshef.ru teencastingporn.com
trkdf.com the-trader.net
truemfilelj.gq thebluffs.com
tizanidine4mg.blogspot.com
top1-seo-service.com
torontoplumbinggroup.com
toursmaps.com
trackmedia101.com
tracksz.co
trudogolik.net
trustedmaccleaner.com trustedmaccleaner.com
trustl.life u17795.netangels.ru
tubeoffline.com
turizmus.us
tvory.predmety.in.ua
ufa.xrus.org
ugogo.info ugogo.info
ukrobstep.com uk-zheu20.ru
ural-buldozer.ru ukrtextbook.com
uvozdeckych.info umaseh.com
valoresito.com urll.eu
valsalud.com usbggettwku.ga
videochat.ph vandrie-ict.nl
vigrx-original.ru varbergsvind.se
villakohlanta.nu virtualbb.com
vipsexfinders.com vksaver-all.ru
vitoriacabos.com vladhistory.com
vseigru.one volgograd.xrus.org
vtcdns.com voloslove.ru
vzlomfb.com volume-pills.biz
wholesalecheapjerseysfree.com vvpg.ru
wieseversa.no wedding0venues.tk
windowssearch-exp.com where-toget.com
wingsoffury2.com whos.amung.us
worldis.me witherrom55.eklablog.fr
www.labves.ru ww1943.ru
x-diesel.com xaylapdiendanang.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai xn----itbeirbjbi7bc6bh2d.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xn--80aodinpgi.xn--p1ai
xn----ctbbcjd3dbsehgi.xn--p1ai xxx-treker.ru
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai yoshkarola.zrus.org
yellocloud.be you-shall-not-pass.is74.ru
yoluxuryevents.com youandcredit.ru
yoopsie.com zeroredirect11.com
youbloodyripper.com zixizop.net.ru
yougame.biz znakomstva-piter78.ru
zahvat.ru zoogdinsney.com
zatjmuzu.info zvetki.ru

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.26 0; 10.1.0.10 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.09.3370 ### Version: V4.2022.09.3371
### Updated: Sat Sep 24 22:01:23 UTC 2022 ### Updated: Sun Sep 25 10:03:18 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.09.3370 ### Version: V4.2022.09.3371
### Updated: Sat Sep 24 22:01:23 UTC 2022 ### Updated: Sun Sep 25 10:03:18 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##