vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 11:10:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.11.3501 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-11-30 10:09:20 +00:00
parent c613a6920a
commit 62499543cc
46 changed files with 92244 additions and 92244 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3500 #### Version: V4.2022.11.3501
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.11.3500 #### Version: V4.2022.11.3501
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.11.3500 #### Version: V4.2022.11.3501
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11394
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.11.3500 #### Version: V4.2022.11.3501
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11398
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11398
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11398
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11398
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 1; 10.1.0.162 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 0; 10.1.0.162 0;
127.0.0.1 0; 127.0.0.1 0;

11398
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 1; 10.1.0.162 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 0; 10.1.0.162 0;
127.0.0.1 0; 127.0.0.1 0;

11398
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 1; 10.1.0.162 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 0; 10.1.0.162 0;
127.0.0.1 0; 127.0.0.1 0;

11398
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3500 3501

11398
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11398
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.168.157.165 0; 20.237.190.202 0;

11398
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 1; 10.1.0.162 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 0; 10.1.0.162 0;
127.0.0.1 0; 127.0.0.1 0;

11398
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 1; 10.1.0.162 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 0; 10.1.0.162 0;
127.0.0.1 0; 127.0.0.1 0;

11398
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 1; 10.1.0.162 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 0; 10.1.0.162 0;
127.0.0.1 0; 127.0.0.1 0;

11398
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.168.157.165 0; 20.237.190.202 0;

11398
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.168.157.165 0; 20.237.190.202 0;

11398
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 1; 10.1.0.162 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BDFetch BlackWidow
GermCrawler CCBot
GetWeb ExtractorPro
HTMLparser Humanlinks
Intelliseek MSFrontPage
Mb2345Browser Microsoft URL Control
Nettrack Morfeus Fucking Scanner
RealDownload WallpapersHD
WWW::Mechanize com.plumanalytics
lwp-trivial tAkeOut

318
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
ADmantX 01h4x.com
404checker
AIBOT
ALittle Client
Abonti
Aboundexbot
Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot Aipbot
AiHitBot AlphaBot
Anarchy99 Anarchie
Atomseobot Anthill
BBBike Attach
BLEXBot BDCbot
BDFetch
BackStreet BackStreet
BackWeb Backlink-Ceck
BacklinkCrawler
Badass Badass
Barkrowler
BatchFTP BatchFTP
Bigfoot Battleztar Bazinga
Boardreader BlackWidow
Brandprotect Blackboard
Brandwatch BotALot
BuiltBotTough Buck
BunnySlippers Bullseye
CCBot Calculon
CazoodleBot Cegbfeieh
CheeseBot
ChinaClaw ChinaClaw
Citoid Citoid
Cocolyzebot
Cogentbot
Collector Collector
Copier Copier
Copyscape CopyRightCheck
Cosmos Cosmos
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler
Crescent
Curious
Custo
CyotekWebCopy CyotekWebCopy
DBLBot DBLBot
DSearch
DTS Agent DTS Agent
Deusu DataCha0s
Devil
DigitalPebble DigitalPebble
Disco Discobot
DittoSpyder DomainCrawler
Domains Project
Download Wonder
Dragonfly Dragonfly
Drip
ECCP/1.0 ECCP/1.0
EMail Siphon Ebingbong
EirGrabber Ecxi
Extreme Picture Finder EroCrawler
FDM Exabot
Extractor
ExtractorPro
Ezooms
FHscan FHscan
FemtosearchBot FemtosearchBot
Fimap Firefox/7.0
Foobot
Freeuploader Freeuploader
FrontPage
Fuzz
FyberSpider
Fyrebot
GalaxyBot
GermCrawler
GetRight GetRight
GetWeb
Getintent Getintent
Gigabot Gigabot
Go!Zilla GrabNet
Gotit
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
HTTrack Haansoft
Harvest HaosouSpider
Havij
HonoluluBot HonoluluBot
Humanlinks
IDBot
Iblog
Image Fetch Image Fetch
IndeedBot
InfoNaviRobot
InternetSeer InternetSeer
IstellaBot Iria
JamesBOT
Jbrofuzz Jbrofuzz
Joomla JennyBot
Kinza JustView
Kozmosbot Keybot Translation-Search-Machine
LWP::Simple LWP::Simple
Lanshanbot Leap
Larbin LieBaoFast
Lftp LinkextractorPro
Likse LinkpadBot
LinkScan LinqiaRSSBot
LinkWalker
Linkbot
Lipperhey Lipperhey
Lipperhey Spider
Lmspider
Ltx71 Ltx71
MIDown tool MIIxpc
MQQBrowser MJ12bot
MSFrontPage
MSIECrawler MSIECrawler
MTRobot MTRobot
Mag-Net
Magnet
Majestic SEO Majestic SEO
MarkMonitor Majestic-SEO
MarkWatch
Mass Downloader Mass Downloader
Masscan
Mata Hari
Mb2345Browser Mb2345Browser
MeanPath Bot
Metauri Metauri
Microsoft URL Control Microsoft URL Control
Minefield Mozlila
Moblie Safari Msrabot
Morfeus Fucking Scanner Nameprotect
Musobot Nessus
NICErsPRO NetAnts
NPbot NetSpider
Name Intelligence
NetMechanic
NetZIP NetZIP
NextGenSearchBot Nettrack
Nikto Nikto
NimbleCrawler NimbleCrawler
Ninja Not
Nmap
Nuclei
Octopus Octopus
Offline Explorer
Offline Navigator
OnCrawl OnCrawl
OpenVAS
OrangeSpider
OutclicksBot OutclicksBot
POE-Component-Client-HTTP PECL::HTTP
Page Analyzer PageAnalyzer
PageGrabber
PageThing.com PageThing.com
Pandalytics Pandalytics
Papa Foto Papa Foto
PeoplePal Pavuk
Petalbot Pi-Monster
PictureFinder Picscout
Pixray Picsearch
Pimonster
ProPowerBot ProPowerBot
ProWebWalker ProWebWalker
Probethenet
Psbot Psbot
Pu_iN QueryN Metasearch
Quick-Crawler
RSSingBot RSSingBot
RankingBot RankActive
RankingBot2 RankingBot2
Rankivabot Rankivabot
RankurBot RankurBot
Re-re
RealDownload RealDownload
Ripper
RocketCrawler RocketCrawler
SBIder SBIder
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SEOprofiler
SISTRIX SEOstats
Scanbot SalesIntelligent
Scrapy ScanAlert
ScoutJet
ScreenerBot
ScrepyBot
Seekport
Semrush Semrush
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup
Seomoz
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider SiteSnagger
SiteSucker Sitebeam
SlySearch Sitevigil
SmartDownload Snake
Sociscraper Snapbot
SocialRankIOBot
Sosospider Sosospider
Sqlmap SpankBot
Sqlworm Spanner
Sqworm Spbot
Spinn3r
Stripper
Sucker Sucker
SuperBot SuperBot
SurveyBot
Suzuran
Swiftbot Swiftbot
Szukacz T0PHackTeam
T8Abot T8Abot
Teleport
Telesoft Telesoft
Telesphoreo TheNomad
Thumbor Tracemyfile
Titan TwengaBot
Toata
Toweyabot
Trendiction
True_Robot
Turnitin
Typhoeus
URLy Warning
UnisterBot UnisterBot
Upflow
V-BOT
VB Project
VCI
Vacuum
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VidibleScraper
VoidEYE Virusdie
Voltron WEBDAV
WBSearchBot WWW-Collector-E
WWW::Mechanize
WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0
WallpapersHD
WeSEE WeSEE
Web Auto Web Auto
Web Collage
Web Enhancer
Web Fetch Web Fetch
Web Sauger Web Fuck
WebCollage Web Sucker
WebAuto
WebCopier
WebFetch WebFetch
WebFuck
WebImageCollector
WebLeacher
WebReaper
WebSauger
WebWhacker WebWhacker
WebZIP WebZIP
Webalta WebsiteExtractor
Webshag Whacker
Website Quester
WebsiteQuester
Whatweb Whatweb
Who.is Bot Who.is Bot
Widow
Wonderbot
Woobot Woobot
Wotbox
Wprecon Wprecon
Xaldon_WebSpider Zade
YoudaoBot Zauba
ZoomBot Zermelo
ZoominfoBot Zitebot
ZumBot ZmEu
adscanner arquivo-web-crawler
archive.org_bot arquivo.pt
autoemailspider backlink-check
cognitiveseo cah.io.community
clark-crawler
crawl.sogou.com crawl.sogou.com
crawler.feedback
crawler4j crawler4j
dataforseo.com
demandbase-bot
domainsproject.org
evc-batch evc-batch
facebookscraper heritrix
gopher
instabid instabid
internetVista monitor internetVista monitor
linkdexbot iubenda-radar
lwp-request meanpathbot
mediawords netEstate NE Crawler
pcBrowser oBot
polaris version
ripz
s1z.ru s1z.ru
scan.lol satoristudio.net
seobility seocompany.store
serpstatbot
sexsearcher sexsearcher
sogouspider sogouspider
spyfu
sysscan
trendiction.com trendiction.com
trendiction.de webgains-bot
ubermetrics-technologies.com webmeup-crawler
webprosbot webpros.com
x09Mozilla x09Mozilla
xpymep1.exe x22Mozilla

288
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy 404enemy
ADmantX 80legs
AIBOT AIBOT
ALittle Client ALittle Client
ASPSeek ASPSeek
Abonti
Aboundexbot Aboundexbot
Acunetix Acunetix
AhrefsBot AhrefsBot
AllSubmitter AiHitBot
Alexibot
Anarchy Anarchy
Anthill Anthill
Aspiegel Asterias
Attach Attach
AwarioRssBot BBBike
BDCbot
BDFetch
BackStreet
BackWeb BackWeb
Bandit BacklinkCrawler
Bitacle Barkrowler
Black Hole Blackboard
Blow Boardreader
Bolt
BotALot BotALot
Brandwatch Brandwatch
Buddy
BuiltWith BuiltWith
Bullseye
BuzzSumo BuzzSumo
CODE87 CATExplorador
CazoodleBot CCBot
Cegbfeieh Cegbfeieh
ChinaClaw CensysInspect
Cliqzbot CheTeam
Collector CherryPicker
Crawling at Home Project Chlooe
Citoid
Claritybot
Cloud mapping
Cocolyzebot
CopyRightCheck
Cosmos
Craftbot
Custo
CyotekWebCopy
DIIbot
DTS Agent DTS Agent
DatabaseDriverMysqli DataCha0s
Deusu
Devil Devil
Digincore Digincore
DigitalPebble DigitalPebble
Dirbuster
Discobot Discobot
DnyzBot
DomCopBot DomCopBot
DomainCrawler DomainAppender
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Dotbot Dotbot
Dragonfly Download Wonder
EMail Wolf ECCP/1.0
EroCrawler EMail Siphon
Ebingbong
Evil
Exabot Exabot
ExtLinksBot
Extractor Extractor
ExtractorPro ExtractorPro
EyeNetIE
FHscan FHscan
Firefox/7.0 Flunky
Freeuploader Freeuploader
FrontPage FrontPage
Fuzz
FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GetRight GT::WWW
GermCrawler
GetWeb GetWeb
Getintent
Gigabot Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet GoZilla
GrapeFX Haansoft
GrapeshotCrawler HaosouSpider
HTMLparser Havij
Hloader Hloader
HonoluluBot
Humanlinks
IRLbot IRLbot
Iblog Image Sucker
IndeedBot IndeedBot
Indy Library InfoNaviRobot
InterGET Intelliseek
Internet Ninja Internet Ninja
JOC Web Spider IstellaBot
Joomla Jbrofuzz
Jorgee Jorgee
Jyxobot Jyxobot
Keyword Density
Kinza Kinza
Kozmosbot LNSpiderguy
LWP::Simple LWP::Simple
Larbin
Leap
Lftp Lftp
Lightspeedsystems Libwhisker
Likse LinkScan
Linkbot Linkbot
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Litemage_walker Lmspider
Ltx71
MFC_Tear_Sample
MJ12bot
MQQBrowser MQQBrowser
MSIECrawler MSFrontPage
MTRobot
Majestic12 Majestic12
Microsoft URL Control MarkWatch
Mass Downloader
Masscan
Mb2345Browser
Mediatoolkitbot
MegaIndex.ru
MicroMessenger
Mister PiX
Moblie Safari Moblie Safari
Mojolicious Mozlila
Mr.4x3 Mr.4x3
Msrabot Msrabot
NICErsPRO
Nameprotect
Navroad Navroad
Net Vampire
NetLyzer
NetMechanic NetMechanic
NetSpider
Netcraft Netcraft
Nettrack Nettrack
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler Nibbler
Nikto
NimbleCrawler
Nimbostratus
Ninja Ninja
Nmap
Not Not
Nuclei
Nutch Nutch
Offline Explorer Octopus
Offline Navigator OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
OrangeBot
OrangeSpider OrangeSpider
Page Analyzer
PageAnalyzer
PageGrabber PageGrabber
PageScorer PageThing.com
Pandalytics Pandalytics
Papa Foto
Pavuk Pavuk
Pixray Picsearch
Pockey PictureFinder
ProWebWalker ProWebWalker
Probethenet Pump
Pu_iN
PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler RankActive
RankActiveLinkBot Rankivabot
RankFlex Re-re
RankurBot
Reaper Reaper
RebelMouse Recorder
Ripper Rogerbot
RocketCrawler SBIder
SEOkicks SEOkicks-Robot
SEOprofiler SEOprofiler
SEOstats SMTBot
SISTRIX ScanAlert
SalesIntelligent Scanbot
Scrapy Scrapy
Screaming Screaming
ScreenerBot ScrepyBot
Seekport Seekport
SemanticJuice SemrushBot
SentiBot SentiBot
SeobilityBot SeoSiteCheckup
Siphon Shodan
SiteCheckerBotCrawler Site Sucker
SiteExplorer
SiteSnagger
SiteSucker SiteSucker
Sitebeam Sitebeam
Siteimprove
SlySearch SlySearch
Snake
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
Sogou web spider Sogou web spider
Sottopop SpankBot
Spbot Spbot
Spinn3r
SputnikBot SputnikBot
Steeler Steeler
Sucker Stripper
Sucuri SuperBot
SuperHTTP Surfbot
SurveyBot
Swiftbot Swiftbot
T8Abot Szukacz
T0PHackTeam
Telesphoreo Telesphoreo
Telesphorep Telesphorep
The Intraformant The Intraformant
TheNomad
Thumbor Thumbor
TightTwatBot
Toata Toata
Trendiction Toweyabot
Trendictionbot
Turingos Turingos
Turnitin Turnitin
Twice Typhoeus
URLy Warning URLy.Warning
V-BOT Upflow
VB Project VoidEYE
Vacuum WASALive-Bot
VeriCiteCrawler WBSearchBot
Voltron WEBDAV
WISENutbot Wallpapers
WWW-Collector-E WallpapersHD
WWW-Mechanize
WWW::Mechanize
Web Collage
Web Enhancer
Web Fetch
Web Fuck Web Fuck
WebAuto Web Pix
WebBandit WebCopier
WebLeacher WebWhacker
WebPix Webalta
WebReaper WebsiteExtractor
WebSauger WebsiteQuester
Webshag Who.is Bot
Website Quester
Widow
WinHTTrack
WiseGuys Robot
Wonderbot Wonderbot
Wprecon Woobot
Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot YoudaoBot
ZmEu Zauba
ZyBorg Zitebot
arquivo.pt ZoomBot
backlink-check autoemailspider
coccocbot
cognitiveseo cognitiveseo
domainsproject.org crawler.feedback
gopher gopher
heritrix
instabid instabid
internetVista monitor
ips-agent ips-agent
isitwp.com
iubenda-radar iubenda-radar
lwp-request linkdexbot
lwp-trivial magpie-crawler
meanpathbot
mediawords mediawords
netEstate NE Crawler netEstate NE Crawler
oBot
plumanalytics plumanalytics
satoristudio.net ripz
scan.lol
seocompany.store seocompany.store
serpstatbot seostar
sp_auditbot sexsearcher
trendiction.de sitechecker.pro
sogouspider
ubermetrics-technologies.com
voyagerx.com voyagerx.com
webgains-bot webmeup-crawler
x09Mozilla webprosbot
x22Mozilla
zauba.io
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
acheterviagrafr24.com dekoration.us
adconscious.com eldiariodeguadalajara.com
celebsopics.com elektrozigarette1.dreamwidth.org
compliance-donald.xyz goldenggames.com
edwinkonijn.com.au googst2.ru
feorina.ru hazardky.net
flowersbazar.com moxo.com
oz-offers.com naperehresti.info
spywarebegone.com oxotl.com
track112.site xn--1-8sbcpb0bdm8k6a.xn--p1ai

938
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
03p.info 007angels.com
20pascals.nl 00author.com
24videos.tv 1-99seo.com
2ads.co.uk 123movies.love
4kpics.rocks 12u.info
8lufu.com 1webmaster.ml
5tey7463.weebly.com
6hopping.com
8si.ru
9med.net 9med.net
aad1f4acb0a373420d9b0c4202d38d94fa.com a.frcls.fr
add-add.men aa625d84f1587749c1ab011d6f269f7d64.com
adinterax.com aa8780bb28a1de4eb5bff33c28a218a930.com
adprotect.net aa9bd78f328a6a41279d0fad0a88df1901.com
adtech.us aanapa.ru
advokateg.xyz abclauncher.com
adzpower.com abiente.ru
abovetherivernc.com
admatic.com.tr
afftrack001.com
afora.ru afora.ru
aktobe.xkaz.org albuteroli.com
alekseevec.ru allblogroll.com
alert-fdm.xyz allcredits.su
alevinefesleri.com
allmarketsnewdayli.gdn
alphacarolinas.com
alta-realestate.com alta-realestate.com
amateurgalls.com alvtank.se
android-vsem.org amateurlivechat.org
apartment.ru amoi.tn
apessay.com anal-acrobats.hol.es
apibrowsesmartne-a.akamaihd.net angkortours.vn
apple.com-webbrowsing-security.science apimegabrowsebiz-a.akamaihd.net
apptester.tk apioutoboxnet-a.akamaihd.net
arate.ru appartement-stumm.at
arenanews.com.ua apps-analytics.net
articlesdirectoryme.info appsecurityr.com
astrochicks.com apxeo.info
atdedinotuho.tk ardimobileinfo.ml
availit.weebly.com arius.tech
avkzarabotok.info artel116.ru
avon-severozapad.ru asscenihotosidea.blogspot.com
avtolombard-krasnodar.com autoblogger24.info
azartniy-bonus.com autodo.info
azinoofficial777.ru aviapanda.ru
azlex.uz aviav.co
backlink4u.net avirasecureserver.com
bamo.xsl.pt awency.com
banki76.ru babosas.com
beauty-bracelet.com bagcionderlab.com
best-seo-solution.com balitouroffice.com
bestofferswalkmydogouteveryday.gq banan.tv
bestpriceninja.com bausparen.bz.it
besttorrentknifta.weebly.com beclean-nn.ru
bestwebsitesawards.com bestcurs.org
bet-prognoz.com betune.onlinewebshop.net
bezlimitko.xyz bigames.online
bg6s0.com bigcities.org
bibys.com bingo8888.com
big-cash.net biznesschat.net
bioscorp.ru blackhatworth.com
bizlist.com.de blog.remote-computer.de
blackplanet.com
bloggedporn.com bloggedporn.com
blogs.rediff.com blogig.org
bmw.afora.ru blogos.kz
boeuklad.com bolegapakistan.com
boleznikogi.com bombla.org
bolezniorganov.ru bonky.biz
bongacams.com boobsimge.com
brandehk.dk bouda.kvalitne.cz
brateg.xyz brendbutik.ru
brus.city brk-rti.ru
buehne-fuer-menschenrechte.de burkesales.com
bupropion-sr-150-mg.us bus-offer.com
buscarfoto.com business-made-fun.com
buyscabiescream.com butstrap.space
buzzonclick.com bvlgaribracelet.xyz
cah.io.community calcularpagerank.com
canrioloadm.gq calvet.altervista.org
camsex.xxx-cam.webcam
candytiner.com
canrole.com
canvas.gloverid.site
cardiosport.com.ua
cardul.ru
carina-sy.de
carrauterie.be carrauterie.be
carsdined.org cartierbraceletsreplica.pw
cartechnic.ru cartierlovebracelet.xyz
cartierreplica.pw casite-513049.cloudaccess.net
cdnnetwok.xyz cbozhe.com
cgi2.nintendo.co.jp cdncash.com
cheapwebsitehoster.com cheapjerseysfootballshop.com
christianlouboutinoutlet.win cheapmoncler.pw
cjmc.info cheapmoncler.win
clean-start.net chiptuninger.com
clicksor.com christianlouboutinreplica.win
christianlouboutinsaleoutletonline.us
citysecurity.nu
clipartnew.com clipartnew.com
cognitiveseo.com club-samodelkin.ru
compliance-andrew.xyz co.lumb.co
compliance-ivan.top com-onlinesupport.host
consorzioilmosaico.org commoncrawl.org
cornerstone-countertops.com compliance-andrew.top
cracksplanet.com compliance-barak.xyz
craftburg.ru compliance-brian.top
cymbaltaandtizanidine.blogspot.com compliance-jane.top
cyprusbuyproperties.com compliance-john.top
daymusam.com compliance-josh.top
ddpills.com coolyarddecorations.com
deanmoore.ie copblock.org
degerlund.net counciltally.com
delitime.info country-chic.ru
demon-tweeks.com cryptoswap.biz
detskie-zabavi.ru cs-passion.pl
dgfitness.co cupday.com
diarioaconcagua.com customcollegeessays.net
discountliv.com darkages.info
docs4all.com dating-app.ru
dodlive.mil davidovic.info
dogoimage.com dcj-nn.ru
domain-tracker.com decenttools.com
domain2008.com deda-moroza-zakaz.ru
dostavka-v-krym.com deplim.com
druzhbany.ru detmebel.su
druzhininevgeniy63.blogspot.com dev78.dioniqlabb.se
duplicashapp.com digital-video-processing.com
dvr.biz.ua diusyawiga.tk
e-avon.ru djihispano.com
e-commerce-seo1.com domaincheck.io
e-poker-2005.com domashneeruporno.com
ecookna.com.ua downloaddy.net
elektrischezigarette2.devhub.com doyouknowtheword-flummox.ml
eloxal.ru drivotracker.com
eredijovon.com drunkmoms.net
erotikstories.ru dvervmoskvu.ru
erotische-geschichten-xxl.com e-biznes.info
essay-edu.biz e-kwiaciarz.pl
essayservicewriting.org e90post.com
eurosamodelki.ru egovaleo.it
evaashop.ru ekaterinburg.xrus.org
event-tracking.com elektrischezigarette1.onsugar.com
exchanges-bet.com elmacho.xyz
exci.ru energydiet24.ru
exhibitionplus.eu enews.tech
exoclick.com envaseslotusama.com
exportshop.us es5.com
extrabot.com escortplius.com
ezigarettenkaufen2.dreamwidth.org espaceinventoristes.com
f-loaded.de estibot.com
f07.de etur.ru
f0815.de eugenevaultstorage.com
facecup.top europages.com.ru
fakehandbags.xyz extractorandburner.com
fashion.stellaconstance.co ez8motelseaworldsandiego.com
fellowshipoftheminds.com ezaz.info
femdom.twiclub.in ezigarettekaufen2.blox.pl
fermersovet.ru fantasticpixcool.com
fillmewithhappiness.com farm26.ru
findsexguide.com fashion-mk.net
flashbannernow.com fastfixing.tech
flipper.top feeriaclub.ru
fortevidyoze.net fickluder69.com
fortunejack.com filesdatabase.net
frameimage.org filesvine.com
froggytube.com financeloan.us
funnel.co.za fishingwholesale.us
flex4launch.ru
floating-share-buttons.com
forms-mtm.ru
forodvd.com
fotopop.club
freelifetimefuckbook.com
fuckingawesome.com
fun-mobi.pl
fxtips.ru
fyl.com.ru
g33.org g33.org
galeria-zdjec.com gavuer.ru
galleryawesome.com gay-file.com
game-mmorpg.net genericviagrasildenafiled.net
gameprimary.com getresponse.com
garciniaxt.us girlsfuckdick.com
gay.adultgalls.com glopages.ru
generic-pills-online.com googlepositions.com
gerhardhealer.com googst2.ru
getmyads24.com
ggiaro.com
ghostvisitor.com
given2.com
globatur.ru
glogow.pl
gne8.com
gogps.me
goldandcard.ru
goodly.pro
goodprotein.ru
gosreg.amchs.ru
gox.com.ua gox.com.ua
greatdealshop.com green-tea.tv
gsmtlf.ru gta-top.ru
guarrasdelporno.xxx guod.me
gunsvicceadadebt.tk hasfun.com
gwebtools.com hd720kino.ru
gwebtools.com.br hdimagelib.com
hao123.com herokuapp.com
hdhc.ru himgaws.pw
hermesreplica.win
homedecoguide.info homedecoguide.info
hoverboard360.at hotel-mkad.ru
hoverboard360.de house.sieraddns.com
huhn.altervista.org howopen.ru
huimin764128.com hyiphunter.org
hybrid.ru igadgetsworld.com
i-service.kz igrovyeavtomaty777.ru
idc.com.ua imagez.co
imfamous.info img.wallpaperstock.net
imgata.com imperia31.ru
inboundlinks.win insta-add.pro
inboxdollars.com interesnie-faktu.ru
incest-ru.com
infogame.name
interfucks.net interfucks.net
investpamm.ru intervsem.ru
iphantom.com iskalko.ru
irkutsk.online-podarki.com istizanidineanopiate.blogspot.com
itch.io ivanstroi.ru
izhevsk.xrus.org jimmychoosale.online
izhevsk.zrus.org jobgirl24.ru
jacago.com joessmogtestonly.com
jerseywholesalebizchina.com jofucipiku.tk
joy-penguin.com jrcigars.com
justprofit.xyz junglenet-a.akamaihd.net
jwss.cc
jyvopys.com
kaac.ru kaac.ru
kalandranis.gr kabbalah-red-bracelets.com
kamalsinha.com kam-dom.ru
karusel-market.ru kamorel.com
kazan.xrus.org karaganda.xkaz.org
keyhantercume.com keywordteam.net
kirov.zrus.org khadastoafarde.tk
kit-opt.ru kicknights.gq
kino2018.cc
kinobest.pl
kinogolos.ru
kinostorm.net
kleine-titten.biz
kongoultry.net
kongruan.com kongruan.com
konoplisemena.com kostenloser-sex.com
kovesszucs.atw.hu kountrylife.com
kozhniebolezni.com krasnodar.zrus.org
krafte.ru layola.biz.tc
krasnodar.ru lemon-ade.site
kredit-pod-zalog-krasnodar.ru
kvartira-sutochno.com
laapp.com
lacloop.info
ladov.ru
lafourmiliaire.com
landinez.co
larger.io
lazymanyoga.com
lenpipet.ru lenpipet.ru
liffa.ru lida-ru.com
light.ifmo.ru lider-zhaluzi.kiev.ua
linerdrilling.com linksharingt.com
linkbux.ru littleberry.ru
liran-locks.com littlesexdolls.com
loadingpages.me live-sexcam.tk
loadopia.com loadopia.com
locatellicorretor.com.br longgreen.info
lomb.co ltvperf.com
lunchrock.co lyrics.home-task.com
m0r0zk0-krava.ru mainlinehobby.net
magazin-pics.ru maju.bluesalt.co
manifestation.betteroffers.review makis.nu
mapquestz.us manyresultshub-a.akamaihd.net
masturbate.co.uk map028.com
mature.free-websites.com marcowebonyodziezowe.pl
maxxtor.eu matb3aa.com
meddesk.ru medtherapy.ru
mediafresh.online megaapteka.ru
metallosajding.ru meltwater.com
meteocast.net merryhouse.co.uk
middlerush-a.akamaihd.net metaxalonevstizanidine.blogspot.com
mielec.pl mikozstop.com
mikrobiologies.ru mikrobiologies.ru
militarysale.pro
mindeyegames.com mindeyegames.com
minharevisao.com minharevisao.com
mir-business-24.ru minyetki.ru
mixtapetorrent.com mir-betting.ru
mmofreegames.online mockupui.com
moesen-ficken.com moivestiy.biz
monetizer.com-01.site moneyteam24.com
money-for-placing-articles.com monthlywinners.com
monsterdivx.com moomi-daeri.com
moscow.xrus.org moroccosurfadventures.com
moskva.nodup.ru morpicert.pw
mowser.com moscow-region.ru
mydownloadengine.com mug-na-chas-moscow.ru
myphotopipe.com musicas.baixar-musicas-gratis.com
muz-baza.net
mwtpludn.review
mxgetcode.com
mybackgroundlandscape.blogspot.com
mybinaryoptionsrobot.com
mydirtyhobby.com
mydoctorok.ru
mylesosibirsk.ru
mzdish.site mzdish.site
na15.ru nacap.ru
nagdak.ru nalogovyykodeks.ru
news-readers.ru narco24.me
newstaffadsshop.club narkologiya-penza.ru
nfvsz.com net-radar.com
nightvision746.weebly.com newstraveller.ru
nikhilbahl.com newtechspb.ru
niroo.info nextrent-crimea.ru
norththeface.store nonews.co
novatech.vn november-lax.com
novosibirsk.xrus.org nufaq.com
novosti-hi-tech.ru nuup.info
nowtorrents.com o-o-8-o-o.com
nudejapan.net
nullrefer.com
o-dachnik.ru
o-o-8-o-o.ru
obesidadealgarve.com obesidadealgarve.com
obiavo.kz offer.wpsecurity.website
ochistka-stokov.ru offf.info
odoratus.net okeinfo.online
of-ireland.info oneclickfiles.com
offside2.5v.pl onlineserialy.ru
okel.co onthemarch.co
olympescort.com openfrost.com
oneminutesite.it optibuymac.com
online-zaymy.ru oralsexfilme.net
openmediasoft.com orgasmatrix.com
optitrade24.com ownshop.win
osb.se11.ru ozas.net
ourtherapy.ru pandroid.co
outrageousdeal-a.akamaihd.net pearlisland.ru
pacificair.com pelfind.me
panchro.co.uk perfectplanned.com
panicatack.com picswe.com
panouri-solare-acoperis.com pipki.r.acdnpro.com
partners-ship.pro piulatte.cz
pcimforum.com pl-vouchers.com
perm-profnastil.ru plastgran.com
perosan.com play-mp3.com
peugeot-club.org poisk-zakona.ru
picture-group.com pokemongooo.ml
picturesify.com pons-presse.com
pillscheap24h.com popup-fdm.xyz
pinkduck.ga popup-jdh.xyz
pix24x7.com pornhub-forum.ga
pizdopletka.club pornhubforum.tk
plastweb.ru porno-play.net
podshipniki-nsk.ru porno-transsexuals.ru
poligon.com potoideas.us
popugauka.ru preventheadacheguide.info
pornosee.info princevc.com
porodasobak.net printdirectforless.com
powitania.pl printie.com
predominant-invent.tk priora-2.com
produkto.net prostitutki-almata.org
promodj.com prostitutki-belgoroda.org
pron.pro pst2017.onlinewebshop.net
pulseonclick.com purchasepillsnorx.com
pussysaga.com pussyfleet.com
putitin.me quickbuck.com
q-moto.ru quizzitch.net
quangcaons.com qwarckoine.com
quickcashlimited.com
qwertty.net qwertty.net
rapevideosmovies.com rada.ru
rating-bestcasino.com ranksays.com
recipedays.com ranksignals.com
redirect.trafficreceiver.club ranksonic.org
refererx.com recordpage-a.akamaihd.net
reining.lovasszovetseg.hu regionshop.biz
remont-ustanovka-tehniki.ru registratciya-v-moskve.ru
renewablewealth.com remont-komputerov-notebook.ru
responsinator.com remontvsamare.su
rixpix.ru ribieiendom.no
romhacking.ru rockingclicks.com
rockma.se
rogervivierforsale.com
rollercoin.com
rome2rio.com
rosbalt.com.ua rosbalt.com.ua
rossanasaavedra.net rospromtest.ru
rusoft-zone.ru russia-today-video.ru
rvtv.ru rutor.group
sack.net sady-urala.ru
saletool.ru sarafangel.ru
samara.rosfirm.ru satellite.maps.ilovevitaly.com
sbornik-zakonov.ru sbetodiodnye-lampy.ru
sbwealthsolutions.ca scanner-irvin.top
seoprofiler.com scansafe.net
service.adtech.us scrapy.org
servingnotice.com screentoolkit.com
seadragonherbery.com
seeingmeerkat.com
semaltmedia.com
sex-tracker.com
sexkontakte-seite.com
sexkrasivo.net sexkrasivo.net
sexy.babes.frontend-stack.top sexpornotales.com
shemalegalls.blogporn.in shmetall.com.ua
shoesonlinebuy.cn shoesonlinebuy.cn
shoplvlv.us shopfishing.com.ua
shopperifymac.com sideeffectsoftizanidine.blogspot.com
sicfor.bcu.cc sientalyric.co
siha.de sitebeam.net
sinel.info siteripz.net
site-analyzer.com sladkoevideo.com
skanninge.se slonechka.ru
skladvaz.ru smart-scripts.com
slkrm.ru snegozaderzhatel.ru
slowmacfaster.trade snjack.info
slujbauborki.ru socialbutton.xyz
smartadserver.com soft-program.com
smartpet.ru soheavyblog.com
smokewithrabbits.com soochi.co
snowplanes.com spinazdrav.ru
social-button.xyz spy-sts.com
social-search.me steelmaster.lv
socialbookmarksubmission.org storehouse.ua
socialbuttons.xyz stpicks.com
socialmadesimple.com streetfire.net
socialseet.ru strigkaomsk.ru
softtor.com stroymonolit.su
solitaire-game.ru strv.se
spbchampionat.ru students-cheapskate.ml
spywareit.com success-seo.com
spywarespy.com suchenindeutschland.com
stal-rulon.ru superinterstitial.com
steamoff.net supermesta.ru
streamin.to svbur.ru
stretchingabuckblog.com swiped.su
strongsignal-a.akamaihd.net szamponrevita.pl
stuffpride.com tattooha.com
sundrugstore.com techart24.com
superfish.com tecnoteakviareggio.it
superlist.biz the-torrent-tracker.blogspot.com
supermama.top themeforest.net
svetodiodoff.ru themestotal.com
svolze.com thepantonpractice.co.uk
syndicate.fun timdreby.com
tam-gde-more.ru titangel-vietnam.com
taximytishi.ru titslove.yopoint.in
teenfuck.tv tizanidine4mgstreetvalue.blogspot.com
teguh.info tizanidine6mg.blogspot.com
telefonsexi.com tizanidinecost.blogspot.com
telegraf.by tizanidinezanaflex.blogspot.com
terraclicks.com top-study.work
terrafootwear.us topads.men
theallgirlarcade.com topappspro.com
thefds.net topkarkas.com
timetorelax.biz topquality.cf
tizanidinedrugtest.blogspot.com tourismvictoria.com
tizanidineduringpregnancy.blogspot.com tracklead.net
tizanidinefibromyalgia.blogspot.com trafficstars.com
tizanidinenarcotic.blogspot.com traffique.net
tmm-kurs.ru
tnctrx.com
toursmaps.com
toxicwap.com
tracxn.com
trafficmonetize.org
traffixer.com traffixer.com
traxdom.ru traxdom.ru
trichizobswiv.agddns.net tri-slona.org
trk-4.net tuckpointingmasonrysystems.com
trubywriting.com tula.howotorg.ru
trustl.life twitlinks.com
turbo-suslik.org uchil.net
tvand.ru ufa.xrus.org
tvnewsclips.info uggsale.online
twittruth.com uhdtv.website
tyumen.xrus.org upupa.net
uggbootsoutletsale.us urengoy.pro
uhodzalijami.ru usiad.net
ukrtextbook.com vapeface.club
umaseh.com
umnovocaminho.com
unacittaconte.org
underthesite.com
uprour.com
uptime-as.net
vancleefreplica.pw
viagengrarx.com
videochat.bz videochat.bz
videochaty.ru
videos-for-your-business.com
videotuber.ru
viel.su
vielporno.net
vinsit.ru
visitcambridge.org visitcambridge.org
vita.com.hr
vkak.ru vkak.ru
voluumtracker1.com vote-up.ru
voprosotvet24.ru vpnhowto.info
vpnmouse.com vse-pesni.com
vrnelectro.ru wahicbefa31.soup.io
vroze.com wallpapers-best.com
vseuznaem.com warezkeeper.com
wallpapersdesk.info watch-movies.ru
warning.or.kr webcamtalk.net
watchdogs-2.ru website-audit.com.ua
watracker.net
web-betting.ru
website-speed-up.top website-speed-up.top
whereverdesperate.gq wemarketing.se
wma-x.com whatsappbot.flyland.ru
worldhistory.biz whatsappdownload10.com
whosonmyserver.com
wildnatureimages.com
winner7777.net
womensplay.net
worldinternetauthority.com worldinternetauthority.com
www.kabbalah-red-bracelets.com wowcasinoonline.ooo
www.pinnacle-bets.com wpthemedetector.co.uk
www.wohnkabinen-shop.de wrz0iuebwhp5fg.freeddns.com
x-diesel.info wygraj-teraz.com
xblognetwork.com x-mix.info
xep.info xchat26.myfreecams.com
xingzi-vision.com xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn----dtbndd4ae7eub.top xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai xn--80ab4aa2g.xn--p1ai
xn--80atua3d.xn--p1ai xn--h1ahbi.com.ua
xtraffic.plus xn--l1aengat.xn--p1ai
xtremeeagles.net xportvusbdriver8i.snack.ws
xvideosporn.biz xsfetish.org
xzlive.com xvideosbay.com
yaoguangdj.com yaoguangdj.com
youandcredit.ru youradexchange.com
youjizz.vc yuarra.pluto.ro
yugk.net zakonobosago.ru
yunque.pluto.ro zaobao.com.sg
zaloadi.ru zarabiaj-dzis.pl
zeg-distribution.com zeroredirect5.com
zixizop.net.ru zmoda.hostreo.com
zocaparj.kz zona-aqua.ru
zygophyceous.womanstars.site zoogdiesney.com
zskdla.site
zvezdagedon.ru
zvooq.eu

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 11pikachu.ru
08800.top 14b.info
12masterov.com 24videos.tv
178evakuator178.ru 3w24.com
5kstore.com 6hopping.com
8si.ru 7yue.org
aab94f698f36684c5a852a2ef272e031bb.com aaa38852e886ac4af1a3cff9b47cab6272.com
abclauncher.com abcguru.xyz
abwa.tk add-add.men
actices.com adktrailmap.com
admeasures.com affordablewebsitesandmobileapps.com
advertisingtag.net afftrack001.com
aero2.ru akuhni.by
aibolita.com alessandraleone.com
alldezservice.kz alienwheels.de
amazingpic.net allbizne.co.ua
anal-acrobats.hol.es alloysteel.ru
apibatbrowsecom-a.akamaihd.net amateurlivechat.org
apishops.ru apartmentbay.ru
aproposde.com apiadanaknet-a.akamaihd.net
astana.xxxkaz.org apparel-offer.com
audiobangout.com apper.de
autoplate.info arenanews.com.ua
aviva-limoux.com asiavirtualsolutions.com
avkzarabotok.com atmovs.com
azazu.ru azartclub.org
azbuka-mo.ru babs.com.ua
azinoofficial777.ru baikaleminer.com
backgroundpictures.net bfz.biz
bagcionderlab.com bidbuy.co.kr
baladur.ru bloglag.com
balans.shahterworld.org bloke.com
bannerbank.ru bronzeaid-a.akamaihd.net
basedecor.ru buy-meds24.com
batanga.net buypillsorderonline.com
bif-ru.info bycontext.com
bin-brokers.com carsplate.com
bouda.kvalitne.cz carstrends2015.com
brainzpod.win cerev.info
btvn.ru cezartabac.ro
business-made-fun.com charmstroy.info
buy-forum.ru cheapestjerseys-wholesale.com
buyessaynow.biz check-host.net
buyk.host.sk chocolateslim-u-srbiji.com
buyscabiescream.com cleanpcnow.com
carfax.com.ua compliance-elena.top
cbb1smartlist12.click compliance-ivan.top
celebsopics.com compliance-mark.top
cheapsergiorossi.online connectionstrenth.com
clickcash.com copyrightclaims.org
coccoc.com cpabegins.ru
collegerentals.com culturevie.info
customboxes4less.com custodieva.ru
dickssportinggoods.com custom-product-labels.com
dirtpics.pw customerpromos-a.akamaihd.net
div.as datsun-do.com
doctissimo.fr dekoration.us
draniki.org diamond-necklace.info
drillsaw.ru download-wallpaper.net
drunkenstepfather.com edge.sharethis.com
dstroy.su eduinfosite.com
dtm-spain.com emergencyneeds.org
elexies.info escort-russian.com
englate.com eshop4u.jp
eupornstar.info eurosamodelki.ru
everytalk.tv extractorandburner.com
exonline.info farmprofi.net
f5mtrack.com fatfasts-4tmz.com
falopicm.pw favornews.com
fantasticpixcool.com fefo.gdn
felizporno.com fetroshok.ru
ferretsoft.com fidelityfunding.com
filmi-onlain.info footballfarrago.com
flowersbazar.com forexunion.net
footbalive.org foto-sisek.porngalleries.top
freegamesplay.online free-social-buttons.com
freza-sverlo.ru free-traffic.xyz
gael-s.ru freesitetest.com
gg.zzyjxs.com frvo.alptandem.ru
girlporn.ru fuckingawesome.com
gnuetella.com funnymama.com
greatdealshop.com gigapeta.com
gsasearchenginerankerdiscount.com gosmeb.ru
gwagka.com gq-catalog.gq
honyaku.yahoofs.jp gsbs.com.ua
hoverboard360.at gwebtools.com
hvd-store.com handicapbathtubarea.com
hydropump.su hasfun.com
ie.57883.net hdimagegallery.net
imgarcade.com hermesbracelets.xyz
indianmedicaltourismshop.com herokuapp.com
investmac.com hikesearch.net
irkutsk.zrus.org hol.es
iscblog.info howlongdoestizanidinestayinyoursystem.blogspot.com
ivoiretechnocom.ci hspline.com
jav-fetish.com iboss.com
idc.com.ua
ihc2015.info
imagerydatabase.com
in-tandem.co
incanto.in.ua
inclk.com
innoslicon.com
investsuccess.org
ivearchenceinflu.cf
jennyfire.ru
jeremyeaton.co
jerseywholesalebizchina.com jerseywholesalebizchina.com
journalhome.com jetsli.de
justdating.online jofucipiku.tk
kakablog.net keyhantercume.com
keywesthideaways.co kidskunst.info
keywordhut.com kiinomaniak.pl
keywordsuggest.org kino2018.club
kinky-fetishes.com kleine-titten.biz
kinocccp.net kvartiry-remont.ucoz.ru
klikbonus.com lapitec.eu
knowsitall.info lerporn.info
kol-energo.ru
kollekcioner.ru
konpax.com
kovesszucs.atw.hu
krasnodar-avtolombards.ru
krasnodar.zrus.org
laexotic.com
lernur.net
lexiangwan.com lexiangwan.com
lifeinsurancekb.com livesex-amateure.info
liquimondo.com loftdigital.eu
lonfon.xyz
longadventure.com longadventure.com
lotto6888.com lookingglassemb.com
love-baby.cz lrac.info
lovelycraftyhome.com mac-shield.com
lyrster.com macnewtech.com
m292.info magicalfind-a.akamaihd.net
m5home.ru maju.bluesalt.co
magicdiet.gq matsdale.com
marketingvici.com meccadumps.net
mb140.ru medizinreisen.de
medanestesia.ru metalonly.info
meendo-free-traffic.ga
microsearch.ru
mini-modus.ru mini-modus.ru
mir-limuzinov.ru mir-business-24.ru
mojpregled.com mistr-x.org
moneyviking-a.akamaihd.net mitrasound.ru
morlat.altervista.org mobifunapp.weebly.com
moscow.online-podarki.com moroccosurfadventures.com
mturkcontent.com mossmesi.com
muizre.ru moyakuhnia.ru
mundoaberrante.com mp3ritm.top
mybackgroundlandscape.blogspot.com mriyadh.com
myfreemp3.eu musezone.ru
myfreetutorials.com mydeathspace.com
mylesosibirsk.ru myra.top
myshopmatemac.com nalogovyykodeks.ru
mzdish.site netoil.no
naj-filmy24.pl nikitabuch.com
najaden.no nl.netlog.com
namecrumilchlet.tk nofreezingmac.work
ningessaybe.me nordstar.pro
noclegonline.info nottyu.xyz
orenburg-gsm.ru november-lax.com
pandarastore.top obnal.org
panouri-solare-acoperis.com onemactrckr.com
parlament.biz online247.ml
paydayonlinecom.com oralsexfilme.net
pc-test.net origin-my.ru
photokitchendesign.com oynat.info
picswe.com palvira.com.ua
pl.id-forex.com parfusale.se
plaff-go.ru partnerworkroom.men
pliks.pl perfectpracticeweb.com
pornhive.org perosan.com
porno-dojki.net phelissota.xyz
pornoinn.com piccshare.com
powitania.pl picscout.com
pozvonim.com piratecams.com
prchecker.info plusnetwork.com
prizefestival.mobi pogodnyyeavarii.gq
prointer.net.ua pornokorol.com
promover.org pornoslive.net
pukaporn.com portal-eu.ru
purplesphere.in pp-budpostach.com.ua
pussyspace.net predmety.in.ua
questionmarque.ch printdirectforless.com
rankexperience.com privatbank46.ru
rarbg.to prosperent.com
ratemodels.net q-moto.ru
recordpage-a.akamaihd.net quebec-bin.com
registry-clean-up.net quit-smoking.ga
rixpix.ru ranksays.com
rybalka-opt.ru reklamuss.ru
s8-nowy-wygraj.comli.com remont-mobile-phones.ru
sa-live.com remote-dba.de
sa-rewards.co.za rethinkwasteni.info
sanidumps.com rockingclicks.com
scanner-barak.top rootandroid.org
scanner-donald.top s-forum.biz
scanner-fred.top samolet.fr
schlampen-treffen.com sashagreyblog.ga
sdsjweb.com scalerite.co.za
semxiu.com segol.tv
seo-platform.com semalt.com
seobility.net sextracker.be
serpstat.com sherlock.se
sexphoto.site shoplvlv.us
sexuria.net shoppingjequiti.com.br
sexy-screen-savers.com shopvilleroyboch.com.ua
sexy.babes.frontend-stack.top sibdevice.ru
site3.free-share-buttons.com sideeffectsoftizanidine.blogspot.com
sk.golden-praga.ru sitebeam.net
smart-scripts.com sitevaluation.com
social-s-hhh.xyz skytraf.xyz
socialmediasuggest.com slavic-magic.ru
solution4u.com sotkal.lark.ru
sortthemesitesby.com sport7777.net
sptslmtrafms.com stroy-portal22.ru
steelmaster.lv sudexpert66.ru
strongholdsb.ru superkanpo.com
studiofmp.com swinginwithme.ru
sugvant.ru tackletarts.co
takethatad.com thenetinfo.com
taqywu51.soup.io thetoiletpaper.com
taranerymagesswa.blogspot.com
tecnoteakviareggio.it
testbotprocessor44.com
token-lab.org
topclickguru.com
torrentred.games torrentred.games
torrents-tracker.com toyota.7zap.com
traderzplanet.in trafficmania.com
trafficmonetize.org traffique.net
trahvid.com trailer.cinemaflix.website
tuberkuleznik.ru
turbo-suslik.org
twittrading.com twittrading.com
underthesite.com udsgame.online
unitexindia.com unionmarkt.de
vadimkravtcov.ru uprour.com
valoresito.com utrolive.ru
valueclick.com valmetrundan.se
viagroid.ru viagra-soft.ru
videochat.life videokrik.net
visa-china.ru videosbox.ru
visitcambridge.org vip-dom.in
watchmygf.net vksex.ru
webcamdevochka.com volgograd.xrus.org
wesharepics.info voprosotvet24.ru
westen-z.life vsdshnik.com
westum.se warezkeeper.com
where-toget.com wdfdocando.com
whiteproduct.com wejdz-tu.pl
wingsoffury2.com westen-v.life
winwotgold.pl wildnatureimages.com
wordpress-crew.net writingservices17.blogspot.ru
wormix-cheats.ru www.labves.ru
wrona.it xjlottery.com
x-diesel.org xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xn--80aanaardaperhcem4a6i.com
xn----itbkqkfiq.xn--p1ai xn--80ahdheogk5l.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai xxx-cam.webcam
xn--d1acah0c.xn--p1ai zaloadi.ru
zoompegs.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.218 0; 10.1.0.162 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3500 ### Version: V4.2022.11.3501
### Updated: Tue Nov 29 22:01:14 UTC 2022 ### Updated: Wed Nov 30 10:03:29 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.11.3500 ### Version: V4.2022.11.3501
### Updated: Tue Nov 29 22:01:14 UTC 2022 ### Updated: Wed Nov 30 10:03:29 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##