vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 11:40:22 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.08.3295 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-08-17 10:09:39 +00:00
parent c4dd391017
commit f4e01a4ffc
46 changed files with 86412 additions and 86412 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3294 #### Version: V4.2022.08.3295
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.08.3294 #### Version: V4.2022.08.3295
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.08.3294 #### Version: V4.2022.08.3295
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10666
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.08.3294 #### Version: V4.2022.08.3295
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10670
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10670
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10670
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10670
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10670
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10670
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10670
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3294 3295

10670
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10670
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.10.171.116 0; 20.97.8.216 0;

10670
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10670
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10670
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

10670
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.10.171.116 0; 20.97.8.216 0;

10670
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.10.171.116 0; 20.97.8.216 0;

10670
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 1; 10.1.0.146 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BBBike AhrefsBot
Dirbuster Bigfoot
FHscan Cogentbot
Kinza EasyDL
Not EroCrawler
SalesIntelligent GT::WWW
ScoutJet Kozmosbot
Trendictionbot Pandalytics
URLy.Warning Toweyabot
zgrab WebmasterWorldForumBot

310
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404enemy 404enemy
80legs 80legs
ADmantX Aboundexbot
AIBOT
Acunetix Acunetix
AiHitBot
Aipbot Aipbot
AllSubmitter AllSubmitter
Alligator Anarchie
Anarchy Anarchy
Ankit Apexoo
Anthill
Asterias
Atomseobot Atomseobot
AwarioRssBot BBBike
AwarioSmartBot
BLEXBot BLEXBot
BackStreet BackWeb
Bandit Bandit
Barkrowler Boardreader
Bigfoot Buck
Bitacle Buddy
Black Hole Bullseye
Blackboard BunnySlippers
Bolt
BuiltWith
CATExplorador CATExplorador
Cegbfeieh CCBot
CherryPicker
ChinaClaw ChinaClaw
Citoid Chlooe
Cloud mapping Cloud mapping
Collector Cogentbot
Copier Cosmos
CopyRightCheck
Craftbot Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler
Crescent
CrunchBot CrunchBot
Curious
DBLBot DBLBot
DIIbot DSearch
DTS Agent
DataCha0s DataCha0s
Demon DatabaseDriverMysqli
Discobot Devil
DigitalPebble
Dirbuster
Disco
Discoverybot Discoverybot
Dispatch Dispatch
DnBCrawler-Analytics DittoSpyder
DomCopBot DomainAppender
Domains Project
EMail Siphon
EMail Wolf
Ebingbong Ebingbong
Ecxi Ecxi
Express WebPictures Evil
Extractor FHscan
Ezooms FemtosearchBot
FDM FlashGet
Flunky Flunky
G-i-g-a-b-o-t Freeuploader
GT::WWW GT::WWW
Genieo
GermCrawler GermCrawler
GetRight
GetWeb
Getintent Getintent
Gigabot
Go!Zilla
Go-Ahead-Got-It
GoZilla GoZilla
Gotit Gotit
GrapeFX Grafula
GrapeshotCrawler
GridBot GridBot
HMView HEADMasterSEO
HTTrack HTMLparser
Haansoft Haansoft
HybridBot HonoluluBot
Humanlinks
IDBTE4M IDBTE4M
Iblog Iblog
Id-search Id-search
Image Sucker IlseBot
Intelliseek Indy Library
Internet Ninja InfoNaviRobot
Iria InternetSeer
JamesBOT JamesBOT
JennyBot Jbrofuzz
JetCar Jetty
JikeSpider
Jorgee Jorgee
Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kinza Keyword Density
LNSpiderguy Kozmosbot
Lanshanbot
Larbin Larbin
Leap
LeechFTP LeechFTP
LexiBot LibWeb
Libwhisker Libwhisker
LinkScan Lightspeedsystems
LinkpadBot Likse
LinkWalker
Linkbot
LinksManager LinksManager
LinqiaMetadataDownloaderBot LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker
Lmspider Lmspider
MIIxpc MIDown tool
MJ12bot MJ12bot
Majestic SEO MSIECrawler
Majestic-SEO Majestic-SEO
Majestic12
MarkMonitor MarkMonitor
MarkWatch Meanpathbot
Mata Hari Mediatoolkitbot
MauiBot MegaIndex.ru
Microsoft URL Control
Moblie Safari Moblie Safari
Mojolicious Mojeek
MolokaiBot
Morfeus Fucking Scanner
Mozlila Mozlila
Msrabot Mr.4x3
Musobot
Name Intelligence Name Intelligence
Navroad Navroad
NearSite
Nessus
NetAnts NetAnts
NetZIP Nettrack
Netvibes NextGenSearchBot
Niki-bot
Nikto Nikto
NimbleCrawler Nimbostratus
Ninja Nuclei
Nutch
Offline Explorer Offline Explorer
Offline Navigator Offline Navigator
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openvas
OrangeBot
OrangeSpider OrangeSpider
OutclicksBot
PECL::HTTP PECL::HTTP
PageThing.com PHPCrawl
Pandalytics Page Analyzer
PageScorer
Pavuk Pavuk
PeoplePal
Petalbot Petalbot
Pi-Monster
Picscout
Picsearch
PictureFinder
Piepmatz Piepmatz
Pixray ProPowerBot
PleaseCrawl ProWebWalker
Pockey
Probethenet Probethenet
Psbot Psbot
QueryN Metasearch PxBroker
Quick-Crawler RSSingBot
RankFlex
RankingBot2
Rankivabot Rankivabot
ReGet Re-re
RealDownload Recorder
RebelMouse RepoMonkey
RedesScrapy Rogerbot
RocketCrawler
SEOkicks SEOkicks
SEOstats SEOkicks-Robot
SalesIntelligent SEOprofiler
Screaming Scrapy
ScrepyBot SearchmetricsBot
Seomoz Seekport
Shodan SentiBot
SeobilityBot
Siphon Siphon
SiteLockSpider Site Sucker
Sitebeam SiteExplorer
Sitevigil Sitevigil
SlySearch
Snake Snake
Snapbot SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider SpaceBison
Spammen
SpankBot
Spanner Spanner
Spbot Spbot
Spinn3r SputnikBot
Sqlmap Steeler
Sqlworm SuperBot
Sqworm SuperHTTP
Stripper SurveyBot
Surfbot Suzuran
Swiftbot TeleportPro
Telesoft
Telesphoreo
TheNomad
Thumbor Thumbor
TightTwatBot
Titan Titan
Toata
Toweyabot
Turingos
Turnitin Turnitin
TwengaBot URLy Warning
URLy.Warning UnisterBot
Upflow Upflow
V-BOT
VCI VCI
Vagabondo Vacuum
VidibleScraper VelenPublicWebCrawler
Virusdie
VoidEYE VoidEYE
Voil Voltron
WBSearchBot WASALive-Bot
WEBDAV
WPScan
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW-Mechanize
Web Auto WWWOFFLE
Wallpapers
Wallpapers/3.0
Web Fuck Web Fuck
Web Pix WebBandit
WebAuto WebCollage
WebCopier
WebEnhancer
WebFetch WebFetch
WebPix WebFuck
WebSauger WebGo IS
WebStripper WebLeacher
WebWhacker
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
Website Quester Webshag
WebsiteExtractor
WebsiteQuester
Whack
Whatweb
Widow Widow
WinHTTrack WinHTTrack
WiseGuys Robot Woobot
Wprecon
Xaldon WebSpider Xaldon WebSpider
YoudaoBot YoudaoBot
Zitebot Zauba
Zeus
ZmEu ZmEu
ZoominfoBot ZyBorg
archive.org_bot archive.org_bot
arquivo-web-crawler cognitiveseo
coccocbot crawler4j
crawl.sogou.com demandbase-bot
crawler.feedback
eCatch
facebookscraper facebookscraper
gopher ips-agent
heritrix isitwp.com
internetVista monitor iubenda-radar
lwp-request linkdexbot
lwp-trivial lwp-trivial
magpie-crawler muhstik-scan
netEstate NE Crawler
page scorer page scorer
polaris version pcBrowser
probe-image-size
ripz ripz
satoristudio.net
scan.lol
seobility seobility
seocompany.store seocompany.store
seoscanners seoscanners
seostar sexsearcher
sogouspider siteripz
sp_auditbot
spyfu
sysscan sysscan
trendiction.com
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webprosbot
x09Mozilla x09Mozilla
zauba.io x22Mozilla

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404checker 404checker
80legs 404enemy
ALittle Client ALittle Client
Abonti Aboundexbot
AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Aipbot Alexibot
AllSubmitter AllSubmitter
Ankit Anarchy
Asterias Anarchy99
Atomseobot Anthill
BBBike Apexoo
Attach
AwarioRssBot
BDCbot BDCbot
BLEXBot BLEXBot
BackDoorBot BacklinkCrawler
BackStreet Barkrowler
BackWeb Battleztar Bazinga
BlackWidow BlackWidow
Blackboard Blackboard
BlowFish
Boardreader
Bolt Bolt
BotALot Brandprotect
Brandwatch
Buck Buck
BuiltBotTough CATExplorador
BunnySlippers CCBot
BuzzSumo Calculon
CODE87 Cegbfeieh
CSHttp Copier
CazoodleBot CopyRightCheck
CherryPicker Craftbot
Citoid DBLBot
Claritybot DSearch
Cliqzbot
Cogentbot
Copyscape
Cosmos
Crescent
DataCha0s DataCha0s
Discobot Dispatch
DittoSpyder DittoSpyder
DnyzBot DnyzBot
Download Wonder DomainCrawler
Dragonfly DomainSigmaCrawler
EMail Wolf DomainStatsBot
Drip
ECCP/1.0
EasyDL EasyDL
Ebingbong Ecxi
Exabot
Express WebPictures
ExtLinksBot ExtLinksBot
FDM Extreme Picture Finder
EyeNetIE
FHscan
FemtosearchBot FemtosearchBot
FlashGet Firefox/7.0
Foobot FyberSpider
FrontPage G-i-g-a-b-o-t
GT::WWW
GalaxyBot GalaxyBot
GetWeb Genieo
Go!Zilla Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
Gotit Gotit
GrabNet GrabNet
Grabber Grafula
GridBot HEADMasterSEO
HMView
HTMLparser HTMLparser
HTTP::Lite HTTrack
Harvest Harvest
Havij Havij
Heritrix
Hloader Hloader
Humanlinks
Iblog Iblog
Id-search Id-search
Image Sucker IndeedBot
Intelliseek Indy Library
InternetSeer InfoTekies
Iria Internet Ninja
JOC Web Spider JOC Web Spider
Jbrofuzz Jbrofuzz
JennyBot
JetCar JetCar
Jetty Jetty
JikeSpider JikeSpider
Joomla JustView
Jyxobot Jyxobot
Kinza Kenjin Spider
Keyword Density
LNSpiderguy
Larbin Larbin
LeechGet Libwhisker
LexiBot
LieBaoFast LieBaoFast
LinkpadBot Lightspeedsystems
LinksManager Likse
LinqiaMetadataDownloaderBot Linkbot
LinqiaRSSBot LinqiaScrapeBot
Lipperhey Spider
Lmspider
MIDown tool
MIIxpc MIIxpc
MJ12bot MQQBrowser
MSIECrawler Mag-Net
Magnet Magnet
Majestic-SEO Majestic SEO
MarkMonitor Mass Downloader
MauiBot
Mb2345Browser Mb2345Browser
MeanPath Bot
Meanpathbot
MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control
Mister PiX
Moblie Safari Moblie Safari
Mojeek
Mojolicious Mojolicious
Mozlila MolokaiBot
Mr.4x3 Mr.4x3
Msrabot
Musobot
NPbot NPbot
Navroad Name Intelligence
NearSite Nessus
NetAnts Net Vampire
NetMechanic NetLyzer
NetSpider Netcraft
Netvibes Nettrack
NextGenSearchBot
Nibbler
NimbleCrawler
Nimbostratus Nimbostratus
Nmap
Nutch
Offline Navigator Offline Navigator
OnCrawl OnCrawl
Openfind Openfind
Openvas
OrangeBot
OrangeSpider OrangeSpider
OutfoxBot OutclicksBot
PECL::HTTP
PHPCrawl
Page Analyzer Page Analyzer
PageGrabber PageAnalyzer
PageScorer
PageThing.com
Papa Foto
PeoplePal PeoplePal
Petalbot Petalbot
Pi-Monster
PictureFinder
Piepmatz
Pixray Pixray
PleaseCrawl PleaseCrawl
Pockey
ProWebWalker
Psbot Psbot
Pu_iN Pu_iN
Pump
PxBroker PxBroker
PyCurl PyCurl
QueryN Metasearch QueryN Metasearch
Quick-Crawler RSSingBot
RankActiveLinkBot RankActive
RankingBot
Rankivabot
Re-re Re-re
Reaper
RebelMouse RebelMouse
Recorder
RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
RocketCrawler RocketCrawler
SEOkicks SISTRIX
SEOkicks-Robot SalesIntelligent
SEOlyticsCrawler
SEOprofiler
ScanAlert ScanAlert
Screaming
ScrepyBot ScrepyBot
SeobilityBot Searchestate
Seekport
SemanticJuice
Siphon Siphon
Site Sucker
SiteLockSpider SiteLockSpider
SiteSnagger Siteimprove
Snake Snoopy
SocialRankIOBot Sociscraper
Sogou web spider
Sosospider Sosospider
SpaceBison
Spammen Spammen
Spbot SpankBot
Spinn3r
SputnikBot SputnikBot
Sqlmap Sqworm
Sucuri
SuperBot SuperBot
Suzuran SuperHTTP
Swiftbot Surfbot
T0PHackTeam T8Abot
Teleport TeleportPro
Telesoft Telesoft
Telesphoreo Telesphorep
The Intraformant The Intraformant
Thumbor TheNomad
TightTwatBot
Toweyabot
Tracemyfile
True_Robot True_Robot
Turingos Turingos
TurnitinBot
Twice Twice
Typhoeus UnisterBot
URLy Warning
URLy.Warning
Upflow
V-BOT V-BOT
VB Project
Vacuum Vacuum
Vagabondo Vagabondo
VeriCiteCrawler VelenPublicWebCrawler
Voil VidibleScraper
VoidEYE
WBSearchBot WBSearchBot
WWW-Collector-E WISENutbot
WWWOFFLE WPScan
WallpapersHD WWW-Mechanize
Web Collage Web Collage
Web Fetch Web Enhancer
Web Pix Web Pix
WebBandit WebBandit
WebGo IS WebCopier
WebPix WebFuck
WebReaper
WebSauger WebSauger
WebSucker WebSucker
WebWhacker WebWhacker
WebZIP
Webalta Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
WebsiteExtractor WebsiteQuester
Whatweb Whacker
Who.is Bot Who.is Bot
Widow WinHTTrack
WiseGuys Robot Wonderbot
Woobot Woobot
Wprecon Zade
YoudaoBot Zermelo
Zeus Zitebot
ZoominfoBot ZmEu
ZyBorg ZyBorg
archive.org_bot arquivo.pt
autoemailspider
backlink-check backlink-check
cah.io.community
clark-crawler clark-crawler
crawl.sogou.com
crawler4j
domainsproject.org
facebookscraper facebookscraper
instabid gopher
internetVista monitor isitwp.com
iubenda-radar iubenda-radar
magpie-crawler
meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot
pcBrowser
plumanalytics
probe-image-size probe-image-size
s1z.ru s1z.ru
scalaj-http scan.lol
seobility seoscanners
seocompany.store seostar
serpstatbot
siteripz siteripz
sogouspider trendiction.de
sp_auditbot ubermetrics-technologies.com
sysscan webgains-bot
webpros.com x09Mozilla
xpymep1.exe xpymep1.exe
zauba.io zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
abcdeg.xyz anonymous-redirect.com
as5000.com global-ics.co.za
dailystrength.org intellego.info
finejewelryshop.ru mojpreskumanie.com
hem.passagen.se mylida.org
home-task.com rating-bestcasino.com
myblogregistercm.tk tizanidineduringpregnancy.blogspot.com
nsatc.net vereo.eu
spb.ru videotuber.ru
xxart.ru westermarkanjou.se

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1008.su 3rbseyes.com
101raccoon.ru 40cg.com
123any.com 4rent.ru
1688.com 55wmz.ru
2drittel.de 5kstore.com
3xforum.ro 5ws.dating-app.ru
4replicawatch.net aac500b7a15b2646968f6bd8c6305869d7.com
4ureyesonly.com abwa.tk
5elementov.ru academia-nsk.org
5tey7463.weebly.com acortarurl.es
9icmzvn6.website add-add.men
DomainStatsBot adinterax.com
aaa38852e886ac4af1a3cff9b47cab6272.com adloads.net
aanapa.ru aerogo.com
abclauncher.com afftrack001.com
acads.net alessandraleone.com
actulite.com alf-img.com
adhome.biz alibestsale.com
advancedsoftwaresupport.com alldownload.pw
ahrntal.verymes.xyz aloofly.com
aktobe.xkaz.org amateurlivechat.org
akvamaster.dp.ua anaseracresar.tk
albuteroli.com angel1777.kz
alienwheels.de api.stathat.com
all4invest.info apiholdingmypage-a.akamaihd.net
amanda-porn.ga apilinkswiftco-a.akamaihd.net
amazon-seo-service.com apiwoodensealcom-a.akamaihd.net
amospalla.es appiq.mobi
animebox.com.ua arquivo.pt
anonymizeme.pro asmxsatadriverin.aircus.com
anticrawler.org avitocars.ru
apiallgeniusinfo-a.akamaihd.net avtoarenda.by
apidiamondatanet-a.akamaihd.net avtolombard-krasnodar.com
apifasterlightin-a.akamaihd.net awency.com
apple.com-cleaner.systems
applyneedy.xyz
appsecurityr.com
arendovalka.xyz
asrv-a.akamaihd.net
atdedinotuho.tk
auto4style.ru
autogrep.ru
aviav.co
avramstroy.ru
azartniy-bonus.com azartniy-bonus.com
azbaseimages.net azinoofficial777.ru
babieca.com backgroundpictures.net
baersaratov.ru backiee.com
barnfurnituremart.com bankiem.pl
basisches-wasser.net belsetirehafi.tk
bdsmgalls.net berrymall.ru
bestbuy.ca bestbrainprod.win
bestcoin.cc
bestplacetobuyeriacta.jw.lt bestplacetobuyeriacta.jw.lt
bestssaker.com bestpriceninja.com
bestwaystogetpaid.us biketank.ga
beta.hotkeys.com bioscorp.ru
betonka.pro blavia.00author.com
betterscooter.com blogig.org
bibys.com bloke.com
blackhatworth.com bluerobot.info
blackplanet.com
blog.koreadaily.com
blog.yam.com
blogs.rediff.com
bmsco.co bmsco.co
bolezniorganov.ru book-bets.com
bongacams.com brandov.ru
bongiornos.info breastaugmentation.co.za
boo-arts.com builtwith.com
bristolhotel.com burn-fat.ga
brk-rti.ru buy-meds24.com
bugof.gq cah.io.community
buntube.net
buqyxa.rincian.info
burkesales.com
butstrap.space
buttons-for-your-website.com
buy-cheap-online.info
buytizanidineonlinenoprescription.blogspot.com
calvet.altervista.org calvet.altervista.org
cam-kontakte.org cam-kontakte.org
canrioloadm.gq camel-beach.com
canoncdriverq3.pen.io
canrole.com
canvas.gloverid.site canvas.gloverid.site
carmuffler.net carina-sy.de
cartierloveringreplica.xyz carloans.com
cbcseward.com carson.getenjoyment.net
cdnnetwok.xyz cartierbraceletsreplica.pw
celejihad.info cartierlovestore.xyz
chatroulette.video casinos4dummies.co.uk
checkhit.com cenoval.ru
christianlouboutinsaleoutletonline.us cfjump.com
civilwartheater.com cheapestjerseysonwholesale.com
classiquebijoux.ru cheapwebsitehoster.com
cleanpcnow.com cheatcode-lita12.rhcloud.com
clickaider.com checktext.org
clickhype.com chinese-amezon.com
clickso.com chocoslim.pro
clickzzs.nl christianlouboutinsaleonline.us
cognitiveseo.com ciekawinki.pl
colehaanoutlet.store clubfashionista.com
compliance-brian.xyz coccoc.com
computernetworksonline.com coindirect.io
comsysnet.com collegerentals.com
constantaservice.net comissionka.net
coolgamechannel.com community.allhiphop.com
coolyarddecorations.com compliance-checker.info
copyrightclaims.org compliance-donald.xyz
costablue.xyz compliance-elena.xyz
counciltally.com compliance-irvin.xyz
compliance-ivan.xyz
construmac.com.mx
continent-e.tv
cool-wedding.net
coolwallpapers-hd.com
coolwallpapers4k.info
cornerstone-countertops.com
cougfan.info
countercrazy.com countercrazy.com
country-chic.ru cscwtalkto.site
covadhosting.biz
creditmoney.com.ua
culturevie.info
cutt.us cutt.us
cyclobenzaprinevstizanidine.blogspot.com cwetochki.ru
czat.wp.pl d-black.bz
dailystrength.org d2jsp.org
danceuniverse.ru data1.scopich.com
darkages.info davebestdeals.com
dataloading.net day-news.info
dayibiao.com ddospanel.com
delfin-aqua.com.ua decenttools.com
deplim.com decoratinghomes.ga
designdevise.com deda-moroza-zakaz.ru
diffbot.com deshevo-nedorogo.ru
digitalassetmanagement.site design-ideas.info
dinosaurus.site dfiles.me
dirtpics.pw dichvuvesinhhanoi.com
docstoc.com directivepub.com
domznaniy.ru discountbarbour.online
doxysexy.com dms-sw.ru
drillsaw.ru doctissimo.fr
driving.kiev.ua doeco.ru
drunkenstepfather.com dojki-hd.com
e-commerce-seo1.com domaincrawler.com
e2click.com domainsphoto.com
eager-nash.188-93-233-196.plesk.page dostavimvdom.ru
earl-brown.info dostavka-v-krym.com
easync.io dosugrostov.site
easytuningshop.ru downloaddy.net
educontest.net downloads-whatsapp.com
ege-russian.ru druzhininevgeniy63.blogspot.com
ekobata.ru dumpsmania24.com
eloconcream.blogspot.com dumuelave.xyz
energy-ua.com e90post.com
engines-usa.com easyshoppermac.com
escort-russian.com ec-file.info
eshop.md ecommerce-seo.com
essay-writing.work edge.sharethis.com
elektir.ru
elektronischezigarette2.mex.tl
elektronischezigarettekaufen1.beeplog.com
englishgamer.com
epngo.bz
erotikchat-24.com
esnm.ru
etm-consult.de
europages.com.ru europages.com.ru
evaashop.ru everytalk.tv
evening-dating-club.info extads.net
exhibitionplus.eu extener.org
extremepornos.net
extstat.com extstat.com
fakehandbags.xyz ezrvrentals.com
falcoware.com factorynightclub.com
familyholiday.ml fanrto.com
faptitans.com faptitans.com
fdzone.org farmingworm.com
fearcrow.com felizporno.com
filmetricsasia.com femmesdenudees.com
financeloan.us fickkontakte.org
finteks.ru filmgo.ru
flex4launch.ru filmidivx.com
florida-tourism.net financepoints.eu
fok.nl fiverr.com
fortwosmartcar.pw flooringinstallation-edmonton.com
forum.poker4life.ru floridahuntingfishingadventures.com
free-fb-traffic.com flowwwers.com
flytourisme.org
fortevidyoze.net
freshwallpapers.info freshwallpapers.info
futbolkisales.ru fusoradio.info
fx-brokers-review.com fxund.us
gamerscorps.com gael-s.ru
gateway.zscalerone.net galaxyflowers.ru
gazporno.com galaxys6manual.info
gdcentre.ru gambarkataku.co
getpopunder.com games.kolossale.ru
gfaq.ru gaytube.com
go2album.com gazoblok.net.ua
go2mike.ru gemgrab-a.akamaihd.net
gotcher.us genetworx.com
get-your-social-buttons.info
getaclueamerica.com
gg.zzyjxs.com
ghazel.ru
golaya.pw
goldadpremium.com
gpirate.com gpirate.com
greatdealshop.com greatidea.marketing
gribokstop.com greatzip.com
gsasearchengineranker.space gsasearchengineranker.xyz
gsasearchenginerankerdiscount.com gta-club.ru
hard-porn.mobi guidefs.ru
harmonyglen.us guitar-master.org
hauleddes.com h2monline.com
haveinc.xyz hahashka.ru
hoporno.com handicapbathtubarea.com
hotchatdate.com handicapvansarea.com
hotel-mkad.ru hatdc.org
hoterika.com hazardky.net
hoverboard360.de hdwalls.xyz
howopen.ru healgastro.com
healing-dysplasia.ru
highstairs-a.akamaihd.net
histock.info
homemade.gq
hotblog.top
hplaserjetpdriver8y.pen.io
hulfingtonpost.com
husky-shop.cz husky-shop.cz
idc.com.ua hyiphunter.org
ifirestarter.ru
iinstalll-fii1leis.jus0wil.pp.ua
imabase.com imabase.com
images-free.net imagine-ex.co
imagez.co infodocsportal.com
iminent.com invivo.hu
inbabes.sexushost.com iplogger.org
incep.imagine-ex.co iqbazar.ru
inclk.com ispaniya-costa-blanca.ru
indianmedicaltourismshop.com it-max.com.ua
intellekt21.ru janavibekken.no
internetproviderstucson.com janettabridal.com
intervsem.ru
itsdp3.com
ivanstroi.ru
izoll.ru
jagg.info
japan-bearings.ru
jav-fetish.com jav-fetish.com
jerseyswholesalechinalimited.com jav-idol.com
javrip.net
javxxx18.com
je7.us
jestr.org jestr.org
juliadiets.com jo24news.com
jurajskie.info
jus0wil.pp.ua jus0wil.pp.ua
jwss.cc kanimage.com
jyvopys.com katushka.net
kasino-money.pw keki.info
keywordbasket.com kevblog.top
kicknights.gq keywordsdoctor.com
kinocccp.net kichenaid.ru
klin3952.ru kino-key.info
knowyournextmove.com kinobest.pl
klejonka.info
konkursov.net konkursov.net
kovesszucs.atw.hu krynica.info
kozhniebolezni.com lacasamorett.com
kraljeva-sutjeska.com landoftracking.com
kriokomora.info lasvegaslockandsafe.com
kursy-ege.ru lawyers.cafe
kurwa.win
laapp.com
law-check-two.xyz
laxdrills.com
layola.biz.tc
le-clos-des-alouettes.com
lemon-ade.site
letmacwork.world
letmacworkfaster.site letmacworkfaster.site
lexiangwan.com letolove.ru
lida-ru.com lexixxx.com
lider-zhaluzi.kiev.ua lflash.ru
lifebyleese.com lignofix.ua
limads.men likesuccess.com
locationdesaison.com lion.playtap.us
lolnex.us lion.snapmap.us
lottospring.com liumimgx.pw
managerpak204.weebly.com liverpool.gsofootball.com
manifestyourmillion.com lizace.com
matsdale.com lkbennettoutlet.online
mcadamssupplyco.com localmatchbook.com
mecash.ru louboutinreplica.pw
medispainstitute.com.au lovi-moment.com.ua
medkritika.ru ltvperf.com
member-quiz.com lunchrock.co
metalonly.info m3gadownload.pl
michaelkorsoutletstore.net m4ever.net
middlerush-a.akamaihd.net mailemedicinals.com
mielec.pl manimpotence.com
mmostrike.ru mararoom.ru
mnimmigrantrights.net master-muznachas.ru
moetomnenie.com masterseek.com
mojpregled.com med-zdorovie.com.ua
mokrayakiska.com megahdporno.net
moncleroutletonline.pw meget.co.za
mensandals.xyz
metaxalonevstizanidine.blogspot.com
mirmedinfo.ru
missclub.info
mobstarr.com
mojaocena.com
monarchfind-a.akamaihd.net
morepoweronmac.trade morepoweronmac.trade
moroccosurfadventures.com
moscow.online-podarki.com moscow.online-podarki.com
motherboard.vice.com mosrif.ru
mountainstream.ms mototsikl.org
moyakuhnia.ru music.utrolive.ru
muizre.ru musicas.baixar-musicas-gratis.com
music7s.me musicdaddy.net
mustwineblog.com musicvidz.ru
muz-tracker.net my-bc.ru
mvpicton.co.uk mybackgroundlandscape.blogspot.com
my-cash-bot.co mygameplus.com
myhydros.org myliveblog.ru
mystats.xyz mysexpics.ru
narkologiya-orel.ru najaden.no
negociosdasha.com naphukete.ru
new7ob.com naughtyconnect.com
nomuos.it nbsproject.ru
nordvpn.com neks.info
notasprensa.info netvouz.com
npoet.ru newhairstylesformen2014.com
nzfilecloud.weebly.com nhl17coinsforps3.gratisblog.biz
nicefloor.co.uk
niki-mlt.ru
nikitsyringedrivelg.pen.io
nonameread45.live
nottyu.xyz
nuit-artisanale.com
nullrefer.com
o-o-11-o-o.com o-o-11-o-o.com
obesidadealgarve.com obiavo.by
obiavo.in obuv-kupit.ru
okeinfo.online office-windows.ru
openmultipleurl.com ohmyrings.com
onemactrckr.com
ooo-gotovie.ru
openlibrary.org
openmediasoft.com
optikremont.ru optikremont.ru
orion-v.com oranga.host.sk
panouri-solare-acoperis.com outpersonals.com
partner-high.men ozas.net
partner-stop.men p-business.ru
pc-services.ru panchro.xyz
perfectpracticeweb.com paperwritingservice17.blogspot.ru
pewit.pw park.above.com
php-market.ru pechikamini.ru
picscout.com pendelprognos.se
picturesify.com pensplan.com
pierrehardysale.online percin.biz.ly
pinkduck.ga perfectplanned.com
pogosh.com phormchina.com
poppen-nw.net pictures-and-images.net
porndairy.in pinwallpaper.xyz
porno-play.net piski.top
porno-raskazy.ru pixshark.com
pornogig.com plaff-go.ru
pornohd1080.online playfortuna-play.ru
prchecker.info poems.com.ua
prefersurvey.net pojdelo.weebly.com
porno-home365.com
pornonik.com
pornosemki.info
positive2b.ru
preconnubial.usuby.site
pregnant.guru pregnant.guru
pretty-mart.com presleycollectibles.com
printie.com
prize44.com prize44.com
promoheads.com proprostatit.com
pronorm.fr prostitutki-yaroslavlya.org
prostitutki-kharkova.org proxyelite.biz
prostitutki-novgoroda.org
prostitutki-tyumeni.org
prpops.com
pufip.com
pulseonclick.com
push-ad.com push-ad.com
pwwysydh.com quangcaons.com
qualitymarketzone.com questionmarque.ch
quickcashlimited.com quit-smoking.ga
qwertty.net
r-e-f-e-r-e-r.com
radiodigital.co radiodigital.co
ranksays.com randalljhoward.com
rapevideosmovies.com rankexperience.com
rapidgator-porn.ga ranking2017.ga
rating-casino2021.ru rating-bestcasino.com
razvratnoe.org relax.ru
real-time-analytics.com
rebrand.ly
recordpage-a.akamaihd.net
redirlock.com
refudiatethissarah.info
reklamuss.ru
remont-komputerov-notebook.ru
remontgruzovik.ru remontgruzovik.ru
renecaovillasale.online replicalouboutin.xyz
rentaremotecomputer.com rethinkwasteni.info
replica-watch.ru rfd-split.hr
replicaclub.ru riders.ro
responsinator.com ringporno.com
rootandroid.org rn-to-bsn.com
ruscopybook.com rome2rio.com
ruspdd.com royal-betting.net
rutor.vip russian-postindex.ru
sammlungfotos.online russkoe-zdorovie.ru
samo-soznanie.ru ryetaw.com
scanhub.ru s.lollypopgaming.com
scanner-mary.top samoiedo.it
scanner-nelson.top satoristudio.net
seksotur.ru sayyoethe.blogspot.co.za
sel-hoz.com scanner-ivan.top
selectads.men scanner-jess.top
seo-prof1.xyz se-welding.ru
seo-tools-optimizing.com se.bnt-team.com
seoexperimenty.ru search-goo.com
seojokes.net search.pch.com
secret.xn--oogle-wmc.com
seo-traffic-ranking.info
seorankinglinks.us seorankinglinks.us
serdcenebolit.com sevendays.com.ua
service.adtech.us sexytrend.ru
sevenstars7.com
sexcamamateurchat.com
sexkrasivo.net
sexsearch.com
sexystrippe.info
shakhtar-doneck.ru
share-buttons-for-free.com
shareyards.com
shariki-zuma-lines.ru shariki-zuma-lines.ru
shemalegalls.blogporn.in shopsellcardsdumps.com
shillyourcoins.com shtaketniki.ru
shtaketniki.kz sildenafilcitratemed.com
sigmund-freud.co.uk sindragosa.comxa.com
site-auditor.online sitiz.club
site-speed-checker.site sizeplus.work
siteaero.com skincrate.net
siteexpress.co.il smart-scripts.com
siteimprove.com snegozaderzhatel.ru
sk.golden-praga.ru snip.tw
slavic-magic.ru snsdeainavi.info
snow.nvr163.com social-fun.ru
sobecjvuwa.com.ru socialsignals24.com
social-buttons.com sogimlecal.tk
social-s-hhh.xyz spravka130.ru
sodexo.com sps-shop.com
softlinesolutions.me sr-rekneskap.no
spacash.com srecorder.com
spanking.to steamoff.net
spasswelt.net sticken.co
spravka-medosmotr.ru success-seo.com
sribno.net sudexpert66.ru
startwp.org sugarlyflex.pw
steame.ru
stroicol.net
strongholdsb.ru
styro.ru
subj.ukr-lit.com
sugvant.ru sugvant.ru
summerlinhomes411.info superinterstitial.com
superiends.org surgut.zrus.org
svetlotorg.ru syndicate.fun
swinon.site szucs.ru
tam-gde-more.ru t-bygg.com
tasteidea.com tako3.com
tedy.su taqplayer.info
thebestphotos.eu tastyfoodideas.com
theguardlan.com td-l-market.ru
themestotal.com technopellet.gr
tizanidine4mguses.blogspot.com teenbbw.yopoint.in
tizanidinedosageforsleep.blogspot.com teesdaleflyballclub.co.uk
tizanidineformigraines.blogspot.com telesvoboda.ru
theheroes.ru
thelottosecrets.com
tizanidinehcl4mgtab.blogspot.com tizanidinehcl4mgtab.blogspot.com
tizanidinestreetprice.blogspot.com
tkanorganizma.ru tkanorganizma.ru
tomatis.gospartner.com token-lab.org
touchmods.fr toloka.hurtom.com
tourcroatia.co.uk top-deal.com.pl
traderzplanet.in topcasinoratings.ru
trafficcentr.xyz topdownloads.ru
trafficgenius.xyz torrents.life
trafficreceiver.club tpu.ru
trk-4.net trackmedia101.com
trustl.life tradedeals.biz
ufa.xrus.org traffmonster.info
uhod-za-sobakoj.ru tvteleport.ru
uhodzalijami.ru twojebook.pl
ultimateclassicrock.com tx41tclega.ru
unacittaconte.org tzritel.tk
universals.com.ua uamtrk.com
urengoy.pro uchil.net
ussearche.cf udsgame.online
vellings.info umityangin.net
venta-prom.ru unrealcommander.com
videochat.mx userequip.com
vids18.site usswrite.com
vigrx-original.ru vacances-voyages.info
vinsit.ru ventelnos.com
vintontech.info video-camer.com
vipromoffers.com video-chat.love
vitoriacabos.com video-production.com
vizitki.net videogamesecrets.com
vkontarkte.com videonsk.com
voluumtracker1.com viktoria-center.ru
vremya.eu vip-parfumeria.ru
vse-pesni.com vipps.com.my
wallpapers-best.com visa-china.ru
watchmyfb.pl vita.com.hr
we-are-gamers.com vladhistory.com
webextract.profound.net vladimir.xrus.org
weblo.com vladimir.zrus.org
webmonetizer.net voloomoney.com
webnode.me vzlomtw.com
webpromotion.ae wallpaperaccess.com
websiteexplorer.info wanker.us
whosonmyserver.com waterefficiency.co
word-vorlagen.xyz weblibrary.win
www.souvenirua.com wholesalejerseychinaoutlet.com
wzgyyq.com wingsofrefuge.net
x-lime.net winner7777.net
x-musics.com woodyguthrie.se
x-stars.ru wordpresscore.com
xclicks.net xchat26.myfreecams.com
xedserver.com xjrul.com
xmronta.com xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf xn----9sbubg3ambdfl1j.xn--p1ai
xn--e1afanlbnfckd7c3d.xn--p1ai xn--80aafb2a.xn--p1ai
xn--oogle-wmc.com xn--80ajbshivpvn2i.xn--p1ai
xsion.net xn--80aodinpgi.xn--p1ai
xportvusbdriver8i.snack.ws
xtremeeagles.net xtremeeagles.net
yginekologa.com xxxdatinglocal.us
yoopsie.com yaaknaa.info
youbloodyripper.com yellocloud.be
youbrainboost.asia youbrainboost.asia
youdesigner.kz youdesigner.kz
yougetsignal.com youghbould.wordpress.com
youtubologia.it youthreaders.com
zaidia.xhost.ro youtoner.it
zakazvzloma.com z-master.ru
zakon-ob-obrazovanii.ru zakon-ob-obrazovanii.ru
zapiszto.pl
zarabotat-na-sajte.ru zarabotat-na-sajte.ru
zeroredirect12.com zarepta.com
znaturaloriginal.com zawyna.ua
zeg-distribution.com
zero1.it
zeroredirect2.com
znakomstvaonlain.ru
zytpirwai.net

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
10bet.com 00webcams.com
12u.info 18ps.ru
1xbet4you.com 1stat.ru
24-ak.ru 2000k.ru
4kplayer.pl 2rich4bitches.com
5elementov.ru 3weekdiet.com
6128786.com 5-steps-to-start-business.com
aaa38852e886ac4af1a3cff9b47cab6272.com 8gold.com
adexprts.com 8lufu.com
ads.gold aac52006ec82a24e08b665f4db2b5013f7.com
adserver-e7.com abovetherivernc.com
agysya.ru absugars.com
anaseracresar.tk addblueoff.com.ua
apiappenableinfo-a.akamaihd.net ads.tremorhub.com
artdestshop.eu adult3dgames.com
artpicso.com advancedcleaner.com
autolombard-krasnodar.ru aihelen.net
aviav.co aktobe.xkaz.org
avirasecureserver.com ald2014.org
alienwheels.de
all-number.com
alvtank.se
amazingninja.com
anabolics.shop
anjalika.co.in
ascat.porn
autodo.info
azbaseimages.net azbaseimages.net
bannerads.de bablonow.ru
beget.tech bannerspace.com
besthomemadepornsites.com beachfix.co
betshuckclean.com best-deals-products.com
beyan.host.sk bio-market.kz
big-boards.info bioscorp.ru
bingo8888.com blackcurranthumidifiers.site
bio.trade-jp.net blackle.com
bizlist.com.de blobar.org
bookmark4you.com bloke.com
bristolhostel.com bombla.org
bugof.gq bookmark4you.com.biz
business-made-fun.com boole.onlinewebshop.net
butstrap.space brainboosting.club
bvlgariwallet.xyz buqayy0.livejournal.com
c1.onedmp.com bvlgariring.xyz
callmd5map.com callawaygolfstore.online
cartierbracelet.xyz calstaterealty.us
cbcseward.com cheapjerseysap.com
children-learningreading.info cheapmoncler.xyz
chimiver.info chocolateslim-en-france.com
chlooe.com cialischsrx.com
clickintext.net clknsee.com
clickprotects.com coindirect.io
community.allhiphop.com com-cleaner.systems
compliance-margo.top com-secure.download
compliance-mark.top cometorussia.net
copro.pw compliance-nelson.top
cpabegins.ru copypaste.traderzplanet.in
crafthubs.com covid-schutzmasken.de
customergrowthsystems.com cracksplanet.com
dailystrength.org customchocolate.business-for-home.com
dating-time-now.com czat.wp.pl
dividendo.ru data-mining.tk
dmmspy.com dbmkfhqk.bloger.index.hr
dogsrun.net dignitasdata.se
dojki-hd.com dimkino.ru
dokfilms.net discountbarbour.online
downloadeer.net downloader12.ru
dtm-spain.com doyouknowtheword-flummox.ml
e-kwiaciarz.pl e90post.com
earn-from-articles.com educatemetv.com
eas-seo.com elmacho.xyz
easytuningshop.ru empis.magix.net
egvar.net expertblog.info
eksprognoz.ru ezaz.info
elvenar.com ezigarettenshop2.myblog.de
essayservicewriting.org f012.de
estelight.ru felizporno.com
facecup.top fickluder69.com
fast-torrent.ru fiuxy.com
fbdownloader.com freefoto.ca
femdom.twiclub.in funnypica.com
fifa-coins.online gameonasia.com
filmfanatic.com gateway.zscalerone.net
financepoints.eu gg-arena.ru
finstroy.net gidonline.one
fix-website-errors.com gilsonchiro.xyz
folowsite.com girlgamerdaily.com
foto-basa.com girlsatgames.ru
foxjuegos.com goodhousekeeping.com
free-fb-traffic.com gourcy.altervista.org
free-social-buttons.xyz greendream.com.ua
freeuploader.ml haveinc.xyz
frighteningremain.cf hermesbelts.xyz
frvo.alptandem.ru hermesreplica.win
fuckmill.com hidefiles.org
gazobeton-p.com.ua hkladys.com
getaclueamerica.com hoverboard360.se
glall.ru hvd-store.com
gogps.me igithab.com
goodly.pro impresagaia.it
guod.me infazavr.ru
hao123.com investmac.com
hawaiisurf.com istizanidineanarcoticdrug.blogspot.com
hayate.biz jikoman.info
hdwallpapers-free.com johannesburgsingles.co.za
histats.com journeydownthescale.info
hybrid.ru jrcigars.com
ifmo.ru junketjuice.blogspot.com
increasewwwtraffic.info justdating.online
inome.com.ua katjimej.blog.fc2.com
intellektmedia.at keyword-suggestions.com
internetartfair.com klin3952.ru
iptool.xyz klosetkitten.com
iqupdatetmz.win kosova.de
irkutsk.zrus.org kovrenok.ru
iron-age.info kriokomora.info
isotoner.com laapp.com
iusstf.org laulini.soclog.se
ixora.pro lerporn.info
javxxx18.com letsrepair.in
jo24news.com li-er.ru
joy-penguin.com lion.bolegapakistan.com
kadashihotel.com livejasmin.com
keywesthideaways.co lonely-mature.com
ladsblue.com manyresultshub-a.akamaihd.net
landscaping.center maximpartnerspr.com
laparfumotec.com mediawhirl.net
linkpulse.com medizinreisen.de
locustdesign.co medkritika.ru
louboutinbooties.xyz meduza-consult.ru
machicon-akihabara.info meltwater.com
meble-bogart.info mir-betting.ru
middlerush-a.akamaihd.net mnogolok.info
mil-stak.com mockupui.com
milblueprint.com morocco-nomad-excursions.com
mindbox.co.za mosrif.ru
monetizer.com-01.site most.gov.iq
mp3ritm.top mrvideospornogratis.xxx
multgo.ru my-big-family.com
mvpicton.co.uk mycouponizemac.com
myperiod.club mypets.by
myxdate.info myprintscreen.com
mzdish.site national-today-winning-winner.club
naperehresti.info navalwiki.info
narkologiya-voronezh.ru nfvsz.com
net-radar.com nikhilbahl.com
nextconseil.com nisuturnetdgu.tk
nhl09.ru nuit-artisanale.com
nibbler.silktide.com nuker.com
nomuos.it o-o-8-o-o.com
northfacestore.online obiavo.net
notfastfood.ru ofanda.com
o-o-6-o-o.ru
oakridgemo.com
obiavo.ru
oconto.ru
oda.as
okayimage.com okayimage.com
pawli.eu onclickpredictiv.com
pcads.ru openmultipleurl.com
perfectpracticeweb.com panicatack.com
picquery.com parajumpersstore.online
picture-group.com pixgood.com
pipki.r.acdnpro.com
piter.xrus.org
pivka.xyz
pizdeishn.com
pl.aasoldes.fr pl.aasoldes.fr
popup.matchmaker.com pogodnyyeavarii.gq
pogruztehnik.ru
polcin.de
porndairy.in porndairy.in
porno-chaman.info pornosee.info
pornosmola.info pornosemki.info
predmety.in.ua preg.marketingvici.com
priceg.com prostitutki-yaroslavlya.org
privacyassistant.net registratciya-v-moskve.ru
privatov-zapisi.ru reimageplus.com
prombudpostach.com.ua remontgruzovik.ru
prostitutki-tolyatti.org rennlist.com
psoriasis-file.trade responsive-test.net
pussysaga.com retreatia.com
quit-smoking.ga rff-cfal.info
razamicroelectronics.com robotixix.com
recinziireale.com roofers.org.uk
redirectme.net rospromtest.ru
renecaovillasale.online rukino.org
rethinkwasteni.info runtnc.net
rus-teh.narod.ru sabaapress.com
rutor.vip scanner-alexa.top
samo-soznanie.ru screentoolkit.com
satellite.maps.ilovevitaly.com sdrescher.net
seadragonherbery.com searchtooknow-a.akamaihd.net
secret.xn--oogle-wmc.com searchwebknow-a.akamaihd.net
secretscook.ru semprofile.com
semalt.com sexpornotales.com
seo-prof1.xyz smartbalanceworld.com
seoboxes.com sotkal.lark.ru
sex-foto.pw speechfoodie.com
sexfreepornoxxx.com speeddream.xyz
sibtest.ru stairliftstrue.com
site5.com stariy-baku.com
slavic-magic.ru steame.ru
snts.shell-pmr.ru studiofaca.com
spabali.org supermama.top
spasswelt.net superstarfloraluk.com
stardevine.com svtrd.com
stroiminsk.org taqplayer.info
strongsignal-a.akamaihd.net thedownloadfreeonlinegames.blogspot.com
svarbit.com titan-ads.life
takethatad.com titan-cloud.life
tako3.com tizanidine4mgstreetvalue.blogspot.com
tasteidea.com tizanidineinteractions.blogspot.com
tattoo33.ru tizanidinevscyclobenzaprine.blogspot.com
tattooha.com tracklead.net
taxi-v-eisk.ru trafficreceiver.club
tdsing.ru trafficstars.com
theendivechronicles.com trion.od.ua
themeforest.net trudogolik.net
thfox.com twelvevisionspartyofcolorado.com
timeallnews.ru twu.com.ua
tizanidineorflexeril.blogspot.com uhodzalijami.ru
tizanidinepills.blogspot.com unacittaconte.org
tizanidinerecreationaluse.blogspot.com uni.me
tizanidinewithdrawal.blogspot.com unitexindia.com
tizanidinezanaflex.blogspot.com
tmtrck.com
toondinsey.com
trucri.me
turbo-suslik.org
txxx.com
u-cheats.ru
unrealcommander.biz unrealcommander.biz
url2image.com upproar.com
urlopener.blogspot.com.au valmetrundan.se
vabasa.inwtrade.com vapeface.club
venerologiya.com velpanex.ru
vepad.com versaut.xxx-cam.webcam
videochat.ph vip.51.la
vipromoffers.com vipps.com.my
w-journal.ru virus-schutzmasken.de
w7s.ru visa-china.ru
walkme.com vodkoved.ru
web-betting.ru vostoktrade.info
webpromotion.ae vpnmouse.com
wedding0venues.tk vriel.batcave.net
whatsappbot.flyland.ru weightatraining.com
works.if.ua wholesalenfljerseys.us.com
worldoffiles.ru witclub.info
withstandingheartwarming.com
wowcasinoonline.ooo wowcasinoonline.ooo
ws.ampower.me www.arenda-yeisk.ru
xchangetrak.com wygraj-skiny.win
xn----itbkqkfiq.xn--p1ai xcc24.pl
xn--e1aggki3c.xn--80adxhks xerox-douglas.cf
yellowfootprints.com xfluro.com
zarajbuilders.com xn----7sbaphztdjeboffeiof6c.xn--p1ai
zed21.net xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai
xvideosporn.biz
yopoint.in
youporn-ru.com
yugk.net
zolotoy-lis.ru zolotoy-lis.ru
zoodrawings.com zootoplist.com
zx6.ru zygophyceous.womanstars.site
zytpirwai.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.173 0; 10.1.0.146 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3294 ### Version: V4.2022.08.3295
### Updated: Tue Aug 16 22:01:21 UTC 2022 ### Updated: Wed Aug 17 10:03:41 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.08.3294 ### Version: V4.2022.08.3295
### Updated: Tue Aug 16 22:01:21 UTC 2022 ### Updated: Wed Aug 17 10:03:41 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##