vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 15:34:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.07.3233 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-07-17 10:09:23 +00:00
parent 22beb0b235
commit e9a1c68407
46 changed files with 99286 additions and 99286 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.07.3232 #### Version: V4.2022.07.3233
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.07.3232 #### Version: V4.2022.07.3233
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.07.3232 #### Version: V4.2022.07.3233
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12276
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.07.3232 #### Version: V4.2022.07.3233
#### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7094](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

12280
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12280
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

12280
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

12280
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 1; 10.1.1.94 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 0; 10.1.1.94 0;
127.0.0.1 0; 127.0.0.1 0;

12280
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 1; 10.1.1.94 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 0; 10.1.1.94 0;
127.0.0.1 0; 127.0.0.1 0;

12280
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 1; 10.1.1.94 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 0; 10.1.1.94 0;
127.0.0.1 0; 127.0.0.1 0;

12280
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3232 3233

12280
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

12280
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.25.185.49 0; 52.188.23.161 0;

12280
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 1; 10.1.1.94 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 0; 10.1.1.94 0;
127.0.0.1 0; 127.0.0.1 0;

12280
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 1; 10.1.1.94 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 0; 10.1.1.94 0;
127.0.0.1 0; 127.0.0.1 0;

12280
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 1; 10.1.1.94 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 0; 10.1.1.94 0;
127.0.0.1 0; 127.0.0.1 0;

12280
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.25.185.49 0; 52.188.23.161 0;

12280
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.25.185.49 0; 52.188.23.161 0;

12280
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 1; 10.1.1.94 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
FDM Aboundexbot
GrapeshotCrawler Anarchie
JetCar Blow
Minefield ExtLinksBot
NetAnts Page Analyzer
SiteSnagger RankingBot2
SlySearch Semrush
SpankBot Siphon
ZmEu WebLeacher
seobility archive.org_bot

292
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider
404checker 404checker
404enemy 80legs
ADmantX ADmantX
AIBOT AIBOT
Abonti Abonti
Aboundex Aboundex
AiHitBot Aboundexbot
AllSubmitter AhrefsBot
Alexibot
Alligator Alligator
AlphaBot AlphaBot
Anthill Anarchie
Apexoo
Aspiegel Aspiegel
Asterias
Atomseobot Atomseobot
Attach BBBike
AwarioSmartBot
BDFetch
BackStreet
BackWeb BackWeb
Badass BacklinkCrawler
Barkrowler Battleztar Bazinga
Blackboard Bigfoot
Boardreader
Bolt Bolt
BuiltBotTough Brandprotect
Buck
Buddy
Bullseye Bullseye
BunnySlippers BunnySlippers
BuzzSumo CATExplorador
CODE87 CODE87
CazoodleBot Calculon
CensysInspect Cegbfeieh
CheTeam
CheeseBot
CherryPicker CherryPicker
ChinaClaw
Citoid
Claritybot
Cloud mapping Cloud mapping
Cocolyzebot Cocolyzebot
Collector Collector
CopyRightCheck Copier
Copyscape Copyscape
Cosmos
Crawling at Home Project
CrunchBot CrunchBot
Curious
Custo Custo
DBLBot DBLBot
DataCha0s DataCha0s
Devil DatabaseDriverMysqli
Dirbuster Demon
Disco Disco
Discoverybot
DittoSpyder DittoSpyder
DnBCrawler-Analytics
DnyzBot DnyzBot
DomainAppender DomainSigmaCrawler
Domains Project
Download Wonder Download Wonder
Drip
ECCP/1.0 ECCP/1.0
EMail Siphon EMail Siphon
EMail Wolf
EasyDL EasyDL
Ebingbong Ebingbong
Ecxi EirGrabber
EroCrawler Exabot
Evil Extreme Picture Finder
ExtLinksBot Ezooms
ExtractorPro FemtosearchBot
FHscan
Fimap
Firefox/7.0 Firefox/7.0
FlashGet G-i-g-a-b-o-t
Flunky
Foobot
Freeuploader
Fuzz
FyberSpider
GT::WWW GT::WWW
GalaxyBot GalaxyBot
GermCrawler GermCrawler
Gigabot Getintent
Go!Zilla GoZilla
Go-Ahead-Got-It Grafula
GrapeFX GrapeshotCrawler
GridBot
HEADMasterSEO HEADMasterSEO
HTTrack HTTrack
Haansoft HaosouSpider
Harvest Harvest
Heritrix Heritrix
HonoluluBot Hloader
Humanlinks
IDBTE4M IDBTE4M
Iblog IRLbot
IlseBot IlseBot
Image Sucker IndeedBot
Indy Library
InfoNaviRobot
InternetSeer InternetSeer
Iria Iskanie
JOC Web Spider
JennyBot
Jetty
Jorgee
JustView
Kenjin Spider
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kozmosbot Kinza
LWP::Simple Larbin
Lanshanbot
LeechFTP
LeechGet LeechGet
Lftp Lftp
LibWeb LinkScan
Libwhisker
LinkWalker LinkWalker
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager
Lipperhey Lipperhey
MFC_Tear_Sample Ltx71
MIDown tool
MJ12bot MJ12bot
Mag-Net MSFrontPage
Magnet Majestic-SEO
MarkMonitor MarkWatch
Masscan Masscan
Mata Hari Mata Hari
Mb2345Browser Minefield
MegaIndex.ru
Metauri
Moblie Safari
MolokaiBot MolokaiBot
Morfeus Fucking Scanner
Mozlila Mozlila
Msrabot Msrabot
Musobot Musobot
NearSite NICErsPRO
Needle NetLyzer
NetAnts NetSpider
NetMechanic NetZIP
Netvibes
NextGenSearchBot
Nikto Nikto
Nmap NimbleCrawler
Nimbostratus
Not Not
Octopus Nuclei
OpenVAS Nutch
Openfind Offline Explorer
Openvas Offline Navigator
OpenLinkProfiler
OrangeBot OrangeBot
OutfoxBot OrangeSpider
PHPCrawl PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer PageScorer
PageThing.com
Panscient Panscient
Pavuk Pavuk
Petalbot Pi-Monster
PictureFinder
Pimonster Pimonster
PleaseCrawl PleaseCrawl
ProPowerBot Pump
QueryN Metasearch PxBroker
RankingBot PyCurl
Quick-Crawler
RSSingBot
RankActiveLinkBot
RankFlex
Rankivabot
RealDownload RealDownload
Reaper RebelMouse
Recorder
RedesScrapy RedesScrapy
Ripper Ripper
RocketCrawler
SBIder SBIder
SEOprofiler SEOkicks
SEOstats SEOkicks-Robot
Scanbot SEOlyticsCrawler
ScoutJet SMTBot
SearchmetricsBot ScanAlert
SemanticJuice Screaming
Semrush ScreenerBot
Seekport
SeobilityBot SeobilityBot
Shodan Shodan
Siphon Site Sucker
SiteLockSpider SiteCheckerBotCrawler
SiteSnagger Sitebeam
SiteSucker
Siteimprove
SmartDownload
Snapbot
Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper
Sosospider
Spammen Spammen
SpankBot
Spanner
Spinn3r Spinn3r
SputnikBot
Sqlworm Sqlworm
Stripper Sqworm
Sucker Steeler
Sucuri
SuperBot SuperBot
Surfbot
SurveyBot
T0PHackTeam T0PHackTeam
T8Abot TeleportPro
Telesoft
Telesphoreo Telesphoreo
Thumbor Thumbor
Titan Titan
Trendictionbot
Turingos Turingos
Turnitin Twice
UnisterBot
Upflow Upflow
V-BOT VCI
VelenPublicWebCrawler Vagabondo
VeriCiteCrawler Virusdie
WASALive-Bot WASALive-Bot
WBSearchBot WBSearchBot
WEBDAV
WISENutbot WISENutbot
WPScan
WWW-Mechanize WWW-Mechanize
Wallpapers/3.0 WallpapersHD
WeSEE
Web Auto Web Auto
Web Enhancer
Web Fetch Web Fetch
Web Fuck WebBandit
WebCollage
WebFuck WebFuck
WebImageCollector
WebLeacher WebLeacher
WebSauger WebPix
WebReaper
WebStripper WebStripper
WebSucker WebSucker
Webalta WebWhacker
Webshag Website Quester
WebsiteQuester
Webster Webster
Whacker Whack
Whatweb
Widow Widow
WinHTTrack WinHTTrack
Wotbox Wprecon
archive.org_bot Xaldon_WebSpider
Zade
Zauba
Zitebot
ZumBot
adscanner
arquivo.pt arquivo.pt
autoemailspider check1.exe
backlink-check clark-crawler
cah.io.community coccocbot
com.plumanalytics com.plumanalytics
crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j dataforseo.com
eCatch demandbase-bot
domainsproject.org
evc-batch
instabid instabid
internetVista monitor
ips-agent
iubenda-radar
lwp-request lwp-request
meanpathbot lwp-trivial
mediawords mediawords
oBot netEstate NE Crawler
page scorer plumanalytics
polaris version polaris version
probe-image-size
ripz
scan.lol scan.lol
seobility seostar
seoscanners siteripz
serpstatbot sogouspider
sp_auditbot
spyfu
trendiction.de trendiction.de
voyagerx.com webmeup-crawler
webpros.com
webprosbot
x22Mozilla
zauba.io
zgrab zgrab

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 80legs
404checker
AIBOT AIBOT
ALittle Client
ASPSeek ASPSeek
Abonti Abonti
Acunetix Aboundexbot
AiHitBot AfD-Verbotsverfahren
Aipbot AllSubmitter
Alexibot Alligator
AlphaBot
Anarchie
Anarchy Anarchy
Ankit Anarchy99
Anthill
Apexoo Apexoo
Aspiegel
Atomseobot Atomseobot
Attach Attach
AwarioRssBot AwarioRssBot
BDCbot BBBike
BDFetch
BLEXBot
BackDoorBot
Backlink-Ceck
BacklinkCrawler BacklinkCrawler
BatchFTP Bandit
Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot BetaBot
Bigfoot Blow
BlowFish BlowFish
Boardreader Bolt
BotALot Brandprotect
Brandwatch Buddy
BuiltBotTough Bullseye
BuzzSumo CODE87
CATExplorador CensysInspect
CCBot CherryPicker
Calculon Collector
CheeseBot Copier
Cliqzbot
Cloud mapping
CopyRightCheck CopyRightCheck
Craftbot Craftbot
CrazyWebCrawler Crescent
Curious
CyotekWebCopy CyotekWebCopy
DIIbot DBLBot
DTS Agent Demon
DataCha0s
Deusu
Digincore Digincore
Dirbuster Discoverybot
Disco DnBCrawler-Analytics
Dispatch
DnyzBot DnyzBot
DomainAppender DomainAppender
DomainCrawler DomainCrawler
Domains Project Domains Project
Dotbot
Download Wonder Download Wonder
Drip EMail Siphon
EasyDL
Ebingbong Ebingbong
Ecxi EirGrabber
EroCrawler
Exabot
ExtLinksBot ExtLinksBot
Ezooms Extractor
Extreme Picture Finder
FDM FDM
FlashGet FemtosearchBot
Foobot FrontPage
FyberSpider FyberSpider
Fyrebot G-i-g-a-b-o-t
GT::WWW GalaxyBot
Genieo
Gigabot Gigabot
Go!Zilla Go!Zilla
Go-Ahead-Got-It
GrabNet
Grafula
GrapeFX
HMView HMView
HTTP::Lite
HTTrack HTTrack
HaosouSpider Haansoft
Havij Harvest
Heritrix
HonoluluBot
Humanlinks
IDBTE4M
IDBot IDBot
IRLbot Iblog
Id-search Id-search
IlseBot
IndeedBot
Indy Library Indy Library
InfoNaviRobot InfoTekies
Intelliseek
Internet Ninja Internet Ninja
Iria Iria
Iskanie Iskanie
IstellaBot JamesBOT
JikeSpider Jbrofuzz
JennyBot
Jorgee Jorgee
Kinza LNSpiderguy
Kozmosbot
LWP::Simple
Larbin
LeechFTP LeechFTP
Lftp LeechGet
Lightspeedsystems LexiBot
Likse LibWeb
LinkWalker LieBaoFast
LinkextractorPro Litemage_walker
LinqiaRSSBot Lmspider
Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIIxpc
MJ12bot
MQQBrowser MQQBrowser
MSFrontPage Majestic SEO
MSIECrawler
Mag-Net
Majestic12 Majestic12
Mass Downloader
Masscan Masscan
Mata Hari Mata Hari
Mb2345Browser MeanPath Bot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru Metauri
MicroMessenger Minefield
Microsoft Data Access
Mister PiX Mister PiX
Mojeek Mojeek
MolokaiBot NICErsPRO
Mozlila
Msrabot
NPbot
Name Intelligence Name Intelligence
Nameprotect Nessus
NearSite Net Vampire
Needle NetMechanic
NetAnts
NetLyzer
NetSpider NetSpider
NetZIP Netcraft
Nikto Netvibes
Ninja NimbleCrawler
Nimbostratus
Nmap Nmap
Octopus Nuclei
Nutch
Offline Explorer Offline Explorer
Offline Navigator Offline Navigator
OpenLinkProfiler OpenLinkProfiler
OpenVAS OpenVAS
Openfind OrangeBot
OutclicksBot OrangeSpider
PHPCrawl PECL::HTTP
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageScorer PeoplePal
Pandalytics Petalbot
Pi-Monster Pi-Monster
PictureFinder Picsearch
Piepmatz Piepmatz
ProPowerBot Psbot
ProWebWalker
Pu_iN Pu_iN
PyCurl PyCurl
RSSingBot RankActiveLinkBot
RankFlex
Rankivabot Rankivabot
RankurBot
Re-re Re-re
ReGet
RebelMouse RebelMouse
Recorder
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
RocketCrawler RocketCrawler
SBIder
SEOkicks SEOkicks
SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SISTRIX
SMTBot SMTBot
Scanbot Scanbot
ScreenerBot ScreenerBot
ScrepyBot
Searchestate
SearchmetricsBot SearchmetricsBot
Seekport Seekport
SemrushBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot Shodan
Site Sucker Siphon
SiteCheckerBotCrawler SiteExplorer
SiteSnagger
SiteSucker
Sitebeam
Sitevigil Sitevigil
SlySearch
Snake Snake
Snoopy Snoopy
SocialRankIOBot SocialRankIOBot
SputnikBot SpankBot
Spanner
Spbot
Sqlmap
Sqlworm Sqlworm
Sqworm Sqworm
Steeler SurveyBot
TeleportPro Suzuran
TightTwatBot Swiftbot
Toata Szukacz
T0PHackTeam
T8Abot
TheNomad
Toweyabot
Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot Turingos
Turnitin Turnitin
Twice TwengaBot
VB Project Typhoeus
Vagabondo Upflow
VoidEYE VCI
WISENutbot VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
Voltron
WASALive-Bot
WBSearchBot
WPScan WPScan
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WWW::Mechanize
Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE WallpapersHD
Web Sucker Web Collage
Web Enhancer
Web Fuck
WebAuto
WebBandit WebBandit
WebCollage
WebCopier WebCopier
WebEnhancer WebEnhancer
WebFetch WebFuck
WebGo IS
WebLeacher WebLeacher
WebReaper WebSucker
WebSauger WebWhacker
WebStripper
WebZIP
Webalta Webalta
WebmasterWorldForumBot Wprecon
WebsiteQuester Xaldon_WebSpider
Webster Xenu
Widow Zade
WinHTTrack
WiseGuys Robot
Wotbox
Zeus Zeus
Zitebot Zitebot
ZoominfoBot ZoomBot
archive.org_bot adscanner
arquivo-web-crawler arquivo-web-crawler
autoemailspider autoemailspider
backlink-check
check1.exe
clark-crawler clark-crawler
coccocbot
cognitiveseo cognitiveseo
crawler.feedback demandbase-bot
domainsproject.org domainsproject.org
eCatch
facebookscraper facebookscraper
instabid gopher
ips-agent internetVista monitor
isitwp.com meanpathbot
lwp-request
mediawords
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
probe-image-size oBot
ripz plumanalytics
s1z.ru polaris version
scalaj-http seostar
seoscanners tAkeOut
sitechecker.pro webgains-bot
siteripz x22Mozilla
sp_auditbot zauba.io
spyfu
trendiction.com
webpros.com

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
123any.com abbanreddy.com
123locker.com adssafeprotected.com
crackguru.tk alienwheels.de
dstroy.su gilsonchiro.xyz
gpms.org.my manve.info
ilmexico.com optibuymac.com
indo-export.ru qor360.com
parajumpersstore.online seo-prof1.xyz
playfortuna-play.ru stardevine.com
tomck.com stop-zavisimost.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
007angels.com 00it.com
0lovespells0.blogspot.com 1-free-share-buttons.com
51.la 1j7740kd.website
7yue.org 2itech.ru
8lufu.com 2rich4bitches.com
300richmond.co.nz
3weekdiet.com
4k-player.pl
4rent.ru
4ureyesonly.com
7ooo.ru
888.com
8gold.com
a96527gi.beget.tech a96527gi.beget.tech
aa625d84f1587749c1ab011d6f269f7d64.com aa08daf7e13b6345e09e92f771507fa5f4.com
aaronabel.com aa9d046aab36af4ff182f097f840430d51.com
absolute-s.ru absugars.com
actionnooz.com abwa.tk
adf.ly adnotbad.com
adhome.biz adservme.com
adidas.frwebs.fr alfa-img.com
advancedsoftwaresupport.com alfa9.com
advertex.info allbizne.co.ua
adzerg.com allornamenti.com
afslankpillen2017nl.eu alphahoverboards.com
agapovdg.ru alpinism.ru
agroeconom.kz anatomiy.com
aibolita.com android-style.com
aihelen.net android4fun.org
aktobe.xkaz.org animalrank.com
algarveglobal.com apartmentbay.ru
allmarketsnewdayli.gdn apiallgeniusinfo-a.akamaihd.net
allnews.md asrv-a.akamoihd.net
am15.net availit.weebly.com
amateurlivechat.org avon-novosib.ru
amazon-seo-service.com avtoarenda.by
amigobulls.com awaybird.ru
appmsr.org bag77.ru
aproposde.com balkanfarma.org
ardimobileinfo.ml banan.tv
arewater.com bankofthewext.com
arpelsreplica.xyz bestmouthwash.club
autoblogger24.info bestofferswalkmydogouteveryday.gq
autolombard-krasnodar.ru bestofupload.info
autonew.biz
aviva-limoux.com
avkzarabotok.com
avtocredit-legko.ru
avtovolop.ru
azartclub.org
b00kmarks.com
b2b-lounge.com
backlinkwatch.com
bala.getenjoyment.net
bannerbank.ru
besthoro.ru
betonka.pro betonka.pro
beyan.host.sk bigpictures.club
bibys.com biteg.xyz
bigcareer.info blogig.org
binaryoptionscops.info bluesman.nu
biocn.dx.am bmwhighperformers.com
bird1.ru bodybuilding-shop.biz
blog-hits.com bolegapakistan.com
bmw.afora.ru bolezniorganov.ru
bookhome.info bongiornos.info
bookmark4you.com book-bets.com
braintobe.win brateg.xyz
brandov.ru breastaugmentation.co.za
brus.city brendbutik.ru
buron.pw brokergid.ru
butstrap.space browserprotecter.com
bum.com.ru
bvlgariwallet.xyz
californianews.cf
calstaterealty.us
canacopegdl.com canacopegdl.com
candlewooddentalcentre.com.au cars-modification.net
capture-room.com cartierjusteunclou.xyz
cardsdumps.com cartierlove2u.com
cartierbraceletsreplica.pw cartierwatch.xyz
ccbill.com casinos4dummies.co.uk
cdnanalytics.xyz cayado.snn.gr
celebsopics.com cbcseward.com
cezartabac.ro
cfjump.com
chatroulette.si
cheapjerseysap.com cheapjerseysap.com
checktext.org choice-credit.ru
chee-by.biz cialis-samples.com
chinese-amezon.com cicaki.net
cialisndbrx.com clickso.com
cindymatches.com clicksotrk.com
cinemaenergy-hd.ru co.lumb.co
ciproandtizanidine.blogspot.com codq.info
coaching-netz.info cognitiveseo.com
comeondog.info collegeessay19.blogspot.ru
comissionka.net
companies-catalog.com
compliance-andrew.top compliance-andrew.top
compliance-barak.xyz compliance-brian.xyz
compliance-fred.top compliance-jessica.top
compliance-fred.xyz compliance-walter.top
compliance-george.xyz compliance-willy.top
compliance-olga.top consorzioilmosaico.org
cookie-law-enforcement-ff.xyz constantaservice.net
coolwallpapers4k.info contextualyield.com
culmimg.pw cordstrap.cc
culturevie.info cornerstone-countertops.com
cutalltheshit.com creditnation.ru
cymbaltaandtizanidine.blogspot.com crirussian.ru
datract.com custom-electric-guitar.com
ddlmega.net custom-product-labels.com
deadlinkchecker.com cv.wallhade.co
delitime.info dailyrank.net
deutschehobbyhuren.net damasarenai.info
distver.ru darkages.info
divci-hry.info data1.scopich.com
ddsoldes.fr
delayreferat.ru
delfin-aqua.com.ua
detsky-nabytek.info
deutschland123.de
discountbarbour.online
div.as
doctormakarova.ru
domaineaneblanc.com domaineaneblanc.com
downtuptv.gq donvito.unas.cz
doyouknowtheword-flummox.ml drunkmoms.net
dyshagi.ru dvr.biz.ua
e-biznes.info dynainbox.com
e-poker-2005.com e-kwiaciarz.pl
editmedios.com e705.net
educontest.net educatemetv.com
ee77ee.com elektrischezigarette1.onsugar.com
efnor-ac.com elektrozigarette2.wordpressy.pl
ekaterinburg.xrus.org elektrozigaretten2.yn.lt
electro-prom.com emailaccountlogin.co
electronicadirect.com eredijovon.com
elektir.ru errorfixing.space
elektroniksigaraankara.info etotupo.ru
elektronischezi.livejournal.com euroskat.ru
energy-ua.com everflora.ru
enhand.se everytalk.tv
eropho.com exchanges-bet.com
erotik0049.com exdocsfiles.com
escortslet.net extener.org
esfchat.tk f0815.de
evehemming.blogspot.com.au fable.in.ua
ezigarettenshop2.postbit.com
f07.de
factorynightclub.com factorynightclub.com
falopicm.pw falllow.gq
farmingworm.com fashion.stellaconstance.co
fba-mexico.com fastcrawl.com
filmgo.ru fealq.com
financehint.eu fickkontakte.org
fireads.men filerockstar298.weebly.com
flauntyoursite.com filmfanatic.com
financeloan.us
flash4fun.com
floridamhca.org
flowersbazar.com
flowwwers.com
flprog.com flprog.com
food.dtu.dk foxinsocks.ru
forex21.ru franklinfire.co
forpackningsutveckling.se free-share-buttons.blogspot.com
forum20.smailik.org frivgame250.com
foto-basa.com fun2cell.net
foto-telok.net funcrushgames.com
fotosfotos.eu gambarxkata.co
fotzen-ficken.com
foxtechfpv.com
frankofficial.ru
frcls.fr
free-gluten.ru
free-laptop-reward.com
free-social-buttons.xyz
freetangodownload.com
fullgirl.ru
gabeshop.ru
gambarkataku.co
gamedayhouse.com
gamingspark.com gamingspark.com
gateway.zscalerone.net gay-site.store
gayxperience.com gaz-voshod.ru
gentamicineyedrops.blogspot.com gazporno.com
gilsonchiro.xyz genericlowlatencyasiodriverhq.aircus.com
girlspicsa.com gepezz.info
gk-atlant.info getresponse.com
gktt.ru gifspics.com
globalsurfari.com gigixo.com
gosmeb.ru glavtral.ru
gratuitxblcodes.com gtopstats.com
greatdealshop.com guigyverpo.cf
greenzaim.ru hahashka.ru
gsbs.com.ua hasshe.com
gwebtools.com.br hdhc.ru
health-medical-portal.info hdpixion.com
heliko.no herokuapp.com
hem.passagen.se histats.com
hexpilot.com hobbyhuren-datenbank.com
hjaoopoa.top home-task.com
hmywwogw.bloger.index.hr hosting-tracker.com
hotkeys.com
houdom.net houdom.net
house.sieraddns.com housemilan.ru
humanorightswatch.org hoverboard360.de
ico.re howopen.ru
idegenvezeto.eu hoztorg-opt.ru
ifmo.ru images-free.net
ihtec2019.org imagez.co
ikritikimou.gr imgata.com
imagecoolpub.com inbabes.sexushost.com
imagui.eu inboundlinks.win
imgarcade.com infokonkurs.ru
imperia31.ru internetproviderstucson.com
impisr.edunsk.ru iqoption-bin.com
int.search.tb.ask.com istanbulit.com
intellektmedia.at istripper.com
interferencer.ru itch.io
ipornox.xxx
iqbazar.ru
iwanttodeliver.com iwanttodeliver.com
japan-bearings.ru janerikholst.se
jewelryandfiligree.com jasonpartington.com
jikoman.info javxxx18.com
juliadiets.com jerseyschinabizwholesale.us
juliaworld.net jerseywholesaleelitestore.com
jwcialislrt.com jus0wil.pp.ua
kalandranis.gr jyvopys.com
kamagragelusa.net kamagragelusa.net
katushka.net kanimage.com
kazan.xrus.org keywordbasket.com
kazinogames.lv keywordglobal.co.za
keywordsuggests.com
kidskunst.info
kino-filmi.com kino-filmi.com
kino-key.info kino2018.cc
kinobest.pl kinofak.net
kinoflux.net kinostorm.net
kiwe-analytics.com kongoultry.net
klikbonus.com konkursov.net
kometars.xyz konteiner24.com
kostenlos-sexvideos.com konturkrasoty.ru
kovrenok.ru kovesszucs.atw.hu
krasnodar.ru ladsblue.com
ku6.com lakomka.com.ua
landmania.ru lalalove.ru
landreferat.ru lawyersinfo.org
laptop-4-less.com leftofcentrist.com
larose.jb2c.me lenvred.org
law-check-nine.xyz letmacworkfaster.world
le-clos-des-alouettes.com letolove.ru
lecbter-relationships.ga lifeinsurancekb.com
lechtaczka.net liran-locks.com
ledpolice.ru liveshoppersmac.com
legalrc.biz lkbennettoutlet.online
letto.by locatellicorretor.com.br
limtu.ifmo.ru lottospring.com
linkredirect.biz louboutinshoes.xyz
lion.playtap.us lubetube.com
liveinternet.ro lumb.co
lkbennettstore.online luxmagazine.cf
localflirtbuddies.com
lol-smurfs.com
louboutinbooties.xyz
louboutinreplica.pw
lovasszovetseg.hu
lsp-awak-perikanan.com
luchshie-topcasino.ru
m3gadownload.pl m3gadownload.pl
makemoneyonline.com mackeeper-center.club
mamasuper.prom.ua magazintiande.ru
maranbrinfo.com.br magento-crew.net
marktforschung-stuttgart.com makenahartlin.com
marmitaco.cat manimpotence.com
mastershef.club marcoislandvacations.net
masturbate.co.uk masserect.com
matematikus.info matsdale.com
matpre.top
mature.free-websites.com
maxxtor.eu maxxtor.eu
meddesk.ru mcnamaratech.com
meet-flirt-dating.com medictube.ru
meetlocalchicks.com member-quiz.com
megakino.net mesto-x.com
members.ghanaweb.com mex-annushka.ru
metaxalonevstizanidine.blogspot.com millionare.com
mhtr.be mir-limuzinov.ru
michellblog.online mixed-wrestling.ru
migente.com morf.snn.gr
misandesign.se motherboard.vice.com
misterjtbarbers.com moyaterapiya.ru
mobile-appster.ru mp3films.ru
mobilierland.com mrinsidesales.com
moinozhki.com mrpornogratis.xxx
mompussy.net mwtpludn.review
moneymaster.ru
more-letom.ru
morlat.altervista.org
moscow-clining.ru
mossmesi.com
musezone.ru
musicpro.monster
musicstock.me
musicvidz.ru
mxgetcode.com
my-cash-bot.co my-cash-bot.co
mydeathspace.com myfreecams.com
mydownloadengine.com mylovelibrabry.com
myliveblog.ru myphotopipe.com
myshopmatemac.com
mystats.xyz mystats.xyz
nac-bearings.ru myyour.eu
nalogovyykodeks.ru narosty.com
nardulan.com nero-us.com
narkologiya-penza.ru networkad.net
nash-krym.info new7ob.com
nastydollars.com news-speaker.com
netallergy.ru nibbler.silktide.com
netoil.no no-rx.info
nhl17coins.exblog.jp
nodding-passion.tk nodding-passion.tk
nofreezingmac.click notebook-pro.ru
nootrino.com novgorod.xrus.org
nordvpn.com npoet.ru
november-lax.com nw-servis.ru
o-o-8-o-o.ru nyfinance.ml
o.light.d0t.ru o-o-11-o-o.com
o333o.com obiavo.su
office-windows.ru
oivcvx.website
okonich.com.ua
okuos.com
omsk.xrus.org omsk.xrus.org
onclkads.com onblastblog.online
onlineku.com one-gear.com
onescreen.cc
onko-24.com
online-sbank.ru
onlineporno.site onlineporno.site
onlywoman.org openmultipleurl.com
osagonline.ru ororodnik.goodbb.ru
osoznanie-narkotikam.net outclicks.net
page2rss.com ownshop.cf
panicatack.com ownshop.win
park.above.com panouri-solare-acoperis.com
pdamods.ru papasdelivery.ru
pdfprof.com partner-high.men
partnerline.men
partnersafe.men
partybunny.ru
pattersonsweb.com
pb-dv.ru
pc-services.ru
perederni.net perederni.net
perfectpracticeweb.com perfectpracticeweb.com
perm-profnastil.ru pertlocogasilk.tk
petsblogroll.com peugeot-club.org
photorepair.ru phormchina.com
piccdata.com pic2fly.com
picswe.com picturesify.com
piratecams.com pinkduck.ga
piter.xrus.org plastgran.com
pizda.lol playtap.us
playmsn.com popads.net
pochemychka.net popmarker.com
pogosh.com
pogruztehnik.ru
polybuild.ru
popserve.adscpm.net
popugauka.ru
porndl.org porndl.org
porngalleries.top porno.simple-image.com.ua
pornogad.com
pornohd1080.online
pornolook.net pornolook.net
pornosmola.info pornosok.ru
pornstartits.xblog.in pornstartits.xblog.in
praisong.net positive2b.ru
pricheskaonline.ru pospr.waw.pl
princeadvantagesales.com pr-ten.de
privatamateure.com prchecker.info
privetsochi.ru
prizesbook.online
prod2016.com
prodvigator.ua prodvigator.ua
profitsport.club proekt-mos.ru
profit-opportunity.com
projefrio.com.br projefrio.com.br
propranolol40mg.blogspot.com prombudpostach.com.ua
pskcijdc.bloger.index.hr promover.org
pst2017.onlinewebshop.net pssucai.info
py100.ru quick-seeker.com
qcstrtvt.bloger.index.hr radiogambling.com
qitt.ru realtytimes.com
quangcaons.com redirect.trafficreceiver.club
qwertty.net rednise.com
qwesa.ru
r-e-f-e-r-e-r.com
randki-sex.com
ranksonic.info
raschtextil.com.ua
ratemodels.net
realresultslist.com
refererx.com
registratciya-v-moskve.ru
registrationdomainsite.com registrationdomainsite.com
reimageplus.com remontbiz.ru
remorcicomerciale.ro replica-watch.ru
rentehno.ru ribieiendom.no
ruspdd.com rollercoin.com
rusprostitute.com rp9.ru
scanmarine.info rucrypt.com
scanner-andrew.top russia-today-video.ru
scanner-brian.top saneitconsulting.com
scanner-fred.top scanner-alexa.top
scanner-nelson.top scanner-barak.top
scenicmissouri.us scanner-donald.top
screentoolkit.com scanner-ivan.top
scrnet.biz.ua scanner-marwin.top
searchkut.com schlampen-treffen.com
se-welding.ru
search.pch.com
searchwebknow-a.akamaihd.net searchwebknow-a.akamaihd.net
sendearnings.com seasonvar.ru
seo-smm.kz see-your-website-here.com
semrush.com
seo-traffic-ranking.info
seofirmreviewsus.info seofirmreviewsus.info
seorussian.ru seozoom.it
sexphoto.site sexgalleries.top
sextracker.com sexkvartal.com
sexyteens.hol.es sexpornotales.com
sexs-foto.top
sexsearch.com
sexvideo-sex.com
sexyali.com
shakhtar-doneck.ru
shisha-swag.de shisha-swag.de
sibecoprom.ru shoppytoolmac.com
siha.de sibdevice.ru
similardeals.net silktide.com
sinestesia.host.sk sisi-go.ru
sitevaluation.org site-speed-checker.site
skytraf.xyz skanninge.se
small-games.biz skatestick.bid
smsactivator.ru slimcdn.com
snowplanes.com smartpet.ru
snworks.com snow.nvr163.com
snyatie-lomki-v-stacionare.ru sobecjvuwa.com.ru
soc-proof.su soc-proof.su
socialmediasuggest.com sohoindia.net
socialsignals24.com spammen.de
sockshares.tv special-porn.com
solitaire-game.ru spitfiremusic.com
speedup-my.site sport-video-obzor.ru
sps-shop.com
spy-sts.com
stairliftsarea.com
stal-rulon.ru stal-rulon.ru
stard.shop start.myplaycity.com
starpages.net staynplay.net
sundrugstore.com stmassage.ru
surfbuyermac.com stop-gepatit.te.ua
svolze.com store-rx.com
tabletkinaodchudzanie.com.pl streamin.to
taxi-v-eisk.ru stroicol.net
teesdaleflyballclub.co.uk strongholdsb.ru
telefonsexsofort.tk students-cheapskate.ml
telesvoboda.ru support.nopeas.sk
tetracsaudi.com svbur.ru
thebluffs.com swsociety.se
thecounter.com taihouse.ru
thegalerie.eu taranerymagesswa.blogspot.com
tedxrj.com
theallgirlarcade.com
theprofitsmaker.net theprofitsmaker.net
thetattoohut.com thewebsitetemplate.info
tizanidineoral.blogspot.com titan-ads.life
token-lab.org titan-cloud.life
toolsky.com tizanidinefibromyalgia.blogspot.com
torrent-newgames.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
torrents.cd tizanidineshowupondrugtest.blogspot.com
toy-shop.top tjkckpytpnje.com
traxdom.ru top10-way.com
truebeauty.cc topcasinoratings.ru
tubeline.biz torrentgamer.net
tubeoffline.com torrentred.games
turist-strani.ru track112.site
uggsale.online tracksurf.daooda.com
ugogo.info tradedeals.biz
traffixer.com
trahic.ru
triumf-realty.ru
truck-land.life
tube8.com
tvand.ru
tvorozhnaja-zapekanka-recept.ru
twodollarshows.com
ultimatesetnewfreeallsoftupgradesystems.pw
umnovocaminho.com
unacittaconte.org
unrealcommander.biz unrealcommander.biz
uptime-gamma.net uptime-gamma.net
uptimechecker.com urdoot.win
utiblog.fr
utrolive.ru
uvozdeckych.info
v720hd.ru
vancleefreplica.pw vancleefreplica.pw
vduplo.ru vapeface.club
veles.shop vclicks.net
ventelnos.com versaut.xxx-cam.webcam
viktoria-center.ru video-production.com
vinylvault.co.uk videosbox.ru
vip-parfumeria.ru
virus-schutzmasken.de virus-schutzmasken.de
vita.com.hr visionwell.com.cn
voloslove.ru vltai.com
voronezh.xrus.org volgograd.xrus.org
vpnmouse.com volume-pills.biz
wahicbefa31.soup.io voprosotvet24.ru
wallpaperaccess.com vroze.com
warezkeeper.com vzlomsn.org
watchmyfb.pl wait3sec.org
waysbetter.cn warningzscaler.heraeus.com
watch-movies.ru
waycash.net
wdfdocando.com wdfdocando.com
webcamdevochka.com webradiology.ru
weblibrary.win wedding-salon.net
weblo.com weightatraining.com
website-audit.com.ua wesharepics.site
websiteexplorer.info wikes.20fr.com
websitevaluebot.com windowssearch-exp.com
weprik.ru witherrom55.eklablog.fr
whatsappbot.flyland.ru wordpress-crew.net
womens-journal.net wovis.site
wordkeyhelper.com wowcasinoonline.ooo
worldlovers.ru ww1943.ru
worldmusicfests.com x-porno.video
worldtraveler.world x-true.info
www.888.com xaijo.com
www.arenda-yeisk.ru xerox-douglas.cf
www.pinnacle-bets.com xmnb.net
x-rates.ru xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
x-stars.ru xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xn------7cdbapdecfd4ak1bn0amjffj7afu3y.xn--p1ai xn--b1addnj3cah.xn--p1ai
xn-----6kcaacnblni5c5bicdpcmficy.xn--p1ai xnxx699.com
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xn----9sbubg3ambdfl1j.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--hxazdsfy.blogspot.com
xolodremont.ru xolodremont.ru
xpresscare.ru
xtremeeagles.net
xvideosporn.biz xvideosporn.biz
xxxhdvideo.site xxx-cam.webcam
xxxkaz.org xxxmania.top
youporn-ru.com y8games-free.com
yoursearch.me yaderenergy.ru
yuarra.pluto.ro yellocloud.be
yuweng.info yopoint.in
zapatosenventa.info youtubedownload.org
zapiszto.pl youtuhe.com
zarepta.com zaimite.ru
zdesoboi.com zdesformula.ru
zlatnajesen.com zeleznobeton.ru
znaniyapolza.ru zetgie.com.pl
znaturaloriginal.com zigarettenonl.canalblog.com
zog.link zionstar.net
zoosexart.com zreloeporno.tv
zophim.me zscalerone.net
zrus.org

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
6hopping.com 000free.us
abiente.ru 11pikachu.ru
adsloads.com 12masterov.com
1314dh.com
1hwy.com
1stat.ru
3xforum.ro
888.com
addblueoff.com.ua
adviceforum.info
afora.ru afora.ru
agadelha.com.br alert.scansafe.net
aludecor.info allbizne.co.ua
amazingpic.net altamayoztourism.com
alvtank.se
amateurlivechat.org
amehdaily.com
android-style.com
angigreene.com angigreene.com
apiallgeniusinfo-a.akamaihd.net anti-virus-removal.info
ardimobileinfo.ml apartment.ru
arkkivoltti.net apibetweenlinesn-a.akamaihd.net
asacopaco.tk appfastplay.com
asia-forum.ru arendakvartir.kz
asrvrep-a.akamaihd.net autochoixspinelli.com
ausmepa.org.au bazaronline24.ru
banki76.ru besthomemadepornsites.com
bbsoldes.fr binaryoptionscops.info
bdsmgalls.net bio-optomarket.ru
best-way.men bioscorp.ru
bestbrainprod.win bizfly.info
bg6s0.com blagovest-med.ru
bibys.com bonzbuddy.com
bio.trade-jp.net bradleylive.xyz
bioca.org brokergid.ru
biteg.xyz bugshoot.cn
blackplanet.com buqayy0.livejournal.com
bluesalt.co buttons-for-your-website.com
bongacams.com c1.onedmp.com
builtwith.com carabela.com.do
buybest1.biz cardsdumps.com
buyk.host.sk casinos4dummies.co.uk
bzero1jewelry.net cdnnetwok.xyz
canrioloadm.gq cheapbelstaff.online
carstrends2015.com cherrypointplace.ca
castingbank.ru ciproandtizanidine.blogspot.com
checkpagerank.net citysecurity.nu
chocolateslim-u-srbiji.com clicksor.com
christianlouboutinreplica.pw cocolyze.com
cjmc.info collegeessay19.blogspot.ru
clickhype.com companies-catalog.com
clothesforcash.com copypaste.traderzplanet.in
coaching-netz.info courtshipgift.com
cocyq.inwtrade.com cryptoswap.biz
coindirect.io dairyindia.in
cometorussia.net daooda.com
compliance-alex.top dfwu1013.info
compliance-alex.xyz digitalassetmanagement.site
compliance-donald.xyz div.as
conboy.us diy-handmade-ideas.com
continent-e.tv drugspowerstore.com
cookie-law-enforcement-ff.xyz ecookna.com.ua
cool-wedding.net ed-shop01.ru
copenergo.ru
crest3d.ru
crystalslot.com
dalavia.ru
detskie-konstruktory.ru
detsky-nabytek.info
dev33.dioniqlabb.se
documentbase.net
domznaniy.ru
druzhbany.ru
earl-brown.info
empirepoker.com empirepoker.com
essayassist.com eropho.com
fable.in.ua erotik-kostenlos.net
favorcosmetics.com ezofest.sk
fermersovet.ru facebook-mobile.xyz
filesmonster.porn facecup.top
findpik.com fanpagerobot.com
flowersbazar.com fashion.stellaconstance.co
forum-engineering.ru feargames.ru
forum.poker4life.ru findthe.pet
foxjuegos.com fitness-video.net
foxweber.com forexgb.ru
frameimage.org frankofficial.ru
free-social-buttons.xyz freemags.cc
freegamesplay.online freeuploader.ml
freesitetest.com freshwallpapers.info
froggytube.com freza-sverlo.ru
fun2cell.net funcrushgames.com
fyl.com.ru gambarkatabaru.com
galleryawesome.com glasof.es
gamedayhouse.com glastecfilms.com.my
gardene.ru globalscam.ga
gaygalls.net glogow.pl
get-seo-domain.com goldadpremium.com
gettpromos.com greatfind-a.akamaihd.net
girlsfuckdick.com gwebtools.com.br
glcomputers.ru hahashka.ru
gloverid.site hdfreeporno.net
goo.ne.jp hdpixion.com
goodhousekeeping.com healing-dysplasia.ru
google-liar.ru hitsbox.info
googleseo.com.tr hkdiiohi.skyrock.com
gopro-online.info iboss.com
gsasearchenginerankerhelp.com igtools.club
harmonyglen.us immigrational.info
helpmymacfaster.trade in-tandem.co
hikesearch.net inspiring-desperate.tk
hotxnights.info insurple.com
howlongdoestizanidinestayinyoursystem.blogspot.com intellego.info
https-legalrc.biz io9.com
ideibiznesa2015.ru iomoio.net
ideoworld.org iqoption.pro
ilmexico.com iskalko.ru
imabase.com jackpotchances.com
imk.com.ua javxxx18.com
infobanks.ru
investmac.com
invitefashion.com
iron-age.info
iwantmyfreecash.com
javidol.site
jerseysbizwholesalecheap.com jerseysbizwholesalecheap.com
jerseyswholesalechinalimited.com kalb.ru
jolic2.com kamorel.com
jpcycles.com kerch.site
justbcause.com keywordsking.com
justkillingti.me kino2018.club
kazka.ru kiprinform.com
kevblog.top klejonka.info
keyhantercume.com kongruan.com
kinofak.net konkursowo-24.pl
kladrus.ru kreativperlen.ch
klikbonus.com kunstaktien.info
knogg.net kvartira-sutochno.com
kosmetyki.tm.pl lalalove.ru
kvartiry-remont.ucoz.ru lancheck.net
laapp.com law-check-eight.xyz
labelwater.se lawyers.cafe
leftofcentrist.com le-clos-des-alouettes.com
luckyday.world lechenie-gemorroya.com
lyrics.home-task.com lenpipet.ru
macotool.com lignofix.ua
magicdiet.gq lion.misba.us
maglid.ru liverpool.gsofootball.com
mainlinehobby.net loadingpages.me
lomb.co
makemoneyonline.com makemoneyonline.com
managerpak204.weebly.com masserect.com
meddesk.ru mastershef.club
microstatic.pl mathgym.com.au
midst.eu mearns-tractors.co.uk
mixed-wrestling.ru mebel-vstroika.ru
monitorwebsitespeed.com michellblog.online
moomi-daeri.com mini.7zap.com
mouselink.co movies-in-theaters.net
movpod.in mustat.com
moz.com muzaporn.com
mug-na-chas-moscow.ru myfreecams.com
muztops.ru mymobilemoneypages.com
my-big-family.com nbsproject.ru
myanyone.net oivcvx.website
mykings.pw oklogistic.ru
mylovelibrabry.com onlineporno.site
myphotopipe.com orsonet.ru
myseoconsultant.com paydayonlinecom.com
najaden.no pc-services.ru
nalogovyykodeks.ru perfectplanned.com
narco24.me perl.dp.ua
nash-krym.info pharmacyincity.com
ningessaybe.me picscout.com
nosecret.com.ua picturesmania.com
oecnhs.info pivka.xyz
oranga.host.sk porno.simple-image.com.ua
outshop.ru portal-eu.ru
paramountmarble.co.uk preg.marketingvici.com
partnerworkroom.men privhosting.com
picanalyzer.data-ox.com prizefestival.mobi
pinkduck.ga prizesbook.online
piski.top psoriasis-file.trade
plastweb.ru pst2017.onlinewebshop.net
pony-business.com pushdata.sendpulse.com
pornoslive.net qpypcx.com
portside.xyz ranksignals.com
privat-girl.net registrydomainservices.com
productarium.com remontvsamare.su
professionalwritingservices15.blogspot.ru riralmolamsaca.tk
prosperent.com rosbalt.com.ua
prostitutki-astana.org rrutw.com
push-ad.com ru-mediaget.ru
puzo2arbuza.ru samoiedo.it
pwwysydh.com sarf3omlat.com
py100.ru savingsslider-a.akamaihd.net
rangoman.date sbdl.no
rapevideosmovies.com scanner-jess.top
recordpage-a.akamaihd.net schoolfiles.net
reining.lovasszovetseg.hu seozoom.it
revistaindustria.com sexad.net
rewardit.com sexcamamateurchat.com
ringtonepartner.com sigmund-freud.co.uk
saneyes.com skypasss.com
savememoney.co.za slowmac.tech
seansonline24.pl sluganarodu.ru
sel-hoz.com snegozaderzhatel.ru
semaltmedia.com snowplanes.com
sendearnings.com soblaznu.net
seoheap.com solnplast.ru
shoplvlv.us spaceshipad.com
site.ru spb.ru
sivs.ru sport7777.net
skachat-besplatno-obrazcy.ru standardchartered-forex.com
sloopyjoes.com stats-public.grammarly.io
sodexo.com stillmiracle.com
spywarebegone.com stjamesschool.info
studiokamyk.com.pl stop-zavisimost.com
symphonyintegratedhealthcare.com strana-solnca.ru
tabletkinaodchudzanie.com.pl strigkaomsk.ru
tacbibirfa.tk summerlinhomes411.info
tds-advert002.info surfbuyermac.com
tds-advert005.info svensk-poesi.com
template-kid.com taboola.com
threecolumnblogger.com taximytishi.ru
timetorelax.biz tetracsaudi.com
tizanidinehclsideeffects.blogspot.com thelottosecrets.com
tk-assortiment.ru tizanidinedruginteractions.blogspot.com
topanasex.com torrentred.games
trion.od.ua tracklead.net
uk-zheu20.ru trailer.cinemaflix.website
unacittaconte.org trustedmaccleaner.com
urlcut.ru turbodsp.com
urlopener.blogspot.com.au ucoz.ru
varbergsvind.se uptime-alpha.net
vellings.info userequip.com
via-energy-order.com vadimkravtcov.ru
via-gra.webstarts.com vapeface.club
vikistars.com vapomnoncri.tk
vintontech.info vapsy.com
verymes.xyz
video-chat.love
vkak.ru
voloo.ru voloo.ru
vzube.com vremya.eu
w3javascript.com vsesubwaysurfers.com
waterpurifier.club vucms.com
webix.me wallpaperaccess.com
whatistizanidinehclusedfor.blogspot.com watchdogs-2.ru
wholesalenfljerseys.us.com websiteaccountant.de
win-spy.com webstatsdomain.org
womens-journal.net wesharepics.com
word-vorlagen.net wholesalejerseys-cheapest.com
www.get-free-traffic-now.com whos.amung.us
www.kabbalah-red-bracelets.com winx-play.ru
wygraj-teraz.com wovis.site
x-mix.info xbaboon.com
x5market.ru xkaz.org
xdoza.com xtremeeagles.net
xn--80adaggc5bdhlfamsfdij4p7b.xn--p1ai yaderenergy.ru
xn--80aodinpgi.xn--p1ai youbloodyripper.com
xtubeporno.net youradexchange.com
y8games-free.com yubikk.info
zeg-distribution.com zaimite.ru
zrizvtrnpale.tk zakonobosago.ru
zazagames.org
zed21.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.201 0; 10.1.1.94 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.07.3232 ### Version: V4.2022.07.3233
### Updated: Sat Jul 16 22:01:28 UTC 2022 ### Updated: Sun Jul 17 10:03:28 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.07.3232 ### Version: V4.2022.07.3233
### Updated: Sat Jul 16 22:01:28 UTC 2022 ### Updated: Sun Jul 17 10:03:28 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##