vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.10.2636 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-10-07 10:09:13 +00:00
parent 0721960fb6
commit 512adef51f
46 changed files with 75341 additions and 75341 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2635 #### Version: V4.2021.10.2636
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.10.2635 #### Version: V4.2021.10.2636
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.10.2635 #### Version: V4.2021.10.2636
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9282
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.10.2635 #### Version: V4.2021.10.2636
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9286
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9286
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9286
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9286
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

9286
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

9286
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

9286
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2635 2636

9286
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9286
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.186.54.88 0; 52.170.139.251 0;

9286
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

9286
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

9286
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

9286
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.186.54.88 0; 52.170.139.251 0;

9286
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
52.186.54.88 0; 52.170.139.251 0;

9286
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 1; 10.1.0.195 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
LinksManager Blow
Mag-Net Bolt
Majestic12 CherryPicker
Mister PiX ChinaClaw
Spanner CrazyWebCrawler
Virusdie Metauri
WWWOFFLE Toweyabot
Zitebot WWW::Mechanize
ZyBorg WebCopier
instabid instabid

318
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404checker 80legs
ADmantX
AIBOT AIBOT
ALittle Client ALittle Client
ASPSeek ASPSeek
Aboundex Abonti
AiHitBot AhrefsBot
AllSubmitter Alligator
AlphaBot Anarchy
Anarchie Apexoo
Anthill AwarioSmartBot
Aspiegel BBBike
Asterias
Attach
BDCbot BDCbot
BDFetch BackDoorBot
BLEXBot
Backlink-Ceck Backlink-Ceck
BacklinkCrawler BacklinkCrawler
Badass
Bandit
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bigfoot
Bitacle
BlackWidow
Blackboard Blackboard
BlowFish Blow
Bolt BotALot
BuiltWith Brandprotect
Buck
Buddy
Bullseye
BunnySlippers
BuzzSumo BuzzSumo
CATExplorador CATExplorador
CSHttp CSHttp
Calculon
CazoodleBot CazoodleBot
CheTeam CheeseBot
CherryPicker
ChinaClaw
Chlooe Chlooe
Claritybot Claritybot
Cliqzbot
Cocolyzebot
Collector
Copier Copier
Crawling at Home Project CopyRightCheck
Craftbot
CrunchBot CrunchBot
Curious Curious
DBLBot
DIIbot
DSearch DSearch
DataCha0s DTS Agent
DigitalPebble DatabaseDriverMysqli
Dirbuster Digincore
Dispatch Disco
Discoverybot
DittoSpyder
DnyzBot
DomCopBot DomCopBot
DomainSigmaCrawler Domains Project
Dotbot
Download Wonder Download Wonder
EMail Siphon Dragonfly
EMail Wolf
EasyDL EasyDL
EirGrabber Ebingbong
Evil EroCrawler
Exabot ExtLinksBot
Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms Ezooms
FDM FDM
FHscan
FemtosearchBot FemtosearchBot
Firefox/7.0 Flunky
Foobot
Freeuploader
FrontPage FrontPage
FyberSpider Fuzz
GT::WWW G-i-g-a-b-o-t
Genieo GalaxyBot
Gigabot GermCrawler
Go!Zilla GetRight
GetWeb
Getintent
Go-Ahead-Got-It Go-Ahead-Got-It
GrabNet GoZilla
Grafula Gotit
GridBot GrapeFX
HEADMasterSEO
HMView
HTMLparser HTMLparser
HTTP::Lite Haansoft
HaosouSpider HaosouSpider
Havij Id-search
Hloader IndeedBot
HybridBot
IRLbot
IlseBot
Intelliseek Intelliseek
Internet Ninja Internet Ninja
IstellaBot InternetSeer
JOC Web Spider JamesBOT
Jbrofuzz
JetCar JetCar
Joomla
Jorgee Jorgee
JustView JustView
Kenjin Spider Kozmosbot
Keyword Density Lanshanbot
LWP::Simple
Larbin Larbin
LeechFTP Leap
Libwhisker LeechGet
LieBaoFast LibWeb
LinkpadBot Likse
LinqiaMetadataDownloaderBot LinksManager
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey
Lipperhey Spider Lipperhey Spider
Litemage_walker Litemage_walker
MFC_Tear_Sample Lmspider
MIDown tool MSFrontPage
MJ12bot MSIECrawler
MTRobot
Magnet Magnet
Majestic SEO Majestic12
MarkMonitor MarkWatch
Masscan
MauiBot MauiBot
Mediatoolkitbot Mb2345Browser
MeanPath Bot
Mister PiX Mister PiX
Mojeek Mojeek
Morfeus Fucking Scanner Name Intelligence
Mozlila NearSite
NICErsPRO Nessus
Net Vampire
NetAnts
NetLyzer NetLyzer
NetSpider NetMechanic
NetZIP Netcraft
Nettrack
Netvibes Netvibes
Niki-bot NextGenSearchBot
Nibbler
Nikto
NimbleCrawler
Nimbostratus
Nmap Nmap
Nuclei Offline Navigator
Offline Explorer OnCrawl
OpenVAS
Openfind
Openvas
OrangeBot
OutfoxBot OutfoxBot
PageGrabber POE-Component-Client-HTTP
PageScorer Page Analyzer
PageThing.com PageAnalyzer
Panscient
Papa Foto Papa Foto
PeoplePal
Petalbot Petalbot
PictureFinder
Piepmatz Piepmatz
PleaseCrawl
ProPowerBot ProPowerBot
Pu_iN
Pump Pump
Quick-Crawler QueryN Metasearch
RSSingBot
RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex
RankingBot RankingBot
RankingBot2 RankingBot2
Re-re RedesScrapy
Reaper RepoMonkey
RebelMouse
Rogerbot
SBIder
SEOkicks SEOkicks
SEOlyticsCrawler SEOkicks-Robot
SEOstats
SISTRIX SISTRIX
SMTBot
SalesIntelligent
ScanAlert
ScoutJet ScoutJet
Scrapy
ScreenerBot
ScrepyBot ScrepyBot
SeoSiteCheckup Semrush
Siphon SemrushBot
SentiBot
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker Sitevigil
SmartDownload Snake
Snapbot Snapbot
Snoopy SpankBot
Sottopop Spinn3r
SpaceBison Sqworm
Spammen Steeler
Sqlworm Stripper
Sucker
Sucuri Sucuri
Surfbot SuperBot
Swiftbot SuperHTTP
Szukacz SurveyBot
T8Abot Suzuran
Telesoft Telesphoreo
Telesphorep
The Intraformant
TightTwatBot TightTwatBot
Titan Tracemyfile
Toweyabot
Trendictionbot Trendictionbot
True_Robot TurnitinBot
Turingos Twice
Turnitin
URLy Warning URLy Warning
URLy.Warning
Upflow Upflow
VB Project V-BOT
Vacuum VCI
VeriCiteCrawler
VidibleScraper VidibleScraper
Virusdie Voltron
VoidEYE WISENutbot
WASALive-Bot WWW-Mechanize
Wallpapers/3.0 WWWOFFLE
WallpapersHD
Web Auto Web Auto
Web Fetch Web Collage
Web Enhancer
WebAuto
WebBandit WebBandit
WebCollage WebCollage
WebFetch WebCopier
WebEnhancer
WebGo IS WebGo IS
WebReaper WebImageCollector
WebStripper WebPix
WebWhacker
Webalta
WebmasterWorldForumBot
WebsiteQuester WebsiteQuester
Whack
Whacker Whacker
Widow Widow
WinHTTrack WinHTTrack
WiseGuys Robot
Wprecon
Xenu Xenu
YoudaoBot
ZmEu ZmEu
ZoomBot ZoomBot
ZoominfoBot archive.org_bot
ZyBorg arquivo-web-crawler
arquivo.pt autoemailspider
check1.exe
clark-crawler clark-crawler
cognitiveseo
com.plumanalytics com.plumanalytics
crawl.sogou.com crawl.sogou.com
heritrix crawler.feedback
dataforseo.com
gopher
internetVista monitor internetVista monitor
isitwp.com ips-agent
lwp-request
magpie-crawler magpie-crawler
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
page scorer pcBrowser
polaris version polaris version
probe-image-size probe-image-size
s1z.ru s1z.ru
scalaj-http satoristudio.net
scan.lol seoscanners
seobility serpstatbot
sexsearcher sogouspider
siteripz
sysscan
tAkeOut
trendiction.com trendiction.com
ubermetrics-technologies.com trendiction.de
voyagerx.com
webpros.com
x22Mozilla
xpymep1.exe
zauba.io zauba.io
zgrab

302
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider
404checker
80legs
ADmantX
ALittle Client ALittle Client
Aboundexbot ASPSeek
Acunetix Aboundex
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot Aipbot
AllSubmitter
Alligator Alligator
Ankit AlphaBot
Aspiegel Anarchie
Backlink-Ceck Anarchy
BacklinkCrawler Anarchy99
BetaBot Apexoo
AwarioSmartBot
BBBike
BDCbot
BDFetch
BackDoorBot
Barkrowler
Battleztar Bazinga
Bigfoot Bigfoot
Black Hole
Blackboard Blackboard
BlowFish Buddy
Bolt
BuiltBotTough BuiltBotTough
BuiltWith Bullseye
BuzzSumo BunnySlippers
CCBot CATExplorador
CODE87 Calculon
CSHttp CazoodleBot
Cegbfeieh Cegbfeieh
CheeseBot CheTeam
CherryPicker
ChinaClaw ChinaClaw
Chlooe Collector
Cloud mapping
Cosmos
CrunchBot CrunchBot
CyotekWebCopy
DIIbot DIIbot
DTS Agent Demon
DataCha0s Devil
DatabaseDriverMysqli
Digincore Digincore
DigitalPebble
Dirbuster
Disco
Discobot
Discoverybot Discoverybot
Dispatch DomCopBot
DomainCrawler
DomainSigmaCrawler
DomainStatsBot DomainStatsBot
Download Wonder
Dragonfly Dragonfly
Drip Ecxi
ECCP/1.0 EirGrabber
EasyDL
Ebingbong
EroCrawler EroCrawler
Express WebPictures Exabot
Extreme Picture Finder ExtLinksBot
Extractor
ExtractorPro
EyeNetIE EyeNetIE
Ezooms
FDM FDM
FemtosearchBot
Fimap
FlashGet FlashGet
Foobot
FrontPage FrontPage
Fuzz FyberSpider
G-i-g-a-b-o-t
GT::WWW GT::WWW
Genieo
GermCrawler GermCrawler
GetRight GetRight
Gigabot GetWeb
Gotit Go-Ahead-Got-It
Grafula GrabNet
HEADMasterSEO Grabber
GridBot
HMView HMView
HTTP::Lite HTTP::Lite
HTTrack HTTrack
Haansoft Harvest
Hloader
Humanlinks
IDBot
Iblog Iblog
Id-search
IlseBot IlseBot
Intelliseek Image Fetch
Iria Image Sucker
InfoNaviRobot
Internet Ninja
InternetSeer
Iskanie Iskanie
IstellaBot
JOC Web Spider JOC Web Spider
JamesBOT
Jbrofuzz
JetCar JetCar
Jetty Jetty
Kenjin Spider Joomla
JustView
Keyword Density Keyword Density
Kinza Kinza
Kozmosbot
LNSpiderguy
Leap
LeechGet
LexiBot LexiBot
Lftp Lightspeedsystems
LieBaoFast LinkScan
Linkbot Linkbot
LinkextractorPro LinksManager
LinkpadBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot Litemage_walker
Lmspider Lmspider
MSIECrawler Ltx71
MTRobot
Mail.RU_Bot
Majestic-SEO
Mass Downloader Mass Downloader
MauiBot
Mb2345Browser
MeanPath Bot
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Mister PiX Metauri
NICErsPRO Microsoft URL Control
NPbot Mojolicious
Morfeus Fucking Scanner
Mozlila
Musobot
Nameprotect
Navroad Navroad
NetLyzer Nessus
NetAnts
NetSpider NetSpider
NetZIP NetZIP
Netvibes
NextGenSearchBot
Nibbler
Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Ninja Nmap
Nuclei Nuclei
Offline Navigator Nutch
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openfind
OrangeBot OrangeBot
PHPCrawl OrangeSpider
OutfoxBot
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageAnalyzer
PageGrabber
PageThing.com
Pandalytics Pandalytics
Panscient Pavuk
Papa Foto PeoplePal
Picscout Pi-Monster
Picsearch Picsearch
PictureFinder
Piepmatz
Pimonster Pimonster
Pixray Pixray
Probethenet PleaseCrawl
Psbot ProWebWalker
Pump Pump
QueryN Metasearch
Quick-Crawler
RankActive
RankActiveLinkBot
RankFlex
Re-re
RebelMouse
Recorder Recorder
RedesScrapy RocketCrawler
Rogerbot Rogerbot
SBIder
SEOkicks-Robot
SEOlyticsCrawler
SEOprofiler SEOprofiler
SEOstats SISTRIX
SalesIntelligent Scrapy
Scanbot
ScoutJet
ScreenerBot ScreenerBot
Searchestate SearchmetricsBot
Semrush SemrushBot
SentiBot SentiBot
Seomoz Seomoz
Shodan Shodan
SlySearch Site Sucker
Snake SiteCheckerBotCrawler
Snapbot SiteSucker
SocialRankIOBot Sitebeam
Sogou web spider Siteimprove
Sottopop Sitevigil
Spammen SmartDownload
SpankBot SpaceBison
Spanner
Spbot Spbot
SputnikBot
Sqlmap Sqlmap
Steeler Stripper
Sucuri SuperHTTP
SuperBot SurveyBot
Surfbot
Suzuran Suzuran
Swiftbot T0PHackTeam
TeleportPro T8Abot
Thumbor Teleport
TightTwatBot Telesoft
Tracemyfile Telesphoreo
Telesphorep
TheNomad
Toata
Toweyabot
Trendictionbot
True_Robot True_Robot
Twice Typhoeus
URLy Warning
URLy.Warning
VB Project VB Project
Vacuum Vacuum
VidibleScraper VeriCiteCrawler
Virusdie VoidEYE
WBSearchBot
WEBDAV WEBDAV
WWW-Mechanize WISENutbot
Wallpapers WWW-Collector-E
WWW::Mechanize
WWWOFFLE
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Enhancer Web Collage
Web Fetch Web Fetch
Web Sucker Web Sucker
WebCopier WebBandit
WebCollage
WebEnhancer WebEnhancer
WebFetch WebFetch
WebFuck WebImageCollector
WebLeacher WebLeacher
WebSauger WebSauger
WebStripper
WebWhacker
WebmasterWorldForumBot
Webshag
Whack Whack
Whacker Whacker
Whatweb Whatweb
WinHTTrack Who.is Bot
WiseGuys Robot
Wonderbot
Woobot
Wotbox Wotbox
Xaldon_WebSpider Xaldon_WebSpider
Zade Zauba
Zermelo Zermelo
Zitebot
ZoomBot ZoomBot
ZoominfoBot
ZumBot
archive.org_bot archive.org_bot
arquivo-web-crawler clark-crawler
backlink-check cognitiveseo
coccocbot-web
crawler.feedback crawler.feedback
crawler4j crawler4j
dataforseo.com
demandbase-bot demandbase-bot
domainsproject.org facebookscraper
eCatch internetVista monitor
evc-batch isitwp.com
lwp-request lwp-request
lwp-trivial lwp-trivial
meanpathbot
mediawords mediawords
muhstik-scan
netEstate NE Crawler netEstate NE Crawler
oBot
page scorer page scorer
pcBrowser pcBrowser
polaris version
ripz ripz
satoristudio.net satoristudio.net
scan.lol seoscanners
seobility
seostar seostar
sogouspider sexsearcher
sitechecker.pro
spyfu
tAkeOut tAkeOut
trendiction.com
webmeup-crawler webmeup-crawler
webpros.com
webprosbot
x09Mozilla
zauba.io zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
best-coupon-offer.com acads.net
bestkfiledxd.cf azartmix.com
dinkolove.ya.ru cete.ru
everytalk.tv friendflnder.com
fx-brokers-review.com m5home.ru
gallictures.com make-money-online.com
glavtral.ru millionare.com
onlinedomains.ru searchlock.com
villacoloniale.com videositename.com
zocaparj.kz youtuhe.com

918
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
123kuma.com 14b.info
123movies.love 1hwy.com
1pamm.ru 1pamm.ru
2000k.ru 300richmond.co.nz
2020iscoming.info
3-letter-domains.net
3dracergames.com 3dracergames.com
3weekdiet.com 404.mn
4istoshop.com 4senses.co
4ur.click 4webmasters.org
5forex.ru 51.la
70casino.online 7search.com
888.com aa8780bb28a1de4eb5bff33c28a218a930.com
8lufu.com aa9bd78f328a6a41279d0fad0a88df1901.com
aasoldes.fr adamoads.com
activecampaign.dreamhosters.com adclickservice.com
adconscious.com adidas.frwebs.fr
adloads.com
advancedmassagebysara.com
advertisingtag.net advertisingtag.net
affiliate-fr.com alarmobninsk.ru
agahinameh.com alienwheels.de
agroeconom.kz alive-ua.com
alekseevec.ru alldezservice.kz
alphacarolinas.com allfinweb.com
arabseyes.com allwomen.info
ardimobileinfo.ml amanda-porn.ga
articlesdirectoryme.info ameblo.jp
anal-acrobats.hol.es
android-systems.ru
anlimebel.kiev.ua
apccargo.com
apiappenableinfo-a.akamaihd.net
apidigidocketnet-a.akamaihd.net
approved.su
artdestshop.eu
autochoixspinelli.com autochoixspinelli.com
autotop.com.ua avkzarabotok.info
aviva-limoux.com b2b-lounge.com
avtolombard-krasnodar.ru
ayanaspa.com
ayeartoforget.com
azartmix.com
azbukadiets.ru
babieca.com
backgroundpictures.net backgroundpictures.net
bambi.ck.ua baikaleminer.com
bannerswap.com balkanfarma.ru
bbtec.net bamo.xsl.pt
beachpics.com bannerads.de
belsetirehafi.tk bdsmgalls.net
best-games.today best-deals-products.com
best-way.men
bestdraws.com bestdraws.com
bestkfiledxd.cf bestofferswalkmydogouteveryday.gq
bestpriceninja.com bet365.com
bestwebsitesawards.com betteroffers.review
bif-ru.info beztuberkuleza.ru
biglistofwebsites.com bidr.trellian.com
blagovest-med.ru bikini-image.com
blue-square.biz billiard-classic.com.ua
boanonihaca.tk bimatoprost-careprost.com
boazpower.com bjorkbacken.nu
bobba.dzaba.com blogig.org
bodybuilding-shop.biz bloglag.com
bookmark4you.com.biz bracketsmackdown.com
brains2.biz brainboosting.club
brandehk.dk brandedleadgeneration.com
brillianty.info brendbutik.ru
brokergid.ru brimstonehillfortress.org
buntube.net briomotor.co
burger-imperia.com browserprotecter.com
buttons-for-website.com brus-vsem.ru
buyantiviralwp.com btnativenav.com
buynorxx.com buchananshardware.com
buyscabiescream.com bukleteg.xyz
buzzsumo.com buqayy0.livejournal.com
bvlgaribracelet.xyz buy-cheap-online.info
bwlx.prepedu.cn buyessay3.blogspot.ru
camdenmemorials.com buyparajumpers.online
camdolls.com cah.io.community
candlehandmade.com callejondelpozo.es
carapics.com calstaterealty.us
cardiosport.com.ua cand.jp
cartechnic.ru capture-room.com
cascadelink.org cartujano-pre.de
casinorewards.info cayado.snn.gr
cbb1smartlist12.click celejihad.info
cdn.walkme.com cenoval.ru
celebsopics.com chatroulette.life
cenokos.ru chcu.net
censys.io checkm8.com
chatseo.com christianlouboutinshoes.xyz
checkhit.com cialisovercounteratwalmartusa.com
chocolateslim-en-france.com ciproandtizanidine.blogspot.com
cinemaenergy-hd.ru
cjmc.info
clayvasedesigns.tk
clickbank.net clickbank.net
coinsspb.com clickso.com
com-cleaner.systems clubfashionista.com
compliance-josh.top cmd.kz
confib.ifmo.ru cocyq.inwtrade.com
conocer-sanabria.com coderstate.com
contentlook.co codq.info
compliance-andrew.xyz
compliance-jane.top
compliance-mark.top
construmac.com.mx
cookielawblog.wordpress.com cookielawblog.wordpress.com
copro.pw cool-wedding.net
coprofam.org countbertwistdisp26.soup.io
copypaste.traderzplanet.in courtshipgift.com
cosmeticswomens-womensports.rhcloud.com csgo4.win
cowblog.fr cubs-jerseys.us
crirussian.ru culmimg.pw
cs-passion.pl culturevie.info
culad.com custom-product-labels.com
cv.wallhade.co customboxes4less.com
d-black.bz cypernhuset.se
daretodonate.co danceuniverse.ru
db.speedup.gdn dawlenie.com
dealighted.com deplim.com
decoratinghomes.ga design-lands.ru
decorationspcs.com dikx.gdn
dieswaene.com dimaka.info
directrev.com discountbarbour.online
dolohen.com disruptingdinnerparties.com
domaineaneblanc.com dogsrun.net
domik-derevne.ru dojki365.online
domination.ml dokfilms.net
doublepimp.com dokumentalkino.net
download-walpaperhd.blogspot.com domainsphoto.com
doxyporno.com dorratex.tn
drillsaw.ru downloads-whatsapp.com
drugs-no-rx.info drhomes.biz
ecookna.com.ua druzhbany.ru
eets.net dustyorate.com
elektrischezigaretten1.blogse.nl easyshoppermac.com
elidelcream.weebly.com edudocs.net
encodable.com elektronischezi.livejournal.com
eropornosex.ru elexies.info
eshop4u.jp elitedollars.com
etenininrade.ga energy-ua.com
europages.com.ru erectile.bid
excitacao.com erotik-kostenlos.net
ezigarettekaufen2.blox.pl essay-edu.biz
fable.in.ua essayassist.com
fdzone.org evening-dating-club.info
fetlifeblog.com everypony.ru
fikasound.tk executehosting.com
filmi-v.online expolicenciaslatam.co
findacheaplawyers.com expresstoplivo.ru
findthe.pet eyessurgery.ru
fix-website-errors.com ez8motelseaworldsandiego.com
flipper.top ezigarettekaufen1.hpage.com
floridahuntingfishingadventures.com ezigaretteshop.webs.com
folowsite.com facialporntube.com
footbalive.org fakehandbags.xyz
forensicpsychiatry.ru familienzahnaerzte.com
forexgb.ru favorcosmetics.com
forminecrafters.ru favornews.com
foxtechfpv.com financeloan.us
freakycheats.com fivedwld.cf
free-fb-traffic.com flashahead.info
freefoto.ca flauntyoursite.com
freelotto.com foodgid.net
freewebs.com forum69.info
freshdz.com foto-weinberger.at
frivgame250.com fotopop.club
front.ru frameimage.org
fuckingawesome.com franklinfire.co
free-fbook-traffic.com
free-stock-illustration.com
free-today.com
fuck-paid-share-buttons.xyz
fullfileaccess.com
funtoonez.com
fxgallery.com fxgallery.com
gavuer.ru g7m.pl
gembird.com gamebackyard.com
genetworx.com games.kolossale.ru
get-free-social-traffic.com gamevalue7.weebly.com
getridofstretchmarks.org gaytube.com
gettpromos.com genericlowlatencyasiodriverhq.aircus.com
girlsfuckdick.com getprismatic.com
gk-atlant.info getyourimage.club
goldenggames.com gfaq.ru
goodhumor24.com gifspics.com
goodly.pro gigapeta.com
gopro-online.info girlsatgames.ru
gotorussia.com golmau.host.sk
gratis-sexkontakte.com goo.ne.jp
greenshop.su greatgrace.ru
gsasearchenginerankerdiscount.com growmyfunds.ca
guardlink.com gsasearchenginerankerhelp.com
guardlink.org gtopstats.com
guiadeserraazul.com
guod.me
hanink.biz.ly
hasshe.com hasshe.com
hccoder.info healbio.ru
hd-film.pl hebr.myddns-flir.com
hdpixion.com hikesearch.net
hdseriale.pl homemade.gq
health-medical-portal.info hotblog.top
hildinghr.se hotshoppymac.com
hledejvshopech.cz houseofrose.com
hobbyhuren24.net hoverboard360.es
hobild.net howlongdoestizanidinestayinyoursystem.blogspot.com
hostcritique.com
hoste.octopis.com
hostingclub.lk
hotenergy.ru
hoverboard360.se
ibmdatamanagement.co ibmdatamanagement.co
ictizanidinehcl4mg.blogspot.com ilovevitaly.com
ideoworld.org imgarit.pw
ikritikimou.gr insomniagamingfestival.com
ilovevitaly.ru iomoio.net
imagine-ex.co irkutsk.online-podarki.com
imediadesk.com ispac.org
importchinacoach-teach.com itrevolution.cf
in-tandem.co jabimgo.pw
indiasourcemart.in jav-fetish.site
infektsii.com jaxcube.info
infowarcraft.ru jimmychoosale.online
insurple.com job.icivil.ir
int.search.mywebsearch.com juliaworld.net
irunfar.com kalandranis.gr
isistaylorporn.info kam-dom.ru
istock-mebel.ru keywordbasket.com
istripper.com keywords-monitoring-success.com
it-max.com.ua kfon.eu
iusstf.org kinky-fetishes.com
ix20.ru kinoflux.net
jackwolfskinoutlet.online kinopolet.net
javrip.net kinozapas.org
jennyfire.ru klikbonus.com
jmat.cn konoplisemena.com
jrcigars.com
karadene.com
kellyonline.xyz
keyword-suggestions.com
kletkimehan.ru
komp-pomosch.ru
konteiner24.com konteiner24.com
koopilka.com koptims.tiu.ru
kotaku.com kosmetyki.tm.pl
kovrenok.ru larger.io
krasnodar.ru layola.biz.tc
kreativperlen.ch leboard.ru
kuzinsp.ru lecbter-relationships.ga
ladsup.com lida-ru.com
landinez.co lincolntheatre.com
landreferat.ru linerdrilling.com
laparfumotec.com linkredirect.biz
lashstudia.ru
laudit.ru
ldrtrack.com
letslowbefast.site
lidoradio.com
likrot.com
limads.men
linkbux.ru
linksharingt.com
lion.bolegapakistan.com
liveu.infoteka.hu
locksmith.jp locksmith.jp
loftdigital.eu lombardfinder.ru
lomb.co lombia.com
lomza.info losangeles-ads.com
lovi-moment.com.ua mackeeper-center.club
luchshie-topcasino.ru macotool.com
lutherstable.org magento-crew.net
lyrster.com mall.uk
macdamaged.tech manifestation.betteroffers.review
machicon-akihabara.info marktforschung-stuttgart.com
machicon-ueno.info masserect.com
magicdiet.gq matpre.top
malls.com max-eclat.men
maximpartnerspr.com mcnamaratech.com
mediaoffers.click
medicineseasybuy.com
medispainstitute.com.au medispainstitute.com.au
mega-polis.biz.ua mega-bony-2017.pl
megahdporno.net megagrabber.ru
memberty.com
mensandals.xyz
mesto-x.com
metaxalonevstizanidine.blogspot.com
mex-annushka.ru
mezaruk.info
michaelkorssaleoutletonline.net
mikrobiologies.ru
mil-stak.com mil-stak.com
miniads.ca miniature.io
mir-business-24.ru mixtapetorrent.com
mirtorrent.net mmofreegames.online
mister-shop.com mmstat.com
mnimmigrantrights.net mockupui.com
moi-glazki.ru modnie-futbolki.net
moesexy.com
mojpregled.com mojpregled.com
monclercoats.xyz monclerboots.xyz
money-every-day.com monclercheap.xyz
money-for-placing-articles.com monsterdivx.com
moneymaster.ru more-letom.ru
moneyteam24.com morefastermac.trade
morpicert.pw mossmesi.com
most.gov.iq
mouselink.co mouselink.co
mrcsa.com.au moz.com
mydearest.co mp3ritm.top
nardulan.com mrmoneymustache.com
narkologiya-peterburg.ru mundoaberrante.com
nastroyke.net music.utrolive.ru
naughtyconnect.com musicktab.com
neks.info musicvidz.ru
neodownload.webcam muznachas-service.ru
my-aladin.com
myblogregistercm.tk
myperiod.club
mystats.xyz
naj-filmy24.pl
najaden.no
narkologiya-penza.ru
naturalshair.site
nbsproject.ru
neobux-bg.info
netanalytics.xyz
netcheckcdn.xyz
netoil.no netoil.no
nfljerseysforsalewholesaler.com new-apps.ru
nikkiewart.ru newstaffadsshop.club
nl.netlog.com nightvision746.weebly.com
novosibirsk.xrus.org nordstar.pro
novosti-hi-tech.ru novatech.vn
obiavo.uz npoet.ru
online-hd.pl nrjmobile.fr
oranga.host.sk oakridgemo.com
paighambot.com ochistka-stokov.ru
oecnhs.info
offergroup.info
okout.ru
oksrv.com
onlywoman.org
ontargetseo.us
ooo-olni.ru
ordernorxx.com
oren-cats.ru
pandarastore.top pandarastore.top
panicatack.com paperwritingservice17.blogspot.ru
partner-cdn.men partner-pop.men
partner-high.men pc-services.ru
partnersafe.men pearlisland.ru
pastaleads.com photosaga.info
pc4download.co picsforkeywordsuggestion.com
pdns.cz pizda.lol
pechikamini.ru pkr1hand.com
pekori.to pod-muzyku.club
pensplan4u.com pokemon-go-play.online
pestomou.info pokemongooo.ml
petrovka-online.com pontiacsolstice.info
pharmacyincity.com popup.matchmaker.com
piski.top pornhive.org
pistonclasico.com porno-transsexuals.ru
pizza-tycoon.com pornodojd.ru
pl-top.pl pornolook.net
platezhka.net pornozhara.com
pogruztehnik.ru
pooleroadmedicalcentre.co.uk
popads.net
popunder.ru
pornhubforum.tk
pornohd1080.online
pornoslive.net
pornzone.tv
portal-eu.ru
portside.xyz
posible.net
pourvous.info pourvous.info
pozdravleniya-c.ru
predominant-invent.tk predominant-invent.tk
privacylocationforloc.com privetsochi.ru
prizefestival.mobi professionaldieselcare.com
proekt-mos.ru proposal-engine.com
projefrio.com.br prosmibank.ru
promodj.com prostitutki-kharkova.org
ptr.ruvds.com pussyfleet.com
puteshestvennik.com puteshestvennik.com
pwwysydh.com ranksonic.org
py100.ru redhotfreebies.co.uk
quickchange.cc reelheroes.net
recipedays.ru registratciya-v-moskve.ru
regdefense.com relatodelpresente.com.ar
reklama1.ru remmling.de
responsive-test.net remont-ustanovka-tehniki.ru
reward-survey.net renecaovilla.online
rightenergysolutions.com.au retailwith.com
rfd-split.hr
riralmolamsaca.tk riralmolamsaca.tk
robertefuller.com rocis.site
rogervivierforsale.com roznica.com.ua
rome2rio.com rrutw.com
royalvegascasino.com sabizonline.com
ruscopybook.com sad-torg.com.ua
sajatvelemeny.com sammlungfotos.online
saltspray.ru sashagreyblog.ga
savetubevideo.com sayyoethe.blogspot.co.za
scanner-fred.top sbornik-zakonov.ru
scanner-irvin.top score-ads.men
scanner-jack.top
school-diplomat.ru
sdrescher.net
search-error.com
search.1and1.com search.1and1.com
sendearnings.com searchlock.com
semalt.com
seo-2-0.com seo-2-0.com
seoanalyses.com seofirmreviewsus.info
seokicks.de seonetwizard.com
serptehnika.ru serptehnika.ru
servisural.ru sexcamamateurchat.com
sex-znakomstva.online sexfreepornoxxx.com
sexflirtbook.com sexpornotales.com
sexobzor.info sexuria.net
sextracker.de shell-pmr.ru
shareyards.com shodanhq.com
shoppingmiracles.co.uk shopsellcardsdumps.com
si-unique.com silvercash.com
siha.de site-speed-check.site
site.ru siteonomy.com
site3.free-share-buttons.com skachat-besplatno-obrazcy.ru
slomm.ru skutecznetabletkinaporostwlosow.pl
slonechka.ru skypasss.com
slowmacfaster.trade slavia.info
slutloadlive.com sloopyjoes.com
snip.to smartpet.ru
smichovbike.cz
snip.tw
snjatie-geroinovoy-lomki.ru
social-button.xyz social-button.xyz
soft-terminal.ru social-buttons.xyz
social-fun.ru
socialbutton.xyz
socialtrade.biz
softonicads.com softonicads.com
spabali.org solution4u.com
spravka130.ru sonata-arctica.wz.cz
spywarenuker.com soserfis.com
sr-rekneskap.no spb-plitka.ru
srecorder.com spb.afora.ru
sstroy44.ru ssconstruction.co
startwp.org steamoff.net
statistici.ro stroiminsk.org
studiofaca.com styro.ru
sugarkun.com supermama.top
suggest-keywords.com supervesti.ru
superstats.com survival.betteroffers.review
swinginwithme.ru swinger-mobil.net
tabletkinaodchudzanie.com.pl swsociety.se
tattooreligion.ru tacbelarus.ru
taximytishi.ru taraz.xkaz.org
teenforporn.com tdsing.ru
tengohydar.tk tedxrj.com
testbotprocessor44.com tehngr.ru
tetracsaudi.com telegramdownload10.com
template-kid.com
tfxiq.com
tgtclick.com tgtclick.com
theautoprofit.ml thebluffs.com
thedownloadfreeonlinegames.blogspot.com thefarmergame.com
theheroes.ru thin.me.pn
thewebsitetemplate.info tizanidine4mgstreetvalue.blogspot.com
thiegs.reco.ws tizanidineinteractions.blogspot.com
tizanidinehcl4mgtablet.blogspot.com toonfamilies.net
tizanidinesleepaid.blogspot.com
top-deal.com.pl
top1-seo-service.com
top10-online-games.com
topanasex.com topanasex.com
topmira.com topdownloads.ru
totu.us toy-shop.top
tourcroatia.co.uk traffic2money.com
toys.erolove.in traffixer.com
traderzplanet.in triplepanda.xyz
trafficstars.com trustaffs.com
traffictrade.life turbodsp.com
trucri.me turizm.bz
tubeoffline.com turvgori.ru
tuckermktg.com
tversvet.ru
tvorozhnaja-zapekanka-recept.ru tvorozhnaja-zapekanka-recept.ru
txxx.com ua-company.ru
u555u.info uasb.ru
ugogo.info undergroundcityphoto.com
umnovocaminho.com
univerfiles.com univerfiles.com
unrealcommander.biz uptime-alpha.net
uptime.com uptime-as.net
uptimebot.net urdoot.win
url-extractor.xyz url2image.com
uroki.net urzedowski.eu
usdx.us usbggettwku.ga
uvozdeckych.info uvozdeckych.info
uytmaster.ru vadimkravtcov.ru
uzungil.com valsalud.com
v-doc.co vapsy.com
vandrie-ict.nl vereo.eu
vapomnoncri.tk
vashsvet.com
velobikestock.com
video-chat.love video-chat.love
vip2ch.com videochat.tv.br
vita.com.hr vitalads.net
vladimir.xrus.org vkonche.com
vozbujdenie.com
vrotike.ru
vtc.pw
w3javascript.com w3javascript.com
w7s.ru wallpapersdesk.info
wallpaperstock.net
wanker.us wanker.us
we-are-gamers.com waysbetter.cn
webads.co.nz wdfdocando.com
webix.biz
webmasterhome.cn webmasterhome.cn
website-stealer.nufaq.com website-speed-up.top
weburok.com websites-reviews.com
wechatdownload10.com websitevaluebot.com
wesharepics.com weburlopener.com
whatistizanidinehclusedfor.blogspot.com whatistizanidinehclusedfor.blogspot.com
wholesalejerseychinaoutlet.com wholesalejerseychinaoutlet.com
whos.amung.us wnhjavlhezp.gq
whosonmyserver.com worldhistory.biz
wildcattube.com wovis.site
windowssearch-exp.com writersgroup580.web.fc2.com
wingsoffury2.com ws.ampower.me
worldoffiles.ru www.ehscloud.cn
wosik-dach.service-for-web.de www.event-tracking.com
www.pinnacle-bets.com www.jbetting.com
x-musics.com xcc24.pl
xchat26.myfreecams.com xn----7sbho2agebbhlivy.xn--p1ai
xfire.com xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
xn----btbdvdh4aafrfciljm6k.xn--p1ai xn--b1adccaf1bzj.xn--p1ai
xn----ctbbcjd3dbsehgi.xn--p1ai xn--c1acygb.xn--p1ai
xn--80ak6aa92e.com
xn--d1abj0abs9d.in.ua xn--d1abj0abs9d.in.ua
xolodremont.ru xn--lifehacer-1rb.com
xsfetish.org
xtubeporno.net
xvideosbay.com
xxxtube69.com xxxtube69.com
yatrk.xyz yebocasino.co.za
ynymnwbm.bloger.index.hr yellowstonesafaritours.com
yourporngay.com yoluxuryevents.com
yourserverisdown.com yopoint.in
yugk.net youporn-forum.ga
zaloro.com youtubologia.it
zastenchivosti.net youtuhe.com
zero1.it zahvat.ru
zhuravlev.info zarepta.com
znakomstvaonlain.ru zeroredirect5.com
zreloeporno.tv zeroredirect9.com
zoompegs.com
zscalerone.net
zvuker.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
13tabs.com 1xbet4you.com
1688.com 20pascals.nl
1adult.com 51.la
28n2gl3wfyb0.ru 51unlim.ru
2pxg8bcf.top 5forex.ru
3xforum.ro a96527gi.beget.tech
4istoshop.com aa81bf391151884adfa3dd677e41f94be1.com
5-steps-to-start-business.com adconscious.com
5kstore.com addray.pro
991mostfm.co.id adidas.frwebs.fr
aa8b68101d388c446389283820863176e7.com adsland.men
aadroid.net advokat-grodno.by
abcdefh.xyz aktobe.xkaz.org
adbetclickin.pink allbizne.co.ua
adcash.com allkrim.com
akita.kz altamayoztourism.com
aksonural.ru an-donut.com
albamargroup.com animal-drawings.com
alborzan.com aquarium-pro.ru
alienwheel.es arcadeplayhouse.com
analnoeporno.tv atofilms.com
anonymizeme.pro aviav.co
appiq.mobi avon-severozapad.ru
apple.com-cleaner.systems azazaporn.com
arius.tech babieca.com
arquivo.pt babyfactory.fr
asrvrep-a.akamaihd.net bagcionderlab.com
audiofree.ru battlecarnival.su
aussie-prizes.com beget.tech
balook.com best-seo-solution.com
blagovest-med.ru bet-prognoz.com
blogrankers.com bigcareer.info
boanonihaca.tk bioca.org
btc4u.ru blogporn.in
buyhoverboard.com bracketsmackdown.com
c-gainsbourg.com braintobe.win
canrioloadm.gq bukleteg.xyz
cartierlovebracelet.xyz burn-fat.ga
cascadelink.org calcularpagerank.com
casinorewards.info cartierreplica.xyz
cdncash.com cdncash.org
celejihad.info cdnnetwok.xyz
cialischstgerts.com cellfun.mobi
citetick.com chelyabinsk.xrus.org
coinsspb.com cherubinimobili.it
compliance-jane.top chocolateslim-u-srbiji.com
contentsexpress.com chuckguilford.com
cornerstone-countertops.com cialisndbrx.com
counterbot.com cialisovercounteratwalmartusa.com
creditnation.ru cinemaenergy-hd.ru
custodieva.ru civilwartheater.com
customchocolate.business-for-home.com coaching-netz.info
cyprusbuyproperties.com codysbbq.com
d-black.bz conocer-sanabria.com
delayreferat.ru cookielawblog.wordpress.com
demian.kz coolyarddecorations.com
deutschehobbyhuren.net coslab.club
dipstar.org credit-online.ws
directivepub.com crystalslot.com
distver.ru cubs-jerseys.us
dkvorota.ru customcollegeessays.net
docsportal.net cyclobenzaprinevstizanidine.blogspot.com
download-walpaperhd.blogspot.com daretodonate.co
downloadmefiranaratb1972.xpg.com.br defenderxtactical.com
drupa.com devochki.top
earncash.com.ua dimfour.com
easyshoppermac.com domik-derevne.ru
el-nation.com domoysshop.ru
eredijovon.com elitepcgames.com
estibot.com emailaccountlogin.co
etotupo.ru emergencyneeds.org
euronis-free.com erotische-geschichten-xxl.com
europages.com.ru escortplius.com
esnm.ru
executehosting.com
extads.net extads.net
eywords-monitoring-your-success.com
ezigarettekaufen2.yolasite.com ezigarettekaufen2.yolasite.com
factorynightclub.com f-online.de
family1st.ca
femdom.twiclub.in
fenoyl.batcave.net fenoyl.batcave.net
fettefrauen.net findpik.com
filesvine.com fix-website-errors.com
footbalive.org fm-upgrade.ru
forex-procto.ru frameimage.org
franch.info franklinfire.co
free-fb-traffic.com free-fb-traffic.com
free-video-tool.com freefoto.ca
freevpn.space freewlan.info
fuckingawesome.com fungamelands.com
gay-site.store fxgallery.com
genetworx.com galaxy-family.ru
gg.zzyjxs.com galeon.com
gkvector.ru game-mmorpg.net
gne8.com game300.ru
goodhumor24.com gateway.zscalertwo.net
grand-chlen.ru genericlowlatencyasiodriverhq.aircus.com
growshop.es girlsfuckdick.com
gymi.name glopages.ru
hacktougroup.ru googlsucks.com
hdimagelib.com gotorussia.com
hdwalls.xyz greatzip.com
hidefiles.org hamacapty.com
high-speed1.net hamilton.ca
hitmuzik.ru hard-porn.mobi
hotdl.in himazin.info
igtools.club hitsbox.info
infazavr.ru hothot.ru
irunfar.com hotloans.ru
istizanidinelikexanax.blogspot.com https-legalrc.biz
imgata.com
indetiske.ya.ru
insider.pro
investsuccess.org
invitefashion.com
ipornox.xxx
isotoner.com
itronics.ca itronics.ca
ivanstroi.ru izhevsk.xrus.org
iyasimasennka.com izismile.com
jackpotchances.com jav-way.site
javidol.site javcoast.com
jerseywholesalechinabiz.com kareliatobacco.ru
journeydownthescale.info kashubadesign.ru
katushka.net katjimej.blog.fc2.com
keywords-monitoring-success.com kaz.kz
kichenaid.ru keywesthideaways.co
kino-rating.ru keywordspay.com
kopihijau.info kmd-pto.ru
kuzinsp.ru koptims.tiu.ru
laapp.com kutikomi.net
labafydjxa.su landinez.co
larutti.ru lankarns.com
ldrtrack.com laptop-4-less.com
lesbian.xyz linkrr.com
letsart.ru lovelycraftyhome.com
letto.by lutherstable.org
lida-ru.com m4ever.net
lkbennettoutlet.online maju.bluesalt.co
lunamedia.co maladot.com
lyrics.home-task.com managerpak204.weebly.com
lyrster.com medanestesia.ru
m292.info medicovi.com
madot.onlinewebshop.net meds-online24.com
magda-gadalka.ru mncrftpcs.com
mamasuper.prom.ua moviezbonkerssk.cf
masserect.com
matb3aa.com
med-zdorovie.com.ua
megahdporno.net
mesto-x.com
microsearch.ru
mielec.pl
miniature.io
mitrasound.ru
mokrayakiska.com
moscow-region.ru
mp3ringtone.info
mrsdalloways.com mrsdalloways.com
muzaporn.com mykings.pw
nakozhe.com mymobilemoneypages.com
namenectar.com nakrutka.cc
net-profits.xyz natali-forex.com
nofreezingmac.click netvouz.com
news-readers.ru
newtechspb.ru
normalegal.ru
norththeface.store
notebook-pro.ru notebook-pro.ru
nuup.info nrv.co.za
nw-servis.ru nudejapan.net
obiavo.kz obiavo.net
ok.ru olympescort.com
onlinebay.ru parajumpersstore.online
oren-cats.ru perfectplanned.com
orion-code-access.net pestomou.info
oxotl.com petsblogroll.com
pandarastore.top phobia.us
partner-pop.men photokitchendesign.com
pio.polytopesexempt.com photosaga.info
pk-services.ru pillscheap24h.com
plusnetwork.com pizda.lol
potolokelekor.ru pizdeishn.com
printie.com pizdopletka.club
prokotov.com pizza-imperia.com
promgirldresses.xyz pl-top.pl
pst2017.onlinewebshop.net plastweb.ru
questionmarque.ch playlott.com
quickloanbank.com porno-dojki.net
quizzitch.net pricheskaonline.ru
pricheski-video.com
privatbank46.ru
psoriasis-file.trade
rapidgator-porn.ga rapidgator-porn.ga
sabaapress.com realitykings.com
sakhboard.ru recinziireale.com
sarahmilne.top relayblog.com
scanner-alexa.top runofilms.ru
scanner-andrew.top rusprostitute.com
scanner-margo.top rybalka-opt.ru
scanner-walt.top sbricur.com
sdrescher.net schlampen-treffen.com
search.pch.com scmor.ilxc.cc
seminarykansas904.ml searchaddis.com
sexxdate.net seorankinglinks.xyz
sexytrend.ru shariki-zuma-lines.ru
shareyards.com shopcheermakeup.info
shodanhq.com skylta.com
sibecoprom.ru social-fun.ru
sideeffectsoftizanidine.blogspot.com soda.media
silvercash.com solution4u.com
sinestesia.host.sk spacash.com
sitevaluation.com staff.prairiesouth.ca
sixcooler.de stillmiracle.com
smart-balancewheel.com stroilka.info
smsactivator.ru strongsignal-a.akamaihd.net
sneakyboy.com studworks.org
soundfrost.org supermesta.ru
sport-video-obzor.ru svarbit.com
sport7777.net tampabaywatch.org
sribno.net taqywu51.soup.io
stats-collector.org td-l-market.ru
store-rx.com tecnoteakviareggio.it
swinon.site thegioixekhach.com
tackletarts.co tizanidine4mgstreetvalue.blogspot.com
teenfuck.tv tizanidineonline.blogspot.com
telemetryverification.net toondinsey.com
tfxiq.com top-l2.com
thewebsitetemplate.info topappspro.com
titslove.yopoint.in topmira.com
tizanidineingredients.blogspot.com topsiteminecraft.com
tizanidinesideeffectsweightloss.blogspot.com track2.shop
tootoo.to unionmarkt.de
top-deal.com.pl unmaroll.ya.ru
trackzapper.com v24s.net
trafficjunky.com vacances-voyages.info
traffique.net valsalud.com
truebeauty.cc videosbox.ru
tsstcorpcddvdwshbbdriverfb.aircus.com viktoria-center.ru
ttsq.fr wallpaperstock.net
turbodsp.com wardreapptokone.tk
twelvevisionspartyofcolorado.com watracker.net
ugogo.info website-stealer.nufaq.com
umaseh.com wedding0venues.tk
usacasino.com wildcattube.com
vizitki.net win-spy.com
vriel.batcave.net
web.cvut.cz
wiosenny-bon-1500.pl wiosenny-bon-1500.pl
xingzi-vision.com withstandingheartwarming.com
xn--b1adccaf1bzj.xn--p1ai wnoz.de
xn--d1aifoe0a9a.top womensterritory.ru
xnxx-n.com www.souvenirua.com
xxlargepop.com wwwadultcheck.com
yaminecraft.ru wzgyyq.com
yebocasino.com xchangetrak.com
yoshkarola.zrus.org xep.info
youjizz.com xn----7sbifcamovvfggw9d.xn--p1ai
zakonobosago.ru xn----btbdvdh4aafrfciljm6k.xn--p1ai
zdesoboi.com xxxhdvideo.site
zed21.net xxxtubesafari.com
zeroredirect5.com yhit.press
zionstar.net yopoint.in
znaniyapolza.ru youdao.com
zscaler.net yunque.pluto.ro
zverokruh-shop.cz

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.108 0; 10.1.0.195 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2635 ### Version: V4.2021.10.2636
### Updated: Wed Oct 6 22:01:13 UTC 2021 ### Updated: Thu Oct 7 10:03:18 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.10.2635 ### Version: V4.2021.10.2636
### Updated: Wed Oct 6 22:01:13 UTC 2021 ### Updated: Thu Oct 7 10:03:18 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##