vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.01.2840 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-01-09 06:39:46 +00:00
parent ca39ed6076
commit 764771dd17
46 changed files with 74127 additions and 74202 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2839 #### Version: V4.2022.01.2840
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.01.2839 #### Version: V4.2022.01.2840
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -13,7 +13,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.01.2839 #### Version: V4.2022.01.2840
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9130
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.01.2839 #### Version: V4.2022.01.2840
#### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7087](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [631](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

9139
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9139
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

9139
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

9139
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

9139
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

9139
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

9139
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2839 2840

9139
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

9139
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.62.58.134 0; 20.112.84.204 0;

9139
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

9139
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

9139
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

9139
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.62.58.134 0; 20.112.84.204 0;

9139
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
168.62.58.134 0; 20.112.84.204 0;

9139
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 1; 10.1.0.69 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Copier CSHttp
DomainSigmaCrawler CherryPicker
Evil Dispatch
LieBaoFast Larbin
RealDownload Scrapy
SiteCheckerBotCrawler Sitebeam
SiteSucker Sucuri
SmartDownload WEBDAV
Website Quester Web Fuck
spyfu WebReaper

308
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 80legs
360Spider
404checker
ADmantX
AIBOT
Abonti Abonti
Aboundex
Aboundexbot Aboundexbot
Acunetix AiHitBot
AfD-Verbotsverfahren Aipbot
Alexibot Alexibot
AllSubmitter AlphaBot
Aspiegel Anarchy99
Ankit
Anthill
Asterias
AwarioRssBot
AwarioSmartBot
BBBike BBBike
BDCbot
BLEXBot
BackDoorBot BackDoorBot
BackStreet BackWeb
Backlink-Ceck BacklinkCrawler
Badass Badass
Barkrowler Blackboard
Battleztar Bazinga
Black Hole
Blow
BlowFish BlowFish
Bolt Boardreader
Brandprotect Brandprotect
Brandwatch Bullseye
Buck BunnySlippers
BuzzSumo BuzzSumo
CODE87 CATExplorador
CSHttp Calculon
CheeseBot Citoid
Claritybot
Cocolyzebot Cocolyzebot
Copier Cogentbot
Copyscape Cosmos
Craftbot Craftbot
CrazyWebCrawler Crawling at Home Project
DBLBot Crescent
Curious
Custo
CyotekWebCopy
DSearch
DTS Agent DTS Agent
Demon Disco
Deusu Dispatch
Devil DnBCrawler-Analytics
Digincore DnyzBot
DigitalPebble DomainAppender
Discoverybot
DittoSpyder
DomainCrawler DomainCrawler
DomainSigmaCrawler DomainSigmaCrawler
Domains Project DomainStatsBot
Drip
ECCP/1.0 ECCP/1.0
EMail Siphon
EasyDL
Ebingbong Ebingbong
EirGrabber Exabot
EroCrawler Ezooms
Evil
Express WebPictures
ExtLinksBot
EyeNetIE
FDM FDM
Freeuploader FHscan
Firefox/7.0
FlashGet
Foobot
Fuzz
FyberSpider FyberSpider
Fyrebot
GT::WWW GT::WWW
Genieo Genieo
Go-Ahead-Got-It GetRight
GoZilla GetWeb
Gotit Gotit
GrabNet
Grabber
Grafula Grafula
GrapeFX GrapeFX
HTMLparser GridBot
HTTP::Lite HTTrack
HaosouSpider Havij
Heritrix Heritrix
HybridBot Hloader
IDBTE4M IDBot
IRLbot IlseBot
Indy Library Indy Library
InfoNaviRobot InfoTekies
Iria InterGET
Internet Ninja
InternetSeer
IstellaBot IstellaBot
JOC Web Spider
Joomla
Jorgee Jorgee
Jyxobot Kenjin Spider
Keyword Density
LNSpiderguy
Lanshanbot Lanshanbot
Larbin
LeechGet
LexiBot LexiBot
LibWeb Lftp
LinkWalker Libwhisker
LieBaoFast
Lightspeedsystems
LinksManager
LinqiaRSSBot LinqiaRSSBot
Lipperhey
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIDown tool
MJ12bot MSIECrawler
MQQBrowser
MTRobot MTRobot
Mag-Net Magnet
Majestic-SEO Mail.RU_Bot
Majestic12
MarkWatch MarkWatch
Mass Downloader
Mata Hari
Mb2345Browser Mb2345Browser
MegaIndex.ru MeanPath Bot
Meanpathbot
Mediatoolkitbot
Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Microsoft URL Control Mojeek
Minefield Morfeus Fucking Scanner
Mojolicious Mozlila
Mr.4x3 Mr.4x3
NearSite Msrabot
Nessus Nessus
Net Vampire NetSpider
NetAnts NetZIP
Netcraft
Netvibes Netvibes
NextGenSearchBot NextGenSearchBot
Niki-bot Nibbler
Nikto NimbleCrawler
Nimbostratus
Ninja Ninja
Not
Nuclei
Nutch
Octopus Octopus
Offline Explorer
Offline Navigator Offline Navigator
OnCrawl OrangeBot
OpenLinkProfiler
OpenVAS
Openfind
PHPCrawl PHPCrawl
Page Analyzer PageThing.com
PageGrabber Panscient
PageScorer
Papa Foto Papa Foto
Pavuk
PeoplePal PeoplePal
Pi-Monster Pi-Monster
Picscout Pimonster
Picsearch PleaseCrawl
Pixray Pockey
ProPowerBot
Probethenet Probethenet
PyCurl Pu_iN
QueryN Metasearch QueryN Metasearch
RankActive RankActive
RankActiveLinkBot
RankFlex RankFlex
RankingBot RankingBot
RankurBot
RealDownload
Recorder Recorder
SEOkicks RedesScrapy
SEOkicks-Robot Rogerbot
SEOstats SBIder
SMTBot SMTBot
SalesIntelligent ScanAlert
Scanbot
Scrapy Scrapy
Screaming Searchestate
ScreenerBot SemanticJuice
SemrushBot
SeobilityBot
Siphon Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker
Sitebeam
Siteimprove
Sitevigil Sitevigil
SmartDownload SmartDownload
Snake SocialRankIOBot
Snapbot Sogou web spider
Sociscraper SpankBot
Sosospider
Sottopop
Spbot Spbot
Spinn3r SputnikBot
Stripper Sqlmap
Surfbot Sucker
SuperHTTP
SurveyBot
Suzuran Suzuran
Teleport T0PHackTeam
Telesphoreo Telesoft
Telesphorep Telesphorep
The Intraformant Thumbor
TheNomad Titan
Toata
Toweyabot Toweyabot
Tracemyfile Tracemyfile
Trendiction Trendiction
Trendictionbot
True_Robot True_Robot
Turingos
TurnitinBot TurnitinBot
TwengaBot TwengaBot
Twice Twice
URLy Warning Upflow
URLy.Warning
VB Project
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler VidibleScraper
Voltron VoidEYE
WEBDAV WEBDAV
WWW-Collector-E WWW-Collector-E
WWW::Mechanize WWWOFFLE
Wallpapers WallpapersHD
Web Auto Web Auto
Web Collage Web Collage
Web Fetch
Web Fuck
Web Sauger Web Sauger
WebAuto
WebBandit WebBandit
WebCollage WebGo IS
WebImageCollector WebImageCollector
WebLeacher
WebPix
WebReaper
WebSauger WebSauger
WebWhacker
WebZIP WebZIP
Webshag WinHTTrack
WebsiteExtractor
WebsiteQuester
Webster
Whack
Whatweb
WiseGuys Robot
Woobot Woobot
Wotbox
Xaldon_WebSpider Xaldon_WebSpider
Xenu
YoudaoBot
Zade Zade
Zauba Zauba
Zermelo
Zeus Zeus
ZmEu Zitebot
ZoominfoBot ZoominfoBot
ZumBot
ZyBorg
arquivo.pt
autoemailspider autoemailspider
cah.io.community
clark-crawler clark-crawler
crawl.sogou.com cognitiveseo
crawler.feedback crawler4j
dataforseo.com
demandbase-bot
eCatch
evc-batch
gopher
heritrix heritrix
isitwp.com internetVista monitor
lwp-request ips-agent
magpie-crawler netEstate NE Crawler
muhstik-scan oBot
page scorer page scorer
plumanalytics pcBrowser
probe-image-size polaris version
scan.lol scalaj-http
seobility
seocompany.store seocompany.store
serpstatbot
sitechecker.pro sitechecker.pro
sogouspider
sysscan sysscan
trendiction.com trendiction.com
ubermetrics-technologies.com trendiction.de
voyagerx.com webgains-bot
webprosbot webmeup-crawler
zauba.io x09Mozilla
zgrab zgrab

308
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,200 +1,194 @@
404checker 404enemy
ADmantX 80legs
ASPSeek AIBOT
Abonti
Aboundex Aboundex
Aboundexbot Aboundexbot
Aipbot Acunetix
AfD-Verbotsverfahren
Alexibot Alexibot
Anarchy Alligator
Anarchy99 Anarchie
Aspiegel AwarioRssBot
Attach BBBike
BDCbot BDFetch
BLEXBot BackDoorBot
Backlink-Ceck Backlink-Ceck
Barkrowler BacklinkCrawler
Bigfoot Badass
BatchFTP
Black Hole Black Hole
BlackWidow
Blackboard Blackboard
Bolt Bolt
Brandprotect BotALot
Buddy Brandwatch
BuiltWith Buck
CCBot CATExplorador
CODE87 CSHttp
CazoodleBot Calculon
CensysInspect
CheTeam CheTeam
CherryPicker
ChinaClaw
Citoid
Claritybot Claritybot
Cogentbot Cloud mapping
Copier Cocolyzebot
Copyscape Collector
CopyRightCheck
Cosmos Cosmos
Craftbot Crawling at Home Project
CrunchBot Curious
DBLBot CyotekWebCopy
DTS Agent DTS Agent
Demon
Deusu Deusu
Devil Devil
DigitalPebble Discoverybot
Dirbuster
Disco
Dispatch
DittoSpyder DittoSpyder
DomCopBot
DomainAppender DomainAppender
Dragonfly Dragonfly
Drip Drip
EMail Siphon
EMail Wolf EMail Wolf
Evil EasyDL
ExtLinksBot Exabot
ExtractorPro
EyeNetIE
Ezooms Ezooms
FHscan FemtosearchBot
Firefox/7.0 Fimap
Fyrebot FlashGet
Genieo Flunky
Go-Ahead-Got-It Fuzz
G-i-g-a-b-o-t
GT::WWW
GalaxyBot
GoZilla GoZilla
Gotit Gotit
Grabber GrabNet
GrapeFX Grafula
GrapeshotCrawler HMView
HEADMasterSEO
HTMLparser
HTTP::Lite HTTP::Lite
HTTrack Haansoft
HaosouSpider
Harvest Harvest
Havij
Heritrix
Hloader
Humanlinks Humanlinks
HybridBot
IRLbot IRLbot
IlseBot Image Fetch
Image Sucker Image Sucker
IndeedBot
InfoNaviRobot InfoNaviRobot
Intelliseek Intelliseek
InterGET Internet Ninja
InternetSeer Iria
IstellaBot
JOC Web Spider JOC Web Spider
Jbrofuzz JamesBOT
JennyBot
JetCar
Jetty
Joomla Joomla
Kenjin Spider Jyxobot
Keyword Density Keyword Density
Kinza Kozmosbot
LNSpiderguy
LWP::Simple LWP::Simple
Larbin
LeechFTP
LexiBot
Lftp
LibWeb LibWeb
Libwhisker Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
Likse
Linkdexbot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey Lipperhey
Litemage_walker
Lmspider Lmspider
MFC_Tear_Sample
MIDown tool
MIIxpc
MJ12bot
MQQBrowser MQQBrowser
MSFrontPage
MSIECrawler
Mag-Net
Magnet Magnet
Mail.RU_Bot Mail.RU_Bot
MarkMonitor Majestic SEO
Mass Downloader Majestic12
MauiBot MeanPath Bot
MegaIndex.ru Meanpathbot
MicroMessenger Metauri
Microsoft URL Control Microsoft Data Access
Minefield Minefield
Mister PiX Mister PiX
Moblie Safari
Morfeus Fucking Scanner
Mozlila Mozlila
Mr.4x3 Msrabot
Musobot
Nameprotect Nameprotect
Navroad
NetAnts
NetSpider NetSpider
NetZIP NetZIP
Netcraft Netcraft
Nettrack Netvibes
NextGenSearchBot Ninja
Niki-bot
NimbleCrawler
Nimbostratus
Not Not
Nuclei
Octopus Octopus
Offline Explorer Offline Explorer
Offline Navigator
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openvas
OrangeBot
OutclicksBot
OutfoxBot OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageScorer
Pandalytics Pandalytics
Panscient
PeoplePal PeoplePal
Picsearch Pi-Monster
PictureFinder Piepmatz
Pimonster Pimonster
Pixray
PleaseCrawl
Pockey
ProWebWalker ProWebWalker
Pu_iN PxBroker
QueryN Metasearch PyCurl
RankActive Quick-Crawler
RankFlex
RankingBot RankingBot
Re-re ReGet
SBIder RealDownload
SEOlyticsCrawler SEOkicks-Robot
SISTRIX
SMTBot SMTBot
Scanbot ScanAlert
Screaming ScoutJet
ScreenerBot
ScrepyBot
Searchestate
Semrush
SemrushBot SemrushBot
SentiBot
SeoSiteCheckup
SeobilityBot
Seomoz
Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
Sitebeam SiteSnagger
Snake Siteimprove
Sitevigil
SmartDownload
Snoopy Snoopy
SocialRankIOBot
Sottopop Sottopop
Spammen SpaceBison
SpankBot Spanner
Spbot Spinn3r
SputnikBot Stripper
Sqlmap Surfbot
Sqworm
SuperHTTP
Swiftbot Swiftbot
Szukacz
T0PHackTeam T0PHackTeam
Teleport Teleport
Telesoft TeleportPro
Titan Telesphorep
True_Robot Toweyabot
Turingos Turingos
Turnitin TurnitinBot
TwengaBot TwengaBot
URLy.Warning Typhoeus
URLy Warning
V-BOT V-BOT
VCI VCI
Vacuum Vagabondo
VelenPublicWebCrawler VeriCiteCrawler
Voil VidibleScraper
Voltron WASALive-Bot
WBSearchBot
WEBDAV
WISENutbot WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize WWW-Mechanize
@ -202,49 +196,55 @@ WWW::Mechanize
WWWOFFLE WWWOFFLE
Wallpapers Wallpapers
Wallpapers/3.0 Wallpapers/3.0
WeSEE
Web Enhancer Web Enhancer
Web Sucker Web Fuck
WebBandit
WebCollage
WebLeacher
Web Pix Web Pix
WebReaper Web Sauger
WebStripper WebAuto
WebBandit
WebEnhancer
WebFuck
WebImageCollector
WebSauger
WebSucker WebSucker
WebWhacker WebmasterWorldForumBot
WebZIP
Webshag
Website Quester
WebsiteExtractor WebsiteExtractor
WebsiteQuester
Webster
Whack Whack
Who.is Bot Who.is Bot
WinHTTrack Widow
Woobot Woobot
Xaldon WebSpider
Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zade
Zauba
Zeus Zeus
Zitebot Zitebot
ZmEu
ZoomBot ZoomBot
ZoominfoBot
ZumBot ZumBot
adscanner adscanner
arquivo-web-crawler cah.io.community
arquivo.pt
coccocbot-web coccocbot-web
com.plumanalytics cognitiveseo
crawler.feedback crawler.feedback
crawler4j eCatch
domainsproject.org internetVista monitor
lwp-trivial ips-agent
isitwp.com
magpie-crawler
meanpathbot
muhstik-scan muhstik-scan
netEstate NE Crawler netEstate NE Crawler
serpstatbot oBot
sexsearcher pcBrowser
sogouspider polaris version
sp_auditbot probe-image-size
spyfu ripz
sysscan seoscanners
voyagerx.com seostar
webmeup-crawler siteripz
webprosbot tAkeOut
x09Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
cobaltpro.ru 1millionusd.xyz
ihtec2019.org cartierbracelet.xyz
macrotek.ru gigixo.com
manifestation.betteroffers.review i-service.kz
o-o-6-o-o.com konoplisemena.com
ons-add.men m0r0zk0-krava.ru
relayblog.com medmajor.ru
rubbed.us play.leadzupc.com
surffoundation.nl swinger-mobil.net
turizm.bz touchmods.fr

928
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00webcams.com 007angels.com
1-99seo.com 0912701309f8ce.com
123cha.com 11235813.webzdarma.cz
14b.info 4kepics.com
273-fz.ru 4replicawatch.net
28n2gl3wfyb0.ru 4webmasters.org
3th.co.in 6hopping.com
abovetherivernc.com 76brighton.co.uk
absolute-s.ru absolute-s.ru
acc.eu.org absugars.com
actionnooz.com academiacsmendoza.org
adanih.com actulite.com
adclickthru.net
admanaerofoil.com
adpremium.org
adprotect.net
adsfresh.men
adult3dgames.com
adultactioncam.com
adultfullhd.com adultfullhd.com
advokat-grodno.by aero2.ru
alessandraleone.com aksonural.ru
all4invest.info akusherok.ru
allfinweb.com aliexpresscashback.club
allforminecraft.ru allnews24.in
alvtank.se allsilver925.co.il
amazingpic.net alyeskaresort.com
ample-awards-today.us amanda-porn.ga
animenime.ru amigobulls.com
antons-transporte.de androids-store.com
apioutoboxnet-a.akamaihd.net anonymousfox.co
artel116.ru apishops.ru
artparquet.ru appartement-stumm.at
ascat.porn apple.com-webbrowsing-security.review
asia-forum.ru apple.com-webbrowsing-security.science
astana.xxxkaz.org arraty.altervista.org
atmovs.com ausergrubhof.info
avon-severozapad.ru autobudpostach.club
awaybird.ru autodo.info
babespcs.com autoplate.info
babyfactory.fr aviav.eu
babyguns.ru azte.ch
backiee.com b00kmarks.com
backuperwebcam.weebly.com bala.getenjoyment.net
bagsonsale.online bangla-chat-uk-paralud.ga
bankiem.pl barnfurnituremart.com
bannerpower.com beauty-things.com
bazaronline24.ru beclean-nn.ru
beachfix.co bedandbreakfast.com
beetpics.pw best-games.today
belinka.com.ua bestempresas.es
beremenyashka.com bibys.com
berrymall.ru bildsuche.ru
bestofferswalkmydogouteveryday.gq bio-japan.net
bestsexyblog.com bjgugu.net.ua
bestwaystogetpaid.us blobar.org
bigcities.org blog.koreadaily.com
billiard-classic.com.ua blogig.org
biznesschat.net blpmovies.com
blackbabesporn.com bmwhighperformers.com
blackcurranthumidifiers.site bobba.dzaba.com
bloggedporn.com bodybuilding-shop.biz
boleznikogi.com bolitgorlo.net
bonzbuddy.com brk-rti.ru
bristolhotel.com builtwith.com
browserprotecter.com buketeg.xyz
buigas.00it.com
buntube.net buntube.net
buron.pw burger-imperia.com
businescoop.men burn-fat.ga
buttons-for-website.com buy-cheap-online.info
buyparajumpers.online
buytizanidineonline.blogspot.com buytizanidineonline.blogspot.com
buzzzg.men bycontext.com
bytimedance.ru c1.onedmp.com
cablecar.us cacheimages.com
candy-glam-hp.com calcularpagerank.com
canyougethighofftizanidine.blogspot.com camdolls.com
cartechnic.ru cartierbracelet.xyz
cartierreplica.win cartierlovestore.com
casinorewards.info cbox.ws
cejewelry.xyz cdncash.com
cercacamion.it chatroulette.si
cheapestjerseysonwholesale.com chee-by.biz
cheapjerseysbizwholesale.us cigarpass.com
check-this-out-now.online cinemaflix.website
cherrypointplace.ca cjs.com.ru
chinacheapelitejerseys.com
christianlouboutinoutlet.win
christianlouboutinshoes.xyz
cialgenisrx.com
cialis-samples.com
cleanpcnow.com
clicksor.com clicksor.com
clippingphotoindia.com clips.ua.ac.be
co.lumb.co co.lumb.co
coinsspb.com comeondog.info
compliance-andrew.xyz comissionka.net
compliance-irvin.xyz compliance-elena.xyz
contentsexpress.com conboy.us
counciltally.com connectionstrenth.com
counter.bloke.com consorzioilmosaico.org
covid-schutzmasken.de contextualyield.com
cracksplanet.com copro.pw
crafthubs.com craftburg.ru
crystalslot.com creams.makeforum.eu
cyprusbuyproperties.com creatives.adbetclickin.pink
cyse.tk czat.wp.pl
d0t.ru date-withme.com
dating-time-now.com dayibiao.com
davidovic.info ddsoldes.fr
deadlinkchecker.com dear-diary.ru
deanmoore.ie dgfitness.co
dellalimov.com diesel-parts28.ru
demon-tweeks.com digest-project.ru
detki-opt.ru dikx.gdn
detskie-zabavi.ru dimkino.ru
dev-seo.blog dipstar.org
dfwu1013.info directrev.com
digital-sale.su doktoronline.no
dignitasdata.se dokumentalkino.net
dildofotzen.net domain-submit.info
diminishedvalueoforegon.com domainsphoto.com
dkvorota.ru domcran.net
dogbreedspicture.net donvito.unas.cz
domaincheck.io dood.live
downloader12.ru downloaddy.net
driving.kiev.ua dpihatinh.gov.vn
drivotracker.com draniki.org
dvd-famille.com droidlook.net
e705.net drugspowerstore.com
earl-brown.info drugstoreforyou.com
elektir.ru dscaas.website
elektrischezigaretten2.beeplog.com duawitchrarato.tk
elektroniksigaraankara.info dyt.net
elektronischezi.livejournal.com ec-file.info
elektronischezigarettekaufen1.beeplog.com ecommerce-seo.com
elektronischezigarettekaufen1.myblog.de edwinkonijn.com.au
enbersoft.com elektrischezigarette2.onsugar.com
elektronischezigarette2.mex.tl
energydiet-info.ru
enews.tech
eralph.tk
erot.co
erotag.com erotag.com
essayassist.com es5.com
essayservicewriting.org everflora.ru
etenininrade.ga exhibitionplus.eu
everypony.ru extstat.com
exmasters.com ezigarettenshop2.postbit.com
exonline.info f1nder.org
expolicenciaslatam.co failingmarriege.blogspot.com
expresstoplivo.ru falopicm.pw
ezigarettenkaufen2.dreamwidth.org fashionavenuegame.com
falcon-images.blogspot.com fealq.com
falllow.gq filesdatabase.net
fandlr.com filesmonster.porn
faz99.com filkhbr.com
feeriaclub.ru filmci.pro
fickenprivat.info finansov.info
findpics.pw
finteks.ru finteks.ru
floridahuntingfishingadventures.com fix-website-errors.com
forum-engineering.ru forms-mtm.ru
forums.toucharcade.com forums.d2jsp.org
foto-weinberger.at free-gluten.ru
fr-bearings.ru freemags.cc
freelinkbuilding.website.tk front.ru
freshmac.space fungamelands.com
funnypica.com galaxy-family.ru
fz139.ttk.ru
gael-s.ru
galeon.com
galeria-zdjec.com
gallerylisting.com gallerylisting.com
gdebestkupit.ru gamingspark.com
geile-lelly.eu geoads.com
getpopunder.com gfaq.ru
gk170.ru gg.zzyjxs.com
glcomputers.ru gifspics.com
gogalleryawesome.com gigapeta.com
googglet.com gok-kasten.net
googlefeud.com goldadpremium.com
googlsucks.com goldandcard.ru
gpirate.com googleseo.com.tr
gsasearchengineranker.xyz green-tea.tv
guruofcasino.com growboxbl.ru
hamptonoaks.ca h2monline.com
hd-filmy.net
hd720kino.ru hd720kino.ru
healgastro.com helicalpile.us
healing-dysplasia.ru hermesreplica.win
heartofbeijing.blogspot.com heroz.fr
hem.passagen.se hostnow.men
hifidesign.ru hotchatdate.com
hitcpm.com hotdl.in
hjaoopoa.top hothot.ru
hmmm.cz
homik.pw
hornymatches.com
host-protection.com
hptwaakw.blog.fc2.com
hspline.com hspline.com
i-service.kz hyhj.info
icaseclub.ru hystersister.com
idealtits.net i-hobot.ru
iflycapetown.co.za iblogpress.xyz
images-free.net ifmo.ru
imagez.co images.gyffu.com
imguramx.pw in-tandem.co
imk.com.ua indiakino.net
inboundlinks.win indianmedicaltourismshop.com
infobabki.ru insomniagamingfestival.com
infodocsportal.com interfucks.net
intermesh.net iptool.xyz
internetartfair.com ivanstroi.ru
islamtoday.co.za jagg.info
istock-mebel.ru jeremyeaton.co
itrevolution.cf jerseysbizwholesalecheap.com
izoll.ru jewelryandfiligree.com
jerseyschinabizwholesale.us justkillingti.me
jerseywholesalechinabiz.com jyrxd.com
joyceblog.top kanimage.com
karpun-iris.ru
karting196.ru karting196.ru
katadhin.co kiwe-analytics.com
kemerovo.zrus.org konteiner24.com
keywordglobal.co.za krafte.ru
keywordsdoctor.com kupit-adenu.ru
kladrus.ru kvartira-sutochno.com
kosmetyki.tm.pl la-fa.ru
kovesszucs.atw.hu lapolis.it
kraljeva-sutjeska.com law-check-nine.xyz
krasnodar.xrus.org law-check-seven.xyz
lafourmiliaire.com lazy-z.com
lafriore.ru lechenie-gemorroya.com
landinez.co lechtaczka.net
lapitec.eu lesbian.xyz
leadn.pl lignofix.ua
leadwayau.com link.ac
lexixxx.com linkrr.com
lida-ru.com lion.misba.us
light.ifmo.ru liupis.com
liky.co.ua livingcanarias.com
lincolntheatre.com llastbuy.ru
lineshops.biz
liver-chintai.org
locationdesaison.com
locimge.pw
lomb.co lomb.co
lombardfinder.ru lovi-moment.com.ua
lottospring.com luckyshop.net.ua
louboutinreplica.pw m-google.xyz
luxup.ru m.b00kmarks.com
m3gadownload.pl magento-crew.net
manuscript.su makedo.ru
mararoom.ru malls.com
marcogrup.com marmitaco.cat
marmotstore.online marwer.info
masturbate.co.uk
mathgym.com.au mathgym.com.au
mebel-ekb.com matsdale.com
members.ghanaweb.com mbiologi.ru
mexicotravelnet.com mebel-vstroika.ru
michaelkorsoutletstore.net mebelcomplekt.ru
michellblog.online meccadumps.net
minecraft-neo.ru megakino.net
mir-business-24.ru mhi-systems.ru
misandesign.se mikrobiologies.ru
mobile-appster.ru minneapoliscopiers.com
mobioffertrck.com misslike.ru
mockupui.com mlf.hordo.win
moinozhki.com mnogolok.info
monarchfind-a.akamaihd.net mobplayer.ru
montazhnic.ru mole.pluto.ro
monclerjacketsoutlet.win
moncleroutletonline.xyz
montredemarque.nl
moz.com moz.com
mrvideospornogratis.xxx mp3ringtone.info
mturkcontent.com msk-diplomat.com
mural.co muschisexbilder.com
musicspire.online musezone.ru
muz-tracker.net musicktab.com
myfreetutorials.com mybackgroundlandscape.blogspot.com
mykings.pw mydeathspace.com
na-telefon.biz myftpupload.com
namenectar.com myyour.eu
nash-krym.info napalm51.nut.cc
naturalbreakthroughsresearch.com naperehresti.info
nextlnk12.com nardulan.com
nextrent-crimea.ru new-apps.ru
nfljerseyscheapbiz.us normalegal.ru
noclegonline.info novodigs.com
novgorod.xrus.org npoet.ru
ntic.fr nudo.ca
nufaq.com nzfilecloud.weebly.com
ok.ru o00.in
okeinfo.online obesidadealgarve.com
onlinadverts.com obiavo.in
offf.info
ohmyrings.com
online-podarki.com
online-templatestore.com online-templatestore.com
onlinewritingjobs17.blogspot.ru onlinefilmz.net
onstrapon.purplesphere.in onstrapon.purplesphere.in
openlibrary.org
openmediasoft.com
opinionreelle.com
orenburg-gsm.ru orenburg-gsm.ru
outshop.ru orhonit.com
owathemes.com paclitor.com
oyster-green.com paintingplanet.ru
panchro.co.uk pandarastore.top
paparazzistudios.com.au paparazzistudios.com.au
paperwritingservice17.blogspot.ru parajumpersstore.online
partners-ship.pro parlament.biz
pateaswing.com partner-stop.men
penisvergrotendepillennl.ovh pcboa.se
pflexads.com pensplan.com
picsforkeywordsuggestion.com pepperstyle.ru
pillscheap24h.com perfection-pleasure.ru
pixell.club photochki.com
pixshark.com pic2fly.com
pizda.lol pinkduck.ga
pirateday.ru
pistonclasico.com
pk-services.ru pk-services.ru
pladform.ru
platezhka.net
play-movie.pl play-movie.pl
poisk-zakona.ru plusnetwork.com
polybuild.ru pogodnyyeavarii.gq
pornhive.org pokemongooo.ml
pornobrazzers.biz pornhub-forum.ga
pornophoto.xyz pornmania.pl
powenlite24.ru porno-video-chati.ru
pr0fit-b0x.com pornoklad.net
pregnant.guru pornokorol.com
printingpeach.com portside.xyz
prefersurvey.net
pretty-mart.com
priceg.com
privetsochi.ru
prize44.com prize44.com
prizefestival.mobi
prod2016.com prod2016.com
prodess.ru profitkode.com
profolan.pl proftests.net
promgirldresses.xyz progress-upakovka.ru
prostitutki-astana.org projectforte.ru
prom23.ru
prostitutki-rostova.org
prpops.com
putevka24.ru
qualitymarketzone.com
quick-seeker.com quick-seeker.com
rankia.com ranksonic.com
rarbg.to razleton.com
razamicroelectronics.com refererx.com
referencemoi.com regionshop.biz
relayblog.com remedyotc.com
remont-komputerov-notebook.ru
resultshub-a.akamaihd.net resultshub-a.akamaihd.net
rewardpoll.com retailwith.com
ricorsogiustizia.org reyel1985.webnode.fr
rocketchange.ru rimedia.org
rome2rio.com ritlweb.com
rostov.xrus.org rniaeba.ga
s.lollypopgaming.com robotixix.com
s8-nowy-wygraj.comli.com rocis.site
sanatorrii.ru rockma.se
sandhillsonline.com roma-kukareku.livejournal.com
sa-live.com
sabaapress.com
samo-soznanie.ru
sampleletters.net
saneitconsulting.com saneitconsulting.com
saneyes.com
sanjosestartups.com sanjosestartups.com
sc-specialhost.com saugatuck.com
scanner-alex.top scanner-margo.top
scanner-irvin.top scenarii-1-sentyabrya.uroki.org.ua
scanner-jess.top scrnet.biz.ua
scanner-jessica.top secret.xn--oogle-wmc.com
scanner-john.top semalt.com
se.bnt-team.com sendearnings.com
searchaddis.com seoboxes.com
searchlock.com
searchtooknow-a.akamaihd.net
seemoreresultshu-a.akamaihd.net
seksotur.ru
seobility.net
seocheki.net
seoexperimenty.ru
seolab.top seolab.top
sexreliz.com serptehnika.ru
sexystrippe.info service.adtech.us
sharebutton.net sexpartygirls.net
shlyahten.ru sfj-ror.no
shopfishing.com.ua sharebutton.to
sildenafilcitratemed.com sheerseo.com
shtaketniki.ru
sibtest.ru
simplepooltips.com simplepooltips.com
site.ru simul.co
siteheart.net sisi-go.ru
sitiz.club skanninge.se
skutecznetabletkinaporostwlosow.pl smart-scripts.com
skypasss.com smartpet.ru
sloopyjoes.com snow.nvr163.com
smailik.org snts.shell-pmr.ru
snjatie-geroinovoy-lomki.ru socialmonkee.com
sockshare.net space2019.top
soda.media
sodexo.com
soft-program.com
soochi.co
soviet-portal.do.am
spabali.org
space-worry.ml
spasswelt.xyz
spidtest.space
sport-video-obzor.ru
sportbetfair.com sportbetfair.com
sports-supplements.us sps-shop.com
statoutlook.info spy-app.info
stats-collector.org spymac.net
stopnarco.ru stariy-baku.com
streamin.to starpages.net
streha-metalko.si strangeduckfilms.com
sundrugstore.com strfls.com
superkanpo.com stroy-portal22.ru
supermodni.com.ua studentguide.ru
surcentro.com subj.ukr-lit.com
sweepstakes.rewardit.com supers.com.ua
tds-advert005.info suralink.com
sureone.pro
surflinksmedical.com
suture.co
svtrd.com
swsociety.se
tacbelarus.ru
tagil.zrus.org
talant-factory.ru
tamada69.com
taximytishi.ru
tdsing.ru tdsing.ru
teencastingporn.com tehngr.ru
terraclicks.com terrafootwear.us
texbaza.by thaismartloan.com
textads.men the-torrent-tracker.blogspot.com
the-usa-games.blogspot.com the-usa-games.blogspot.com
thecoolimages.net thebestweightlosspills.ovh
thegioixekhach.com thebluenoodle.com
thejournal.ru thegameriders.com
thenetinfo.com thepantonpractice.co.uk
thewomenlife.com timeallnews.ru
tiandeural.ru timhost.ru
tizanidineinteractions.blogspot.com tizanidineandcipro.blogspot.com
tizanidinehcl2mgtablet.blogspot.com
tizanidinehcl4mgtablet.blogspot.com
tizanidineonline.blogspot.com
tizanidineoral.blogspot.com tizanidineoral.blogspot.com
tizanidinestreetprice.blogspot.com tizanidinewithdrawal.blogspot.com
tongkatmadura.info toondinsey.com
topappspro.com top10registrycleaners.com
topclickguru.com totu.info
toxicwap.com toy-shop.top
tracxn.com tozup.com
trafaret74.ru tpu.ru
trafficmonetize.org tracfone.com
traffictrade.life trackmedia101.com
tsan.net trafficfactory.biz
tupper-posuda.ru trafficgenius.xyz
turn-up-life.life trafficmania.com
tvand.ru traxdom.ru
tvnewsclips.info truemfilelj.gq
ua.tc trustl.life
ugogo.info tuberkulezanet.ru
ukkelberg.no tupper-shop.ru
ultimatesetnewfreeallsoftupgradesystems.pw twincitiescarservice.com
ultramart.biz uchetunet.su
umnovocaminho.com uginekologa.com
unimodemhalfduplefw.pen.io unblocksit.es
unpredictable.ga undergroundcityphoto.com
upupa.net universals.com.ua
usdx.us uprour.com
v720hd.ru uptime-delta.net
vashsvet.com uptime-gamma.net
urccvfmc.bloger.index.hr
urengoy.pro
ustion.ru
uvozdeckych.info
v-doc.co
valmetrundan.se
vandrie-ict.nl
veerotech.com veerotech.com
video-hollywood.ru vegascosmetics.ru
videochat.cafe vesnatehno.com
videochat.life videochat.ph
videochat.tv.br videochat.world
vids18.site virtuagirl.com
vikistars.com vseigru.one
viktoria-center.ru vzlomsn.org
virus-respirators.com w3data.co
visa-china.ru wait3sec.org
vkmusics.ru wanker.us
vkontarkte.com webalan.ru
voluumtracker1.com webcamdevochka.com
vpnhowto.info weclipart.com
vriel.batcave.net weightatraining.com
vuclip.com westum.se
wallpapersist.com wildnatureimages.com
watchmyfb.pl wjgony.com
we-are-gamers.com worldhistory.biz
weblibrary.win worldmusicfests.com
webmonetizer.net worldoffiles.ru
webscouter.net www.labves.ru
westsextube.com www.wohnkabinen-shop.de
westvilletowingservices.co.za wygraj-teraz.com
wetgames.ru
wholesalejerseychinashop.com
wholesalejerseysgaa.com
wladimirpayen.com
wnoz.de
womensplay.net
world-mmo.com
wovis.site
ww2awards.info
www.bookmaker-bets.com
x-diesel.com
x-diesel.org x-diesel.org
xchangetrak.com xaijo.com
xlovecam.com xbaboon.com
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai xblog.in
xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai xblognetwork.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai xlolitka.com
xn----9sbebi2bvzr7h.xn--p1ai xmlinde.com
xn----itbeirbjbi7bc6bh2d.xn--p1ai xn----7sbahjd3btneuw1joc.xn--p1ai
xn--h1ahbi.com.ua xn--80aafb2a.xn--p1ai
xtube.com xn--80aeahghtf8ac5i.xn--p1ai
yellowads.men xn--hxazdsfy.blogspot.com
yhit.press xvideospornoru.com
yourtemplatefinder.com yebocasino.co.za
youtubedownload.org yottos.com
zagadki.in.ua yourmovies.pl
zaidia.xhost.ro youtube-downloader.savetubevideo.com
zarepta.com yun56.co
zatjmuzu.info za-music.mymobiplanet.com
zed21.net zahvat.ru
zeroredirect8.com zaloadi.ru
zetgie.com.pl zdesformula.ru
zscalerone.net zebramart.ru
zigarettenonlinekaufen2.bloog.pl
znakomstva-piter78.ru
zoogdiesney.com
zoominfo.com
zscaler.net
zvetki.ru
zzlgxh.com zzlgxh.com

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com 2daytrendingnews.com
1kdailyprofit.me 2rich4bitches.com
7search.com 40cg.com
aa625d84f1587749c1ab011d6f269f7d64.com 51unlim.ru
aa8b68101d388c446389283820863176e7.com 5ws.dating-app.ru
abovetherivernc.com 9med.net
adobereader-free.ru aanapa.ru
adtech.fr academia-nsk.org
advocatemsk.ru adcash.com
alert-jdh.xyz adclickservice.com
alfabot.xyz adf.ly
alldubai.biz ads-seo.men
amazingninja.com adultfriendfinder.com
androids-store.com affordablewebsitesandmobileapps.com
animaltoplist.com albamargroup.com
apelsinnik.website ald2014.org
apifasterlightin-a.akamaihd.net animal36.com
arabsexxxtube.com appsaurus.com
arenanews.com.ua aruplighting.com
asmxsatadriverin.aircus.com avkzarabotok.com
ausmepa.org.au avtocredit-legko.ru
automatic-seo.com azartmsl.com
avtointeres.ru bambi.ck.ua
azlex.uz bestssaker.com
b-styles.xyz beta.hotkeys.com
bannerswap.com blackcurranthumidifiers.site
bfz.biz blpmovies.com
bimatoprost-careprost.com boanonihaca.tk
biographiya.com boltalko.xyz
bjanshee.ru bonzibuddi.com
bloggedporn.com breget74.ru
boom-celebs.com burkesales.com
break-the-chains.com buy-cheap-pills-order-online.com
buqyxa.rincian.info californianews.cf
buyessayonline19.blogspot.ru cannibalcheerleader.com
c-english.ru carloans.com
calcularpagerank.com cars-modification.net
canoncdriverq3.pen.io cartierlove2u.com
cartierreplica.pw cenokos.ru
ccbill.com chatroulette.online
cctva.tv chorus.terakeet.com
chcu.net
cinemaenergy-hd.ru
citizenclsdriveri7.pen.io citizenclsdriveri7.pen.io
colobit.biz cityadspix.com
compliance-elena.xyz claytransformations.info
compliance-ivan.top communisave.co.za
compliance-walt.top community.allhiphop.com
conocer-sanabria.com concordexoticrentals.com
copyrightclaims.org cordstrap.cc
covadhosting.biz cxpromote.com
creativehutindia.com dafatiri.com
credit-online.ws dailyfinancefix.com
culad.com daisye.top
cwetochki.ru danashop.ru
daptravel.com demian.kz
decenttools.com divatraffic.com
decorazilla.com domaincdn.xyz
degerlund.net driving.kiev.ua
deluxedumps.com dtm-spain.com
dentfidemountpreach.tk dvrlists.com
devochki.top egvar.net
diusyawiga.tk elektronischezigarettekaufen1.myblog.de
dmmspy.com emediate.eu
dojki-hd.com envaseslotusama.com
domcran.net eorogo.top
drugs-no-rx.info
dyt.net
earncash.com.ua
egovaleo.it
ereko.ru
erotik-kostenlos.net
evening-dating-club.info evening-dating-club.info
expresstoplivo.ru expolicenciaslatam.co
feel-planet.com extremepornos.net
eyemagination.com
failingmarriege.blogspot.com
fetishinside.com
fetlifeblog.com fetlifeblog.com
fillmewithhappiness.com fil.ru
firma-legion.ru firma-legion.ru
flashahead.info firstdrugmall.ru
forexgb.ru
forodvd.com
forsex.info
free-traffic.xyz
freewebs.com freewebs.com
frvo.alptandem.ru fungirlsgames.net
fsakhalin.ru gambarxkata.co
ftns.ru gamezblox.com
fym.com.ru
gay-site.store gay-site.store
getpopunder.com gelezki.com
generic-pills-online.com
get-your-social-buttons.info
getoutofdebtfree.org
ggiaro.com ggiaro.com
glall.ru gilsonchiro.xyz
glavtral.ru girlspicsa.com
gobongo.info goblacked.com
gojiberriess.apishops.ru golden-praga.ru
goodwinmetals.co googleseo.com.tr
gorabagrata.ru grtyi.com
gsasearchengineranker.space handsandlegs.ru
guod.me homelygarden.com
hanink.biz.ly hotcore.info
hard-porn.mobi hotgirlhdwallpaper.com
harmonyglen.us housemilan.ru
hetmanship.xyz hspline.com
homedo.fabpage.com huimin764128.com
hornymatches.com i-midias.net.br
hoverboard360.at iamsport.org
hoverboard360.de imguramx.pw
iminent.com immobiliaremassaro.com
internet-apteka.ru importchinacoach-teach.com
iqoption.pro inbabes.sexushost.com
iron-age.info incest-ru.com
jackwolfskinoutlet.online infazavr.ru
jolic2.com inmate-locator.us
journalhome.com investsuccess.org
jumptap.com ispac.org
kamagragelusa.net itsdp3.com
keki.info japan-bearings.ru
jerseywholesalebizchina.com
jerseywholesaleelitestore.com
joy-penguin.com
kashubadesign.ru
kazinogames.lv
ketoanhanoi.info
keywordsdoctor.com
keywordsuggest.org keywordsuggest.org
kinofak.net kino-rf.ru
kosynka-games.ru kinotorka.ru
kvartir-remont.biz lawyersinfo.org
lanadelreyfans.us liveinternet.ro
letmacworkfaster.world lostaruban.com
libertybilisim.com lottospring.com
lirunet.ru magnetic-bracelets.ru
localmatchbook.com mbiologi.ru
lrac.info medanestesia.ru
luchshie-topcasino.ru microstatic.pl
mail.allnews24.in mini.7zap.com
makedo.ru misswell.net
maladot.com mixed-wrestling.ru
managerpak204.weebly.com mobile-appster.ru
marblestyle.ru moneyteam24.com
mature.free-websites.com movies-in-theaters.net
mediawhirl.net mp3ritm.top
medicineseasybuy.com musicstock.me
medkritika.ru mycaf.it
megagrabber.ru mylida.org
microsoftportal.net myplaycity.com
mikozstop.com nakrutka.cc
minyetki.ru net-radar.com
monclerjacketsoutlet.pw niroo.info
morlat.altervista.org nofreezingmac.click
mrpornogratis.xxx o-dachnik.ru
mrvideospornogratis.xxx ochistka-stokov.ru
muizre.ru okonich.com.ua
muschisexbilder.com okuos.com
mystats.xyz oops-cinema.ru
naperehresti.info orenburg-gsm.ru
narkologiya-belgorod.ru orhonit.com
nfljerseyscheapchinabiz.com orion-v.com
nonews.co pekori.to
nrv.co.za poiskzakona.ru
obiavo.su poptool.net
ohmyrings.com porn-w.org
ons-add.men pornosexrolik.com
openmediasoft.com portadd.men
orsonet.ru portal-eu.ru
oxford-book.com.ua priscilarodrigues.com.br
p-business.ru profit-opportunity.com
paydayloanslocal.com profitkode.com
peekyou.com progress-upakovka.ru
pigrafix.at prokotov.com
piski.top proposal-engine.com
playfortuna-play.ru radiodigital.co
ploenjitmedia.azurewebsites.net rebelmouse.com
plohaya-kreditnaya-istoriya.ru rfd-split.hr
poem-paying.gq ribieiendom.no
popserve.adscpm.net ring4rhino.com
poquoson.org rockprogblog.com
porno-play.net rybalka-opt.ru
privatamateure.com samoiedo.it
profitsport.club savingsslider-a.akamaihd.net
prombudpostach.com.ua seo-traffic-ranking.info
puzo2arbuza.ru
rabotaetvse.ru
refads.pro
renewablewealth.com
rogervivierforsale.com
rollercoin.com
rospromtest.ru
royalcar-ufa.ru
rubanners.com
saveriopiazza.it
scanner-jane.top
scrapinghub.com
seo18.su
seobility.net
seonetwizard.com seonetwizard.com
seorankinglinks.com seozoom.it
sexsaoy.com servicecenter.co.ua
sharpchallenge.com servisural.ru
shell-pmr.ru sexstream.pl
shinikiev.com.ua sibdevice.ru
ship-marvel.co.ua site-auditor.online
shivafurnishings.com sizeplus.work
shmetall.com.ua sotechco.co
sibvitr.ru spymac.net
sildenafilcitratemed.com sribno.net
silvermature.net stop-gepatit.te.ua
sisiynas.ru sumo.com
siteheart.net supervesti.ru
slonechka.ru svarbit.com
socialbookmarksubmission.org symphonyintegratedhealthcare.com
solmarket.by tagil.zrus.org
soundfrost.org
souvenirua.com
stretchingabuckblog.com
stroydetali.ru
superiends.org
surflinksmedical.com
takethatad.com
taraz.xkaz.org taraz.xkaz.org
telesvoboda.ru taxi-v-eisk.ru
thebestweightlosspills.ovh tehngr.ru
thetoiletpaper.com teslathemes.com
tmtrck.com tizanidinehcl4mgtabinfo.blogspot.com
torrents.cd tizanidinemusclerelaxant.blogspot.com
tour-line.net tour-line.net
tpu.ru triplepanda.xyz
track-rankings.online trubywriting.com
trkdf.com tupper-shop.ru
truck-land.life u17795.netangels.ru
truebeauty.cc uchil.net
ttsq.fr uhodzalijami.ru
uhod-za-sobakoj.ru ukrtvory.in.ua
unblocksit.es umekana.ru
uptimebot.net uptimebot.net
upupa.net utrolive.ru
urldelivery.com valmetrundan.se
viandpet.com vapsy.com
vkgaleria.com varbergsvind.se
vykupavto-krasnodar.ru venerologiya.com
vzlom-na-zakaz.com vereo.eu
wallpaperstock.net via-energy-acquistare.com
wildnatureimages.com vigrx-original.ru
windowssearch-exp.com vitoriacabos.com
wnhjavlhezp.gq voluumtrk.com
woodyguthrie.se vriel.batcave.net
ws.ampower.me vykup-avto-krasnodar.ru
wstroika.ru wardreapptokone.tk
x-diesel.info warezkeeper.com
xev.ru webix.biz
xn-----6kcaabbafhu7cskl7akvongwpo7hvjj.xn--p1ai webix.me
xn----7sbbagbq7bd5aheftfllo4m.xn--p1ai website-analyzer.info
xn----ctbbcjd3dbsehgi.xn--p1ai website-speed-check.site
xn--24-glceagatoq7c2a6ioc.xn--p1ai website-speed-up.site
xn--80aodinpgi.xn--p1ai whengirlsgowild.com
xtube.com winterclassichockeyjerseys.com
xuki.us wiosenny-bon-1500.pl
xxxguitars.com wladimirpayen.com
yourtemplatefinder.com wleuaprpxuvr.ga
zebradudka.com wordpresscore.com
zhorapankratov7.blogspot.com xclicks.net
xn----7sbifcamovvfggw9d.xn--p1ai
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
xn----itbkqkfiq.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
yaderenergy.ru
yoursearch.me
youtubologia.it
zastroyka.org
zigarettenonlinekaufen2.drupalgardens.com
zona-aqua.ru
zoompegs.com
zoosexart.com
zs2vm.top

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.109 0; 10.1.0.69 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2839 ### Version: V4.2022.01.2840
### Updated: Sat Jan 8 22:03:36 UTC 2022 ### Updated: Sun Jan 9 06:33:53 UTC 2022
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.01.2839 ### Version: V4.2022.01.2840
### Updated: Sat Jan 8 22:03:36 UTC 2022 ### Updated: Sun Jan 9 06:33:53 UTC 2022
### Bad Bot Count: 631 ### Bad Bot Count: 631
################################################### ###################################################
### Version Information ## ### Version Information ##