vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3037 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-11 10:09:29 +00:00
parent 32f14ba79d
commit bc94d4634f
46 changed files with 87539 additions and 87539 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3036 #### Version: V4.2022.04.3037
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3036 #### Version: V4.2022.04.3037
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3036 #### Version: V4.2022.04.3037
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10808
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3036 #### Version: V4.2022.04.3037
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10812
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10812
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10812
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10812
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 1; 10.1.0.251 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 0; 10.1.0.251 0;
127.0.0.1 0; 127.0.0.1 0;

10812
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 1; 10.1.0.251 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 0; 10.1.0.251 0;
127.0.0.1 0; 127.0.0.1 0;

10812
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 1; 10.1.0.251 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 0; 10.1.0.251 0;
127.0.0.1 0; 127.0.0.1 0;

10812
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3036 3037

10812
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10812
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.100.60 0; 13.91.244.42 0;

10812
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 1; 10.1.0.251 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 0; 10.1.0.251 0;
127.0.0.1 0; 127.0.0.1 0;

10812
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 1; 10.1.0.251 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 0; 10.1.0.251 0;
127.0.0.1 0; 127.0.0.1 0;

10812
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 1; 10.1.0.251 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 0; 10.1.0.251 0;
127.0.0.1 0; 127.0.0.1 0;

10812
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.100.60 0; 13.91.244.42 0;

10812
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.88.100.60 0; 13.91.244.42 0;

10812
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 1; 10.1.0.251 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
BunnySlippers DigitalPebble
Calculon RankActive
Discoverybot Sitebeam
JOC Web Spider Sqworm
Metauri TwengaBot
NimbleCrawler WBSearchBot
RankingBot WWWOFFLE
T8Abot WebAuto
Telesphoreo WebLeacher
Web Sauger pcBrowser

294
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 404checker
ADmantX
AIBOT AIBOT
AfD-Verbotsverfahren ALittle Client
AiHitBot AhrefsBot
Alligator Alligator
Anarchie AlphaBot
Anarchy Anarchy
Anarchy99 Ankit
Anthill Apexoo
Aspiegel Atomseobot
BBBike AwarioRssBot
BDCbot BDCbot
BackDoorBot BackDoorBot
BackWeb
BacklinkCrawler BacklinkCrawler
Badass Badass
Bandit Barkrowler
BatchFTP
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bitacle
Black Hole Black Hole
Blackboard BlackWidow
Blow Blow
BlowFish
Boardreader Boardreader
BotALot
Buddy Buddy
BunnySlippers BuiltBotTough
Cegbfeieh Bullseye
CATExplorador
CCBot
CODE87
CSHttp
CheTeam CheTeam
Chlooe Claritybot
Citoid Cliqzbot
Collector
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Cosmos
Craftbot
CrazyWebCrawler
Crescent Crescent
DBLBot CyotekWebCopy
DTS Agent Disco
Dispatch DnBCrawler-Analytics
DomCopBot DomCopBot
DomainAppender
DomainCrawler
DomainSigmaCrawler
Download Wonder Download Wonder
Dragonfly
Drip Drip
EMail Siphon ECCP/1.0
EasyDL EMail Wolf
Ebingbong EirGrabber
Ecxi Exabot
EroCrawler Express WebPictures
FDM ExtLinksBot
FemtosearchBot Extreme Picture Finder
FHscan
Fimap Fimap
Firefox/7.0 Firefox/7.0
Flunky FlashGet
Foobot Foobot
FrontPage Fuzz
FyberSpider FyberSpider
GalaxyBot Genieo
GermCrawler
GetRight GetRight
GetWeb
Getintent Getintent
Go!Zilla
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla GoZilla
Gotit Gotit
GrabNet Grabber
GrapeshotCrawler Grafula
GrapeFX
HTTP::Lite HTTP::Lite
HaosouSpider
Harvest Harvest
Heritrix
Hloader
HonoluluBot HonoluluBot
Humanlinks
IDBTE4M
IRLbot
Iblog
Id-search Id-search
Image Sucker IlseBot
IndeedBot IndeedBot
InfoTekies
Intelliseek
InterGET InterGET
InternetSeer JOC Web Spider
Jbrofuzz Jetty
Joomla JikeSpider
Jorgee Keyword Density
Kenjin Spider
Kinza
Kozmosbot Kozmosbot
LNSpiderguy
Lanshanbot
Leap
LeechGet
LexiBot LexiBot
Libwhisker LinkWalker
Lightspeedsystems
Linkbot Linkbot
LinkextractorPro
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaScrapeBot
Lipperhey Spider Lipperhey Spider
Litemage_walker
Lmspider Lmspider
Ltx71 MFC_Tear_Sample
MJ12bot MIDown tool
MSFrontPage MIIxpc
Majestic SEO Mag-Net
Masscan MarkMonitor
Mata Hari MarkWatch
Mb2345Browser Mediatoolkitbot
Metauri Metauri
Microsoft Data Access Minefield
Mojeek Mojeek
MolokaiBot Mojolicious
Mr.4x3 Mozlila
NICErsPRO
NPbot NPbot
Navroad Needle
Nessus Net Vampire
Netcraft NetAnts
Nettrack NetZIP
Nibbler Nimbostratus
Nmap Ninja
Octopus Nuclei
Nutch
OnCrawl
OpenLinkProfiler OpenLinkProfiler
OpenVAS
Openvas Openvas
OrangeBot OrangeBot
PECL::HTTP OrangeSpider
POE-Component-Client-HTTP PHPCrawl
PageGrabber PageGrabber
PageScorer PageThing.com
Panscient Pandalytics
Papa Foto Papa Foto
Pavuk Pavuk
PeoplePal
Pi-Monster Pi-Monster
PictureFinder PictureFinder
Piepmatz Piepmatz
Pockey Pockey
ProPowerBot ProPowerBot
Probethenet Probethenet
Psbot Pump
PxBroker PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler RankActive
RSSingBot RankActiveLinkBot
RankFlex
RankingBot RankingBot
RankingBot2 Rankivabot
RankurBot RankurBot
ReGet ReGet
Reaper RealDownload
RebelMouse RepoMonkey
RedesScrapy
Ripper Ripper
RocketCrawler RocketCrawler
SBIder
SEOkicks-Robot SEOkicks-Robot
SEOprofiler
SISTRIX SISTRIX
Scanbot ScanAlert
ScreenerBot Scrapy
SemanticJuice Screaming
Semrush ScrepyBot
SemrushBot Searchestate
Seomoz SearchmetricsBot
Siphon SeoSiteCheckup
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer SiteLockSpider
SiteSnagger SiteSnagger
SiteSucker SlySearch
Siteimprove Snoopy
Sociscraper Sociscraper
Sogou web spider Sosospider
SpaceBison
SpankBot
SputnikBot SputnikBot
Sqlmap Sqlmap
SuperBot Sqlworm
SuperHTTP Steeler
Surfbot Swiftbot
SurveyBot
T0PHackTeam
T8Abot
Teleport Teleport
TightTwatBot TeleportPro
TheNomad
Titan
Toata Toata
Toweyabot Toweyabot
Trendictionbot Tracemyfile
True_Robot Trendiction
Turingos
TurnitinBot TurnitinBot
Twice
URLy.Warning URLy.Warning
Upflow UnisterBot
V-BOT
VB Project VB Project
Vacuum Vagabondo
VidibleScraper VeriCiteCrawler
Virusdie Virusdie
VoidEYE VoidEYE
Voil Voil
Voltron Voltron
WBSearchBot WBSearchBot
WWW-Mechanize WWW-Mechanize
WWW::Mechanize Wallpapers
WWWOFFLE WallpapersHD
Web Auto Web Auto
Web Fuck Web Fuck
WebCopier Web Sauger
WebBandit
WebEnhancer WebEnhancer
WebFetch WebGo IS
WebFuck
WebLeacher
WebPix
WebSauger
WebSucker WebSucker
WebWhacker
WebZIP WebZIP
Webalta WebmasterWorldForumBot
Webshag Webster
WebsiteQuester Whack
Whacker Whacker
Who.is Bot Whatweb
Widow
WinHTTrack WinHTTrack
WiseGuys Robot WiseGuys Robot
Xaldon_WebSpider Xaldon WebSpider
Xenu Xenu
Zade
Zeus
Zitebot Zitebot
ZmEu
ZoomBot ZoomBot
ZumBot
archive.org_bot archive.org_bot
check1.exe arquivo.pt
cah.io.community
coccocbot coccocbot
cognitiveseo cognitiveseo
crawl.sogou.com com.plumanalytics
crawler.feedback crawler4j
dataforseo.com facebookscraper
demandbase-bot gopher
domainsproject.org
heritrix
instabid
internetVista monitor
ips-agent ips-agent
iubenda-radar iubenda-radar
lwp-trivial lwp-request
magpie-crawler
meanpathbot meanpathbot
mediawords muhstik-scan
pcBrowser netEstate NE Crawler
probe-image-size polaris version
s1z.ru
scan.lol scan.lol
seobility serpstatbot
seostar
sitechecker.pro
sogouspider sogouspider
sysscan
tAkeOut
trendiction.com
ubermetrics-technologies.com
webgains-bot webgains-bot
webmeup-crawler
webpros.com
webprosbot webprosbot
x09Mozilla
xpymep1.exe
zauba.io
zgrab

306
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider
404enemy
80legs
ADmantX
AIBOT
ALittle Client
Abonti Abonti
Acunetix Acunetix
AfD-Verbotsverfahren AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Alligator Alexibot
Anarchie AllSubmitter
Anarchy
Anarchy99
Anthill Anthill
Apexoo Apexoo
Asterias Aspiegel
Atomseobot Atomseobot
Attach
AwarioRssBot AwarioRssBot
BBBike BBBike
BDFetch BDFetch
BLEXBot BLEXBot
BackDoorBot BackDoorBot
BatchFTP Bitacle
BetaBot
BlackWidow
Blackboard Blackboard
Brandprotect BlowFish
Buddy Bolt
BuiltWith BuzzSumo
BunnySlippers
CCBot CCBot
CODE87
CSHttp CSHttp
Calculon Calculon
Cegbfeieh CheTeam
CherryPicker Cliqzbot
Collector Cogentbot
Cosmos Copier
Craftbot Copyscape
CrazyWebCrawler
Crescent Crescent
CrunchBot Curious
DBLBot Custo
CyotekWebCopy
DIIbot
DSearch
DTS Agent DTS Agent
DatabaseDriverMysqli Digincore
Demon Dirbuster
Disco Discobot
Discoverybot
Dispatch Dispatch
DittoSpyder DnBCrawler-Analytics
DomainAppender Dotbot
DomainStatsBot Drip
Dragonfly EasyDL
Ebingbong
Ecxi Ecxi
Evil EroCrawler
Express WebPictures Exabot
Extractor
Extreme Picture Finder
Ezooms Ezooms
FDM FDM
FemtosearchBot FHscan
Fimap
Flunky Flunky
FrontPage Foobot
Fuzz Freeuploader
FyberSpider FyberSpider
G-i-g-a-b-o-t Fyrebot
GT::WWW
GalaxyBot GalaxyBot
GermCrawler Genieo
GetRight
GetWeb
Gigabot Gigabot
Go-Ahead-Got-It Go-Ahead-Got-It
GoZilla
Gotit Gotit
GrabNet Grabber
Grafula
GrapeFX GrapeFX
GridBot GridBot
HEADMasterSEO
HTMLparser HTMLparser
HTTP::Lite
HTTrack
HaosouSpider HaosouSpider
Harvest
Humanlinks
IDBTE4M IDBTE4M
Id-search
Image Sucker Image Sucker
Indy Library Indy Library
InfoNaviRobot
InterGET InterGET
Internet Ninja InternetSeer
Iskanie
IstellaBot IstellaBot
JennyBot Kozmosbot
JetCar Lanshanbot
Joomla LeechGet
JustView LibWeb
Kinza
LWP::Simple
Larbin
LexiBot
Libwhisker Libwhisker
LinkScan LinkWalker
Linkbot Linkbot
LinkpadBot LinkextractorPro
LinksManager
Lipperhey Lipperhey
Lipperhey Spider
Litemage_walker Litemage_walker
MJ12bot Ltx71
MIDown tool
MIIxpc
MSIECrawler MSIECrawler
Mail.RU_Bot MTRobot
Majestic SEO Mag-Net
Majestic12 Magnet
MarkMonitor Majestic-SEO
MarkWatch MarkWatch
Mass Downloader
Mata Hari
MauiBot
MeanPath Bot MeanPath Bot
Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri MicroMessenger
Microsoft Data Access Mojeek
Minefield MolokaiBot
Mister PiX Mr.4x3
Mojolicious Msrabot
Morfeus Fucking Scanner Musobot
NPbot NPbot
Nameprotect
NearSite
Needle
NetAnts
NetLyzer NetLyzer
NetMechanic Nettrack
NetSpider
Netvibes
Nibbler Nibbler
Niki-bot
Nimbostratus Nimbostratus
Ninja Ninja
Not
Nuclei Nuclei
Octopus Octopus
Offline Navigator Offline Navigator
OpenVAS OpenVAS
Openfind OrangeBot
OrangeSpider PHPCrawl
OutclicksBot POE-Component-Client-HTTP
OutfoxBot PageAnalyzer
PECL::HTTP Pandalytics
PageGrabber Panscient
PageScorer
PageThing.com
Papa Foto Papa Foto
Pavuk Pavuk
Petalbot PeoplePal
Pockey Picscout
Picsearch
Piepmatz
ProPowerBot
Psbot Psbot
Pu_iN
PxBroker PxBroker
QueryN Metasearch QueryN Metasearch
Quick-Crawler
RankFlex
Rankivabot
Re-re Re-re
ReGet ReGet
Recorder
RedesScrapy
RepoMonkey
Rogerbot
SEOkicks SEOkicks
SEOprofiler SEOlyticsCrawler
SMTBot SMTBot
ScanAlert ScanAlert
Scanbot ScoutJet
Scrapy
ScreenerBot ScreenerBot
ScrepyBot SearchmetricsBot
Searchestate
Seekport Seekport
SemrushBot SemrushBot
SeoSiteCheckup SentiBot
SeobilityBot Seomoz
Site Sucker Shodan
SiteCheckerBotCrawler Siphon
SiteExplorer
SiteLockSpider SiteLockSpider
Sitebeam SiteSnagger
Siteimprove SiteSucker
SmartDownload
Snake Snake
Snoopy Snoopy
SocialRankIOBot
Sogou web spider
Spammen
SpankBot
Spanner
Spbot Spbot
Spinn3r Sqworm
SputnikBot
Stripper Stripper
Sucker SuperBot
SuperHTTP
Surfbot Surfbot
Suzuran Suzuran
T0PHackTeam Szukacz
T8Abot
Teleport
TeleportPro TeleportPro
Telesphoreo TheNomad
Thumbor Thumbor
TightTwatBot Titan
Toweyabot
Tracemyfile Tracemyfile
Trendictionbot Trendiction
True_Robot True_Robot
Turnitin Turnitin
URLy Warning URLy Warning
Upflow Upflow
VelenPublicWebCrawler VB Project
WASALive-Bot Vacuum
WBSearchBot VeriCiteCrawler
WISENutbot Virusdie
Voltron
WWWOFFLE WWWOFFLE
WallpapersHD Wallpapers
WeSEE Web Enhancer
Web Fuck Web Fuck
Web Sauger Web Pix
Web Sucker Web Sucker
WebBandit WebAuto
WebEnhancer WebCollage
WebFetch WebCopier
WebImageCollector
WebPix
WebReaper WebReaper
WebSauger WebStripper
WebSucker WebSucker
WebZIP WebmasterWorldForumBot
Webshag
WebsiteExtractor
Webster Webster
Whacker WiseGuys Robot
Widow Woobot
Xaldon WebSpider Wotbox
Xenu Xaldon_WebSpider
Zauba YoudaoBot
Zermelo Zeus
Zitebot
ZoomBot ZoomBot
ZumBot
ZyBorg ZyBorg
adscanner arquivo-web-crawler
arquivo.pt
autoemailspider autoemailspider
backlink-check backlink-check
coccocbot
cognitiveseo
crawl.sogou.com crawl.sogou.com
crawler.feedback crawler4j
demandbase-bot demandbase-bot
domainsproject.org
facebookscraper facebookscraper
heritrix gopher
isitwp.com instabid
linkdexbot ips-agent
lwp-request iubenda-radar
lwp-trivial lwp-trivial
magpie-crawler
mediawords mediawords
muhstik-scan
netEstate NE Crawler
oBot oBot
polaris version page scorer
probe-image-size ripz
scalaj-http satoristudio.net
scan.lol seocompany.store
serpstatbot seostar
spyfu sogouspider
xpymep1.exe sp_auditbot
sysscan
trendiction.com
voyagerx.com
webmeup-crawler
webprosbot

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
apimountainbikei-a.akamaihd.net bytimedance.ru
architecturebest.com club-musics.ru
casablancamanor.co.za inboundlinks.win
cheapbelstaff.online katjimej.blog.fc2.com
freemaintenancesysforpcandmac.top nottyu.xyz
iframe-toloka.com pokemongooo.ml
krasnodar.xrus.org timecrimea.ru
moesonce.com trustl.life
tizanidineingredients.blogspot.com videogamesecrets.com
veerotech.com yellowproxy.net

922
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00author.com
01apple.com 01apple.com
08800.top 0912701309f8ce.com
0lovespells0.blogspot.com 1-99seo.com
1008.su 101lesbian.xyz
12u.info 12u.info
273-fz.ru 1314dh.com
4rent.ru 1688.com
55wmz.ru 1adult.com
777-club.ru 2girls1cup-free.com
8kisses.eu 2x2fan.ru
DomainStatsBot 3th.co.in
a14download.com 3weekdiet.com
aa8780bb28a1de4eb5bff33c28a218a930.com 3xforum.ro
aab94f698f36684c5a852a2ef272e031bb.com 5000-cotydzien.com
adbetclickin.pink 6128786.com
7wind.ru
aac500b7a15b2646968f6bd8c6305869d7.com
aaronabel.com
abovetherivernc.com
acc.eu.org
adconscious.com adconscious.com
adidas.frwebs.fr
admatic.com.tr
adrunnr.com
ads-seo.men
adsfresh.men adsfresh.men
adverster.com advokat-grodno.by
affrh2015.com advokateg.xyz
agroeconom.kz adzpower.com
ahmedabadwebs.com agapovdg.ru
airlandsea.info akita.kz
alevinefesleri.com allstatesugarbowl.org
all4invest.info alphaforum.ru
allornamenti.com amoi.tn
allpdfmags.net animali.deagostinipassion.it
allproblog.com antons-transporte.de
alyeskaresort.com aosexkontakte.net
amigobulls.com artdestshop.eu
amyfoxfitness.com artel116.ru
apifasterlightin-a.akamaihd.net astana.xxxkaz.org
apishops.ru
appartement-stumm.at
asacopaco.tk
atmagroup.ru atmagroup.ru
atofilms.com aucoinhomes.com
autochoixspinelli.com autotop.com.ua
automobile-spec.com aversis.net
autovideobroadcast.com aviav.ru
avon-novosib.ru avtochehli.by
avtolombard-krasnodar.com azazu.ru
azartclub.org azbaseimages.net
badavit.com.ua babespcs.com
baersaratov.ru bagsonsale.online
baltgem.com bala.getenjoyment.net
bangla-chat-uk-paralud.ga bannertesting.com
bankiem.pl barnfurnituremart.com
bankmib.ru beauty-b0x.pl
bannerswap.com berdasovivan.ru
battle.net best-deals-products.com
beclean-nn.ru best-games.today
benchmarkcommunications.co.uk bestssaker.com
bensbargains.net besttorrentknifta.weebly.com
best-coupon-offer.com betterhdporn.com
bestadbid.com betteroffers.review
bestcalovebracelet.cn bif-ru.info
bestcoin.cc billiard-classic.com.ua
bestfortraders.com bio-optomarket.ru
bestmarriages.com biocn.dx.am
bestpornuha.com biographiya.com
beztuberkuleza.ru bird1.ru
bimmerpost.com bitcoins-live.ru
bingo8888.com bizcheapjerseyswholesalechina.com
bingoporno.com bizfly.info
bio.trade-jp.net bizzliving.com
blue-square.biz blackhatworth.com
bluerobot.info blackle.com
bonanza-fortune.men blobar.org
bonky.biz bloggers.nl
bugshoot.cn bosman.pluto.ro
buigas.00it.com brandbucket.com
buqayy0.livejournal.com brandehk.dk
burger-tycoon.com brothers-smaller.ru
butstrap.space bus-offer.com
buttons-for-your-website.com businescoop.men
buyessay3.blogspot.ru buydissertation.net
bvlgariwallet.xyz bvlgariwallet.xyz
calcularpagerank.com bwlx.prepedu.cn
callejondelpozo.es camdenmemorials.com
carmuffler.net candelluxsklep.pl
candlewooddentalcentre.com.au
canyougethighofftizanidine.blogspot.com
capacitacionyaprendizaje.com
capture-room.com
cars-modification.net
carsplate.com
cartierwatch.xyz cartierwatch.xyz
casablancamanor.co.za caveavins.fr
casinopinup-wins.com centrumcoachingu.com
cayado.snn.gr cheapmoncler.xyz
ccjp.eu
chatroulette.online
cheap-trusted-backlinks.com
cheapbarbour.online
cheapestjerseys-wholesale.com
cheapmoncler.win
checkhit.com
chimiver.info
chinajerseyswholesalecoupons.com chinajerseyswholesalecoupons.com
christianlouboutinshoes.xyz chlooe.com
cjmc.info choice-credit.ru
cleanmypc.com christianlouboutinsaleoutletonline.us
clipartnew.com ci.ua
cmd.kz citysecurity.nu
coccoc.com classicakuhni.ru
collegerentals.com cleansearch.net
compliance-brian.top codq.info
compliance-olga.top compliance-barak.top
crafthubs.com compliance-checker-7.info
crest3d.ru connectionstrenth.com
cryptoswap.biz cookingmeat.ru
cubook.supernew.org coolgramgoods.com
dalavia.ru coprofam.org
datadepths.com counterbot.com
dcj-nn.ru cranly.net
detoxmed24.ru credit.co.ua
dfiles.me customboxes4less.com
digital-sale.su damasarenai.info
disruptingdinnerparties.com danschawbel.com
divatraffic.com davebestdeals.com
docsportal.net dekorkeramik.ru
doctissimo.fr delayreferat.ru
domain-submit.info deluxedumps.com
domainsatcost.com demenageur.com
domashneeruporno.com derevesendeco.com
domik-derevne.ru diamond-necklace.info
dvervmoskvu.ru diffbot.com
e-kwiaciarz.pl digital-video-processing.com
e-poker-2005.com dinkolove.ya.ru
e90post.com docstoc.com
econom.co dogclothing.store
eduserver.net domznaniy.ru
eets.net doublepimp.com
elektrischezigarette1.blog.pl duawitchrarato.tk
duplicashapp.com
earthmagic.info
ebooknovel.club
ed-shop01.ru
ekn-art.se
ekspresihati.info
electrik-avenue.com
electronicadirect.com
elektrischeziga.livejournal.com
elektrischezigarette2.devhub.com elektrischezigarette2.devhub.com
elvenmachine.com elektronischezigarettekaufen1.beeplog.com
enthuse.computernetworksonline.com eloconcream.blogspot.com
evehemming.blogspot.com.au estibot.com
europages.com.ru
everytalk.tv everytalk.tv
exlarseva.webblog.es expdom.com
extads.net extads.net
extener.org extractorandburner.com
ezigarettekaufen1.hpage.com f-loaded.de
ezigaretteshop2.mywapblog.com fandlr.com
f-online.de feel-planet.com
f0815.de fenoyl.batcave.net
f1nder.org fetroshok.ru
falcoware.com ff30236ddef1465f88547e760973d70a.com
familyholiday.ml
fast-torrent.ru
fefo.gdn
fickenbumsen.net
fighrofacciufreesig.ga fighrofacciufreesig.ga
filerockstar298.weebly.com filmi-v.online
fireads.men fitness-video.net
firesub.pl
fitfloponline.store
fivedwld.cf
flex4launch.ru
flowwwers.com
formseo.com
fortevidyoze.net fortevidyoze.net
forum.doctissimo.fr forums.d2jsp.org
forum20.smailik.org forzeronly.com
foto-basa.com free-fbook-traffic.com
franch.info freemags.cc
freecamdollars.com freetangodownload.com
freenom.link freshmac.space
freshberry.com.ua frivgame250.com
front.to fun-mobi.pl
funnypica.com fungirlsgames.net
futbolkisales.ru funponsel.com
fx-brokers-review.com fz139.ttk.ru
gael-s.ru gallerylisting.com
gamebackyard.com gamevalue7.weebly.com
gay.adultgalls.com geckoandfly.com
gazporno.com gelezki.com
gdcentre.ru gg-arena.ru
getaclueamerica.com ghostvisitor.com
gigixo.com gigixo.com
gktt.ru girlspicsa.com
globalscam.ga goforexvps.com
glopages.ru goldandcard.ru
golaya.pw gopixdatabase.com
goo.ne.jp gotomontenegro.net
goodprotein.ru gov.yanao.ru
gotwebsite1.com gribkovye-zabolevaniya.com
graphics8.info gsasearchengineranker.space
gratis-sexkontakte.com gwagka.com
growshop.es hacktougroup.ru
grupografico-pilar.com.ar hahashka.ru
gsasearchengineranker.xyz handicapbathtubarea.com
gsbs.com.ua hebr.myddns-flir.com
gz2.bbsoldes.fr hetmanship.xyz
hamilton.ca hidemyass.com
hatdc.org highspeed5.net
hchha.com home-task.com
hd720kino.ru homeandhealth.ru
hermesreplica.win homegardenlova.com
hidefiles.org homemature.net
hikesearch.net
hollywoodactress.info
homeinns.com
homemade.gq
host-protection.com host-protection.com
hosting-tracker.com hostsshop.ru
hoterika.com
hotsocialz.com
hoverboard360.nl hoverboard360.nl
hscsscotland.com howlongdoestizanidinestayinyoursystem.blogspot.com
hunthillfarmtrust.org https-legalrc.biz
hybrid.ru hully.altervista.org
i-service.kz hundejo.com
ibmdatamanagement.co i-midias.net.br
ico.re id-forex.com
igtools.club imediadesk.com
ihc2015.info impisr.ru
ilikevitaly.com import-sales.com
imguramx.pw infoupdate.org
incanto.in.ua
incep.imagine-ex.co
infobabki.ru
informatiecentro.be
inspiring-desperate.tk
integritylandscapeservices.com
interior-stickers.ru
investsuccess.org
investyb.com
invitefashion.com invitefashion.com
irkutsk.online-podarki.com iron-age.info
isoveti.ru isistaylorporn.info
istmira.ru iskalko.ru
jacago.com istizanidineanopiate.blogspot.com
johnnyhaley.top ivanstroi.ru
justkillingti.me javxxx18.com
jerseywholesalechinabiz.com
jofucipiku.tk
kaac.ru
kambasoft.com kambasoft.com
kandidos.com kamen-e.ru
karpun-iris.ru katadhin.co
ketoanhanoi.info kellyonline.xyz
kfon.eu keywords-monitoring-success.com
khadastoafarde.tk keywordsking.com
kidd.reunionwatch.com keywordspay.com
keywordsuggests.com
kiinomaniak.pl kiinomaniak.pl
kinky-fetishes.com kinogonew.ru
kinofak.net konoplisemena.com
konpax.com kupit-adenu.ru
koversite.info
krasivye-devushki.net
kung-fu-ru.com
kunstaktien.info
kvartira-sutochno.com kvartira-sutochno.com
lanadelreyfans.us kw21.org
lego4x4.ru labelwater.se
levaquin750mg.blogspot.com lacave.ntic.fr
linkbux.ru lapolis.it
linkwithin.com lashstudia.ru
littlecity.ch law-check-eight.xyz
lizace.com leboard.ru
lietaer.com
limads.men
linkdebrideur.xyz
livesex-amateure.info
lmrauction.com
loginduepunti.it loginduepunti.it
lombia.com lol-smurfs.com
longadventure.com lostaruban.com
lrac.info ltvperf.com
m1media.net m1media.net
maranbrinfo.com.br macnewtech.com
mararoom.ru madot.onlinewebshop.net
maximilitary.ru magazintiande.ru
maxxtor.eu maglid.ru
mcar.in.ua makedo.ru
mebel-vstroika.ru manuscript.su
medtherapy.ru map028.com
meet-flirt-dating.com marinetraffic.com
megavolt.net.ua marktforschung-stuttgart.com
menstennisforums.com matrixalchemy.com
metarip.ru mebel-alait.ru
misswell.net mediaoffers.click
mixx.com mega-polis.biz.ua
mlvc4zzw.space methodsmarketing.com
mobsfun.net mindeyegames.com
modnie-futbolki.net miniature.io
mokrayakiska.com missis.top
misslike.ru
misterjtbarbers.com
mistr-x.org
mlf.hordo.win
mobplayer.ru
mojaocena.com
monetizationking.net monetizationking.net
mossmesi.com monetizer.com-01.site
mp3films.ru mouselink.co
msk.afora.ru mp3ritm.top
mturkcontent.com msk-diplomat.com
mural.co mural.co
muz-baza.net muschisexbilder.com
mvpicton.co.uk muycerdas.xxx
my-bc.ru muzaporn.com
mycouponizemac.com my-cash-bot.co
myperiod.club mybinaryoptionsrobot.com
myseoconsultant.com mydoctorok.ru
najaden.no mylesosibirsk.ru
narco24.me myshopmatemac.com
netcheckcdn.xyz mywallpaper.top
newstraveller.ru naj-filmy24.pl
nextconseil.com namenectar.com
nfljerseyscheapbiz.us nardulan.com
nfljerseyscheapchinabiz.com narkologiya-orel.ru
nhl17coinsforps3.gratisblog.biz narkologiya-penza.ru
njkmznnb.ru newyorkhotelsmotels.info
nudepatch.net nhl09.ru
o-o-11-o-o.com nikitsyringedrivelg.pen.io
obiavo.com nomuos.it
obiavo.kz nootrino.com
ofanda.com novatech.vn
okeinfo.online novosti-hi-tech.ru
olvanto.ru nuup.info
onlinemegax.com oakridgemo.com
onload.pw okayimage.com
onthemarch.co okout.ru
origin-my.ru onemactrckr.com
ourville.info onetravelguides.com
paighambot.com online-zaymy.ru
pammik.ru onlyporno.ru
panouri-solare-acoperis.com oralsexfilme.net
papasdelivery.ru osb.se11.ru
parajumpersstore.online osoznanie-narkotikam.net
oxotl.com
paceform.com
paintingplanet.ru
pandroid.co
partner-pop.men
partner-print.men
partnerads.men
partybunny.ru partybunny.ru
pastaleads.com pay2me.pl
pendelprognos.se pcboa.se
perosan.com phormchina.com
peugeot-club.org photo.houseofgaga.ru
picsearch.com photochki.com
picturesmania.com picturesboss.com
piratecams.com piluli.info
pizda.lol pisanieprac.info
pizza-tycoon.com plohaya-kreditnaya-istoriya.ru
placid-rounded-coast.glitch.me poems.com.ua
pochtovyi-index.ru poligon.com
podshipniki-nsk.ru poptool.net
poiskzakona.ru
popunder.net
popup-fdm.xyz
pornhive.org
pornodojd.ru pornodojd.ru
pornoreino.com pornosemki.info
pornstartits.xblog.in pornotubs.com
pornzone.tv pornozhara.com
portal-eu.ru
portnoff.od.ua
predmety.in.ua predmety.in.ua
privatbank46.ru preventheadacheguide.info
priceg.com
priora-2.com
pro-okis.ru
prodess.ru
producm.ru producm.ru
projectforte.ru productarium.com
prostitutki-odessa.org profolan.pl
push-ad.com pronorm.fr
qcstrtvt.bloger.index.hr prostitutki-rostova.org
qld10000.net prostitutki-tyumeni.org
qor360.com pulse33.ru
quelle.ru purchasepillsnorx.com
quickchange.cc pyrodesigns.com.au
rapidsites.pro rankingchart.de
realmonte.net razamicroelectronics.com
remont-ustanovka-tehniki.ru razorweb-a.akamaihd.net
remybutler.fr reckonstat.info
renewablewealth.com reimageplus.com
rewardpoll.com reining.lovasszovetseg.hu
reklama1.ru
reklamuss.ru
remont-komputerov-notebook.ru
remontgruzovik.ru
rezeptiblud.ru
rfserial.net rfserial.net
ringtonepartner.com rique.host.sk
riralmolamsaca.tk royal-betting.net
rusoft-zone.ru rubbed.us
russkie-gorki.ru saddiechoua.com
sa-rewards.co.za saletool.ru
sawin.beth.webd.pl sc-specialhost.com
scanner-alexa.top scanner-brian.top
scanner-george.top scanner-olga.top
screentoolkit.com sdsjweb.com
seeresultshub-a.akamaihd.net seansonline24.pl
semp.net searchaddis.com
searchimage.co
searchinquire.com
seo-platform.com
seo-prof1.xyz seo-prof1.xyz
seoanalyses.com seocheki.net
sergiorossistore.online seofirmreviewsus.info
seogadget.ru
seonetwizard.com
service-core.ru
servicecenter.co.ua servicecenter.co.ua
sexobzor.info serw.clicksor.com
sexxdate.net sharebutton.org
shama-rc.net sharpchallenge.com
share-buttons-for-free.com shell-pmr.ru
shijian.ac.cn shitmovs.com
sierraapps.com shivafurnishings.com
signoredom.com shop-electron.ru
similardeals.net shopcheermakeup.info
simon3.ru shtaketniki.kz
sitechecker.pro simply.net
smadihome.com sinestesia.host.sk
sofit-dmd.ru site-speed-check.site
sitebeam.net
smartbalanceworld.com
snip.tw
snjatie-geroinovoy-lomki.ru
snts.shell-pmr.ru
softlinesolutions.me softlinesolutions.me
space-worry.ml sophang8.com
spasswelt.net sosdepotdebilan.com
splendorsearch-a.akamaihd.net sovetogorod.ru
sps-shop.com squidoo.com
spywarenuker.com sr-rekneskap.no
sribno.net
stair.registrydomainservices.com stair.registrydomainservices.com
stal-rulon.ru standardchartered-forex.com
star61.de
stard.shop
start.myplaycity.com start.myplaycity.com
statistici.ro statustroll.com
stockspmb.info steebook.com
strag-invest.ru streetfire.net
streetfooduncovered.com stretchmate.net
stroicol.net stroymonolit.su
strv.se styro.ru
studworks.org svbur.ru
subj.ukr-lit.com svolze.com
sudexpert66.ru swingerseiten.com
suggest-keywords.com tattomedia.com
suhanpacktech.com tattooha.com
sukirgenk.dvrlists.com tedy.su
surflinksmedical.com telefonsexkostenlos.tk
susanholtphotography.com telefonsexsofort.tk
t-bygg.com teletype.in
taranerymagesswa.blogspot.com template-kid.com
td-33.ru the-usa-games.blogspot.com
tecspb.ru theendivechronicles.com
telegraf.by thefotosgratis.eu
telemetryverification.net threecolumnblogger.com
thebestweightlosspills.ovh timecrimea.ru
ticketsys.inetwd.com tizanidine4mgstreetprice.blogspot.com
tizanidineduringpregnancy.blogspot.com tizanidinepain.blogspot.com
tizanidineformigraines.blogspot.com tmearegion26.com
tizanidinehcl2mgtablet.blogspot.com todohr.com
tizanidinehcl4mgtabinfo.blogspot.com tootoo.to
tizanidinehclsideeffects.blogspot.com totu.us
tizanidinesideeffectsweightloss.blogspot.com track2.shop
topkarkas.com traderzplanet.in
toproadrunner5.info treasuretrack-a.akamaihd.net
tour-line.net trucri.me
trafficjunky.net trumpetedextremes.com
traxdom.ru trustaffs.com
truemfilelj.gq tsyndicate.com
tryrating.com tuckermktg.com
twiclub.in tversvet.ru
tytoona.com twojebook.pl
tyumen.xrus.org twu.com.ua
uamtrk.com udsgame.online
uchil.net uggbootsoutletsale.us
ufa.xrus.org ukrup.com
umg-stroy.ru umg-stroy.ru
umityangin.net umityangin.net
uni.me unacittaconte.org
usacasino.com unblocksit.es
validccseller.com unitexindia.com
uptime-delta.net
urengoy.pro
uroffer.link
usiad.net
uvozdeckych.info
vadimkravtcov.ru
valueclick.com valueclick.com
vandrie-ict.nl varikozdok.ru
varbergsvind.se vip2ch.com
vekzdorov.ru virtualbb.com
vesnatehno.com voditeltrezviy.ru
vezuviy.su voloslove.ru
via-energy-cumpara.com vonradio.com
via-gra.webstarts.com vseigru.one
viddyoze.com vvpg.ru
video-chat.in
videochat.bz
vikistars.com
vinylvault.co.uk
visitcambridge.org
vk-mus.ru
wait3sec.org wait3sec.org
wcb.su wallpapers-best.com
web-betting.ru warezkeeper.com
westsextube.com warningwar.ru
watracker.net
webinstantservice.com
wesharepics.info
wetgames.ru wetgames.ru
whatsappdownload10.com whatistizanidine2mg.blogspot.com
whereverdesperate.gq whiteproduct.com
wiosenny-bon-1500.pl win-spy.com
withstandingheartwarming.com wosik-dach.service-for-web.de
wjgony.com wufak.com
wladimirpayen.com wwwadultcheck.com
woman-h.ru xblog.in
wowcasinoonline.ooo xep.info
wstroika.ru xev.ru
www.get-free-traffic-now.com xn----7sbho2agebbhlivy.xn--p1ai
www.souvenirua.com xn----itbeirbjbi7bc6bh2d.xn--p1ai
x-musics.com xn--80aikhbrhr.net
x-stars.ru xn--80ak6aa92e.com
x5market.ru xn--80aodinpgi.xn--p1ai
xbaboon.com xn--90acenikpebbdd4f6d.xn--p1ai
xchat26.myfreecams.com y8games-free.com
xfluro.com yebocasino.co.za
xgftnlrt.bloger.index.hr youtuhe.com
xmronta.com zdesoboi.com
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai zeleznobeton.ru
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai zeroredirect10.com
xn--b1ag5cfn.xn--p1ai zeroredirect11.com
xnxx-n.com zhacker.net
xsfetish.org zrus.org
xtube.com zverokruh-shop.cz
xxx-treker.ru
xxxhdvideo.site
yopoint.in
yourothersite.com
yuarra.pluto.ro
yur-p.ru
zaim-pod-zalog-krasnodar.ru
zarabotok--doma.ru
zhuravlev.info
zigarettenonl.canalblog.com
zoogdiesney.com
zreloeporno.tv

476
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01apple.com
03e.info
03p.info
100searchengines.com 100searchengines.com
1qingdao.com 123any.com
3rbseyes.com 1j7740kd.website
888.com 2rich4bitches.com
8gold.com 404.mn
adamoads.com 7ooo.ru
adclickthru.net 91abcw.com
adidas.frwebs.fr a96527gi.beget.tech
adloads.net affiliate-fr.com
adtech.fr affrh2015.com
advancedcleaner.com ahhjf.com
adverster.com ahrntal.verymes.xyz
alphacarolinas.com algerianembassy.co.in
amazingpic.net ameblo.top
amung.us apartment.ru
analnoeporno.tv apper.de
analytics-ads.xyz arendatora.ru
artparquet.ru auto-zapchasti.org
asscenihotosidea.blogspot.com azinoofficial777.ru
aucoinhomes.com baditri.com
aviva-limoux.com bagsonsale.online
ayanaspa.com bambi.ck.ua
bestchoice.cf bavsac.com
bestwebsiteawards.com bdsmgalls.net
bildsuche.ru bestsexyblog.com
bimatoprost-careprost.com.ua
binomo.com binomo.com
bjorkbacken.nu bioca.org
blockety.co bioscorp.ru
bloggen.be bodybuilding-shop.biz
blogsfunky672.weebly.com braip.com.br
boleznikogi.com buqayy0.livejournal.com
bonzybuddy.com buscarfoto.com
brakehawk.com carstrends2015.com
brianhenry.co cartierbraceletsreplica.pw
brus-vsem.ru cartierreplica.win
budpost.com.ua catalogs-parts.com
callstevens.com cdnanalytics.xyz
carabela.com.do cenokos.ru
cartierlove2u.xyz cete.ru
cartierlovebraceletreplica.xyz check-host.net
cashonads.com chinese-amezon.com
caveavins.fr cindymatches.com
chiptuninger.com clickso.com
clickaider.com clodo.ru
club-lukojl.ru coccoc.com
cmhomestayagency.com compliance-andrew.top
cocolyze.com compliance-don.top
coindirect.io cool-wedding.net
coinsspb.com coolgramgoods.com
compliance-fred.xyz coolwallpapers4k.info
converse.ddsoldes.fr copypaste.traderzplanet.in
cosmetic.donna7753191.ru covetnica.com
counter.yadro.ru creativehutindia.com
credit-respect.ru cscwtalkto.site
data1.scopich.com custom-product-labels.com
dealwifi.com cutalltheshit.com
delayreferat.ru dailyfinancefix.com
discountliv.com dating-time-now.com
domainsphoto.com dfwu1019.info
domcran.net digitalfaq.com
dpihatinh.gov.vn dikx.gdn
duawitchrarato.tk dispo.de
dverimegapolis.ru dms-sw.ru
ecookna.com.ua doctissimo.fr
ekspertmed.com domik-derevne.ru
el-nation.com doreenblog.online
elektrozigaretten1.webs.com ebonyporn.site
elidelcream.weebly.com efkt.jp
enbersoft.com elektroniksigaraankara.info
erotik-kostenlos.net eloconcream.blogspot.com
exchangeit.gq enginebay.ru
exoneration-project.us envaseslotusama.com
fitness-video.net essaypro.com
forpackningsutveckling.se excitacion.info
fotzen-ficken.com ezigarettekaufen.myblog.de
free-stock-illustration.com ezigarettekaufen2.blox.pl
funtoonez.com ezigarettekaufen2.mpbloggar.se
gambarkatabaru.com ezigarettenkaufen1.pagina.gr
generalporn.org farm26.ru
getaclueamerica.com findpics.pw
gk-atlant.info forpostlock.ru
googglet.com forum.poker4life.ru
google-liar.ru frameimage.org
gta-top.ru fun-mobi.pl
gtopstats.com fun2cell.net
guitar-master.org fym.com.ru
happy.new.yeartwit.com gambarxkata.co
hausfrauensex18.com gdcentre.ru
hollywoodactress.info gepezz.info
honyaku.yahoofs.jp gilsonchiro.xyz
hotsocialz.com goosefishpost.bid
ictizanidinehcl4mg.blogspot.com grand-chlen.ru
iloveitaly.ru gratuitxblcodes.com
imgarcade.com grtyi.com
intim-uslugi.info havepussy.com
invest-pamm.ru healbio.ru
iplogger.org healthcarestore.info
ipornox.xxx hifidesign.ru
isotoner.com hotenergy.ru
jabimgo.pw humbmosquina.tk
joyceblog.top hydropump.su
justdating.online icaseclub.ru
kinozapas.com id-forex.com
kongoultry.net immigrational.info
konteiner24.com impotentik.com
krasivoe-hd.net indo-export.ru
kupiproday.com.ua integritylandscapeservices.com
ladov.ru intellego.info
lancheck.net interior-stickers.ru
laserpen.club internetartfair.com
lechenie-gemorroya.com isoveti.ru
ledpolice.ru jagg.info
lenpipet.ru jikoman.info
lidoradio.com jobgirl24.ru
lowephotos.info jumptap.com
m4ever.net jus0wil.pp.ua
mailemedicinals.com keywordhut.com
medictube.ru keywordspay.com
megaapteka.ru kichenaid.ru
members.ghanaweb.com kinobest.pl
militarysale.pro kmgamex.cf
minharevisao.com konoplisemena.com
mini-modus.ru koopilka.com
mlvc4zzw.space kriokomora.info
mobile-appster.ru ladsup.com
mojowhois.com landinez.co
monarhs.info link.ac
monclerjacketsoutlet.win lisque.batcave.net
morepoweronmac.trade loadingpages.me
morpicert.pw longlifelomilomi.info
mosrif.ru marcoislandvacations.net
musicdaddy.net massage-info.nl
nakrutka.cc medi-fitt.hu
neobux-bg.info mole.pluto.ro
neodownload.webcam mrpornogratis.xxx
net-radar.com msk.afora.ru
netfacet.net music7s.me
novosibirsk.xrus.org muz-baza.net
nvformula.ru nalogovyykodeks.ru
odesproperty.com natali-forex.com
offer.camp negral.pluto.ro
onlinebay.ru netvouz.com
onlineserialy.ru nfljerseyscheapchinabiz.com
orhonit.com obuv-kupit.ru
orsonet.ru ofanda.com
ossmalta.com onlywoman.org
pcboa.se open-odyssey.org
pensplan.com openmultipleurl.com
pepperstyle.ru openstat.com
perederni.net ourtherapy.ru
peugeot-club.org ownshop.win
pinapchik.com pacificair.com
plastweb.ru palvira.com.ua
play-movie.pl panicatack.com
plugingeorgia.com paperwritingservice17.blogspot.ru
popmarker.com pestomou.info
popup.matchmaker.com pics-group.com
positive2b.ru pixelrz.com
pr-ten.de pornhive.org
porno-raskazy.ru
pornoreino.com
porodasobak.net
postclass.com
praisong.net
pretty-mart.com pretty-mart.com
printie.com priscilarodrigues.com.br
prizestohandle.club projefrio.com.br
promgirldresses.xyz prom23.ru
pufip.com prostitutki-kharkova.org
quality-traffic.com pulseonclick.com
rcb101.ru puteshestvennik.com
recipedays.ru qcstrtvt.bloger.index.hr
qexyfu.bugs3.com
qitt.ru
rarbg.to
reelheroes.net reelheroes.net
remorcicomerciale.ro royal-investments.net
resant.ru ruscopybook.com
ribieiendom.no rusenvironmental.net
ruscoininvest.company rvi.biz
salut-camp.ru sanyuprojects.com
sayyoethe.blogspot.co.za sarm.tk
seksvideoonlain.com sc-specialhost.com
semprofile.com scanner-mark.top
seo-platform.com screentoolkit.com
seoprofiler.com seorank.info
service.adtech.us sexs-foto.com
sex-foto.pw shijian.ac.cn
sex-pr.net
sex-tracker.com
sexkrasivo.net
sexobzor.info
sexpartygirls.net
sharebutton.org
shopperifymac.com shopperifymac.com
shymkent.xkaz.org sloopyjoes.com
slimcdn.com smartpet.ru
sonata-arctica.wz.cz smokewithrabbits.com
space4updating.win soc-econom-problems.ru
stardevine.com softomix.com
stockquotes.wooeb.com sosdepotdebilan.com
stroymonolit.su streamin.to
suchenindeutschland.com stroiminsk.com
sudexpert66.ru
surfbuyermac.com surfbuyermac.com
svolze.com surgut.zrus.org
tasteidea.com svarkagid.com
taxi-v-eisk.ru swagbucks.com
technopellet.gr swimpool.ca
swsociety.se
tds-advert002.info
tedxrj.com
tfxiq.com tfxiq.com
theendivechronicles.com
theheroes.ru
theplacetoupdating.pw theplacetoupdating.pw
tizanidineandhydrocodone.blogspot.com thetattoohut.com
tizanidinedruginteractions.blogspot.com tiandeural.ru
tizanidineorflexeril.blogspot.com tizanidineforopiatewithdrawal.blogspot.com
top-karkas.ru todohr.com
top10-way.com tomck.com
topshef.ru topdownloads.ru
track2.shop ttrraacckkrr.com
trafficmania.com
tuckermktg.com tuckermktg.com
turn-up-life.life turn-up-life.life
twodollarshows.com tyumen.xrus.org
ukkala.xyz uasb.ru
ukrtvir.com.ua unionmarkt.de
unpredictable.ga usbggettwku.ga
ural-buldozer.ru usdx.us
v720hd.ru
vacuumscleaner.com
valuado.com
velobikestock.com
versaut.xxx-cam.webcam versaut.xxx-cam.webcam
videochaty.ru via-energy-order.com
via-gra.webstarts.com
video--production.com
video-chat.love
villakohlanta.nu villakohlanta.nu
vkontaktemusic.ru weprik.ru
vpnmouse.com where-toget.com
vut.com.ru while.cheapwebsitehoster.com
wait3sec.org wingsoffury2.com
wardreapptokone.tk worldis.me
webinstantservice.com wormix-cheats.ru
webmasters.stackexchange.com wygraj-teraz.com
website-audit.com.ua x-diesel.com
weburok.com xchat26.myfreecams.com
whatsappdownload10.com xlolitka.com
whereiskentoday.com xmlinde.com
wholesalejerseyscheapjerseys.us.com xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
world-mmo.com xn--b1ag5cfn.xn--p1ai
www.souvenirua.com xn--lifehacer-1rb.com
wygraj-skiny.win xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xn----itbkqkfiq.xn--p1ai xxx-treker.ru
xn--80ahdheogk5l.xn--p1ai zahvat.ru
yaminecraft.ru zajm-zalog-krasnodar.ru
youjizz.com zeroredirect10.com
zaidia.xhost.ro zeroredirect6.com
zog.link
zverokruh-shop.cz

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.15 0; 10.1.0.251 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3036 ### Version: V4.2022.04.3037
### Updated: Sun Apr 10 22:01:23 UTC 2022 ### Updated: Mon Apr 11 10:03:43 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3036 ### Version: V4.2022.04.3037
### Updated: Sun Apr 10 22:01:23 UTC 2022 ### Updated: Mon Apr 11 10:03:43 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##