vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-10 14:56:18 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.12.3503 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-12-02 10:09:07 +00:00
parent 385069efec
commit fc2397c911
46 changed files with 89114 additions and 89114 deletions
Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3502 #### Version: V4.2022.12.3503
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.12.3502 #### Version: V4.2022.12.3503
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.12.3502 #### Version: V4.2022.12.3503
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11002
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.12.3502 #### Version: V4.2022.12.3503
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11006
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11006
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11006
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11006
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

11006
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

11006
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

11006
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3502 3503

11006
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11006
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.83.77 0; 20.14.185.164 0;

11006
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

11006
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

11006
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

11006
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.83.77 0; 20.14.185.164 0;

11006
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
20.245.83.77 0; 20.14.185.164 0;

11006
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 1; 10.1.0.51 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
01h4x.com DomainSigmaCrawler
Aspiegel HTTP::Lite
Cliqzbot Image Fetch
ECCP/1.0 LinqiaScrapeBot
HMView Mass Downloader
Name Intelligence Picsearch
Pixray SEOkicks-Robot
Reaper V-BOT
Sottopop heritrix
Wallpapers/3.0 scalaj-http

308
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
AIBOT 404enemy
ADmantX
ASPSeek ASPSeek
Aboundexbot Abonti
AhrefsBot Aboundex
AiHitBot Acunetix
Alexibot AfD-Verbotsverfahren
Anarchy99 AllSubmitter
Alligator
AlphaBot
Anarchy
Ankit
Aspiegel
Asterias
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BDCbot BBBike
BackStreet Backlink-Ceck
BackWeb Badass
Bandit Bandit
Barkrowler Barkrowler
BlackWidow BetaBot
Blackboard Bigfoot
Bitacle
Black Hole
Boardreader Boardreader
Bolt Bolt
Buddy
BuiltBotTough BuiltBotTough
BuiltWith BuiltWith
Bullseye
BunnySlippers
CATExplorador
CCBot
CODE87 CODE87
Calculon CazoodleBot
Cegbfeieh
CensysInspect CensysInspect
CheeseBot CherryPicker
ChinaClaw Citoid
Cloud mapping
Cogentbot Cogentbot
Collector
CopyRightCheck CopyRightCheck
Copyscape
Cosmos Cosmos
Crawling at Home Project
CrazyWebCrawler
Crescent
Custo Custo
CyotekWebCopy CyotekWebCopy
DSearch DataCha0s
DTS Agent
Deusu Deusu
Digincore DigitalPebble
Discoverybot Disco
Dispatch Discobot
DnyzBot DomainStatsBot
Drip Dotbot
ECCP/1.0 Dragonfly
EMail Siphon EMail Siphon
EMail Wolf EasyDL
EirGrabber EroCrawler
Exabot Evil
Express WebPictures Extractor
ExtractorPro
Extreme Picture Finder Extreme Picture Finder
EyeNetIE
Ezooms
FDM FDM
FHscan FemtosearchBot
FlashGet FlashGet
Flunky Flunky
FrontPage FrontPage
Fuzz
GermCrawler GermCrawler
GetRight Getintent
GetWeb
Go!Zilla Go!Zilla
Gotit GoZilla
GrabNet GrabNet
GrapeFX HMView
GridBot HTMLparser
HTTrack Havij
Haansoft HonoluluBot
Harvest HybridBot
Humanlinks
IDBTE4M
IDBot
Iblog Iblog
Id-search Image Sucker
IndeedBot IndeedBot
Indy Library
InfoNaviRobot InfoNaviRobot
InfoTekies
InternetSeer InternetSeer
IstellaBot Iskanie
Jbrofuzz Jetty
JennyBot
JetCar
Joomla Joomla
Jorgee JustView
Jyxobot Keybot Translation-Search-Machine
Kinza Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot Larbin
LeechFTP LeechGet
LieBaoFast LibWeb
LinkWalker Libwhisker
LinkpadBot LinkpadBot
LinqiaMetadataDownloaderBot LinksManager
LinqiaRSSBot LinqiaRSSBot
Lipperhey Lipperhey
Lipperhey Spider Lmspider
Litemage_walker
MIDown tool
MIIxpc
MJ12bot MJ12bot
MSFrontPage
MSIECrawler MSIECrawler
MTRobot
Mail.RU_Bot Mail.RU_Bot
Majestic-SEO Majestic12
MarkMonitor
MarkWatch
Mass Downloader Mass Downloader
Mata Hari Mata Hari
MauiBot MauiBot
Meanpathbot Meanpathbot
Mediatoolkitbot Mediatoolkitbot
Microsoft Data Access MegaIndex.ru
Mister PiX Microsoft URL Control
Moblie Safari Minefield
Mojolicious Mojeek
Mozlila MolokaiBot
Name Intelligence Mr.4x3
Nameprotect NICErsPRO
NPbot
Navroad
NearSite NearSite
Needle
Nessus
NetAnts
NetLyzer
NetMechanic
Netcraft
Netvibes
NextGenSearchBot
Nibbler Nibbler
Niki-bot
Nimbostratus Nimbostratus
Ninja
Nmap Nmap
Octopus Not
Nuclei
Offline Explorer Offline Explorer
Offline Navigator OnCrawl
Openfind
Openvas Openvas
OutclicksBot OrangeBot
PHPCrawl PageAnalyzer
POE-Component-Client-HTTP PageGrabber
PageScorer
PageThing.com PageThing.com
Pandalytics Pandalytics
Panscient Pi-Monster
Papa Foto Picsearch
PeoplePal
Picscout
Piepmatz Piepmatz
Pixray
PleaseCrawl PleaseCrawl
ProWebWalker
Probethenet Probethenet
Pump
PxBroker PxBroker
PyCurl RankActiveLinkBot
QueryN Metasearch RankFlex
RSSingBot RankurBot
RankingBot2
Re-re
RealDownload RealDownload
Reaper Reaper
RebelMouse Ripper
Recorder RocketCrawler
Rogerbot
SBIder SBIder
SEOkicks SEOkicks-Robot
SalesIntelligent SEOprofiler
SISTRIX
ScanAlert
Scanbot Scanbot
ScoutJet
Scrapy
Screaming Screaming
Searchestate SearchmetricsBot
Semrush SemanticJuice
SemrushBot
SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz
Siphon
Site Sucker Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider
SiteSnagger SiteSnagger
Siteimprove
Sitevigil
SlySearch SlySearch
Snake Snake
Snapbot Spbot
Spammen Spinn3r
Spanner Sqlworm
SputnikBot Stripper
Sqworm
Sucuri Sucuri
SuperBot SurveyBot
SuperHTTP Suzuran
T0PHackTeam T0PHackTeam
Telesoft TeleportPro
TheNomad Telesphoreo
Telesphorep
Thumbor Thumbor
TightTwatBot Toata
Toweyabot
Tracemyfile Tracemyfile
Trendictionbot TurnitinBot
Turnitin Twice
TwengaBot
Upflow Upflow
V-BOT Vacuum
VeriCiteCrawler
Virusdie Virusdie
WPScan Voltron
WWW-Mechanize WWW-Collector-E
WWWOFFLE
Wallpapers
WeSEE
Web Collage
Web Enhancer
Web Fetch
Web Fuck
Web Sauger
WebBandit
Web Collage Web Collage
Web Fetch Web Fetch
WebFuck
WebImageCollector
WebLeacher
Web Pix Web Pix
WebAuto
WebReaper
WebStripper WebStripper
WebWhacker WebWhacker
Webalta Webalta
WebmasterWorldForumBot Webshag
Webster Whack
Whacker Whacker
Who.is Bot Whatweb
Wonderbot Widow
WinHTTrack
Wotbox
Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider
Zauba
Zermelo Zermelo
ZumBot
adscanner
archive.org_bot
arquivo.pt
backlink-check backlink-check
check1.exe check1.exe
clark-crawler
coccocbot coccocbot
cognitiveseo crawl.sogou.com
crawler.feedback crawler.feedback
crawler4j demandbase-bot
domainsproject.org
gopher
heritrix heritrix
instabid instabid
lwp-request ips-agent
isitwp.com
iubenda-radar
lwp-trivial
mediawords
muhstik-scan muhstik-scan
oBot oBot
pcBrowser page scorer
plumanalytics
polaris version polaris version
ripz ripz
s1z.ru
scalaj-http scalaj-http
seobility seobility
seocompany.store
seostar
serpstatbot serpstatbot
sexsearcher
sitechecker.pro
sogouspider sogouspider
trendiction.de
webgains-bot
webmeup-crawler webmeup-crawler
webpros.com webprosbot
x09Mozilla x22Mozilla
zauba.io zgrab

312
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
404enemy 360Spider
AIBOT ADmantX
ALittle Client
Abonti Abonti
Aboundexbot Aboundex
Acunetix Acunetix
AhrefsBot
AiHitBot
Alexibot Alexibot
AllSubmitter AllSubmitter
Anarchy Anarchy
Anarchy99 Anthill
Apexoo
Asterias Asterias
Atomseobot
Attach
AwarioRssBot AwarioRssBot
AwarioSmartBot
BBBike BBBike
BackStreet BetaBot
BackWeb Bigfoot
BacklinkCrawler Bitacle
Bandit
BatchFTP
Battleztar Bazinga
Black Hole Black Hole
BlackWidow
Blackboard Blackboard
BlowFish BlowFish
Bolt Bolt
Brandprotect Brandprotect
Brandwatch
Buck Buck
BuiltBotTough BuiltBotTough
BuiltWith
Bullseye
BuzzSumo
CATExplorador CATExplorador
CODE87 CODE87
CensysInspect Calculon
Cegbfeieh
CheTeam CheTeam
CherryPicker Chlooe
Collector Citoid
Copier Cloud mapping
Curious Cogentbot
Custo CopyRightCheck
DSearch Crawling at Home Project
DataCha0s CrazyWebCrawler
CrunchBot
DTS Agent
Demon Demon
Devil DigitalPebble
Digincore Dirbuster
Discobot Disco
Dispatch Dispatch
DnyzBot Dotbot
DomCopBot Dragonfly
DomainAppender
DomainCrawler
Drip Drip
Ecxi ECCP/1.0
EirGrabber EasyDL
EroCrawler
Evil Evil
Exabot Express WebPictures
ExtLinksBot
Extractor Extractor
Extreme Picture Finder FHscan
FDM FlashGet
Flunky Flunky
Foobot Fyrebot
FrontPage G-i-g-a-b-o-t
FyberSpider GalaxyBot
GermCrawler Genieo
GetRight GetWeb
Go!Zilla Go!Zilla
Go-Ahead-Got-It
Gotit Gotit
Grafula GrabNet
GrapeshotCrawler Grabber
HEADMasterSEO HEADMasterSEO
HTMLparser
HTTP::Lite
Haansoft
HaosouSpider
Harvest
Heritrix Heritrix
Hloader Hloader
Humanlinks Humanlinks
HybridBot HybridBot
IDBot
IRLbot IRLbot
IndeedBot IndeedBot
Indy Library InfoTekies
Intelliseek InterGET
Internet Ninja Iria
IstellaBot
JOC Web Spider
Jbrofuzz
JennyBot JennyBot
JetCar
Jetty Jetty
Joomla JustView
Jyxobot Kenjin Spider
Kinza Keybot Translation-Search-Machine
Kozmosbot LeechGet
Lanshanbot
Larbin
LexiBot
Lftp
LibWeb LibWeb
Libwhisker
LieBaoFast
Lightspeedsystems
Likse
LinkScan
LinkWalker
Linkbot Linkbot
LinkextractorPro
LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey Spider
MJ12bot Litemage_walker
MQQBrowser Ltx71
MSFrontPage MSFrontPage
Mail.RU_Bot Mail.RU_Bot
Majestic SEO Majestic12
MarkMonitor
Mass Downloader Mass Downloader
Mata Hari Mata Hari
MauiBot
Meanpathbot
MegaIndex.ru MegaIndex.ru
Metauri MicroMessenger
Microsoft URL Control Mister PiX
Minefield Mojeek
Moblie Safari MolokaiBot
Mozlila Morfeus Fucking Scanner
Navroad Mr.4x3
Nessus Name Intelligence
NetSpider Nameprotect
NetAnts
NetZIP NetZIP
Nettrack Netvibes
NextGenSearchBot
Nibbler Nibbler
Nikto Nmap
NimbleCrawler Nuclei
Nutch Nutch
Offline Explorer Offline Navigator
OpenLinkProfiler
Openfind
Openvas
OrangeBot OrangeBot
PageGrabber OutclicksBot
PageScorer POE-Component-Client-HTTP
Panscient Page Analyzer
Pandalytics
Papa Foto
Pi-Monster
Picsearch Picsearch
Pixray Pixray
PleaseCrawl PleaseCrawl
ProPowerBot
ProWebWalker
Probethenet Probethenet
Psbot Psbot
Pump RSSingBot
QueryN Metasearch
RankActive RankActive
RankFlex
RankingBot
RankingBot2
Re-re
ReGet ReGet
RebelMouse RealDownload
RedesScrapy RedesScrapy
RepoMonkey RepoMonkey
Ripper Ripper
RocketCrawler
Rogerbot Rogerbot
SEOkicks-Robot SBIder
SEOprofiler SEOkicks
SEOlyticsCrawler
SEOstats
SMTBot SMTBot
SalesIntelligent SalesIntelligent
Scrapy ScanAlert
Screaming ScoutJet
ScreenerBot ScrepyBot
Seekport SearchmetricsBot
Semrush
SentiBot
SeobilityBot SeobilityBot
Site Sucker Shodan
Siphon
SiteExplorer SiteExplorer
SiteSnagger SiteLockSpider
SiteSucker Sitebeam
Siteimprove SlySearch
Sitevigil SmartDownload
Snoopy
SocialRankIOBot
Sociscraper
Sogou web spider
Sosospider
SpaceBison SpaceBison
SpankBot
Spanner Spanner
Spbot Spinn3r
Sqlmap SputnikBot
Sqlworm Steeler
Stripper Stripper
T0PHackTeam Sucker
Surfbot
Suzuran
T8Abot T8Abot
Teleport
TeleportPro TeleportPro
TheNomad Telesoft
Thumbor Telesphoreo
Toata Telesphorep
Toweyabot
Tracemyfile Tracemyfile
Trendiction
Trendictionbot Trendictionbot
True_Robot
Turnitin
TwengaBot TwengaBot
URLy.Warning Twice
UnisterBot
Upflow Upflow
VCI VB Project
Vagabondo
VeriCiteCrawler VeriCiteCrawler
Virusdie VidibleScraper
VoidEYE
WASALive-Bot WASALive-Bot
WEBDAV
WISENutbot
WWW-Collector-E WWW-Collector-E
WWW-Mechanize
WWWOFFLE WWWOFFLE
Wallpapers/3.0
WallpapersHD WallpapersHD
Web Auto WeSEE
Web Collage Web Enhancer
Web Pix WebBandit
Web Sucker WebEnhancer
WebFuck
WebLeacher WebLeacher
WebSauger
WebSucker
Webalta
Webshag
WebsiteExtractor WebsiteExtractor
Webster WebsiteQuester
Widow Whack
WiseGuys Robot Whacker
Wonderbot
Woobot Woobot
Xaldon WebSpider
Xenu Xenu
Zitebot YoudaoBot
Zade
Zeus
ZoomBot
ZoominfoBot ZoominfoBot
ZyBorg arquivo-web-crawler
archive.org_bot
autoemailspider
clark-crawler clark-crawler
coccocbot crawl.sogou.com
demandbase-bot crawler4j
eCatch dataforseo.com
evc-batch evc-batch
facebookscraper
gopher gopher
heritrix
instabid instabid
iubenda-radar isitwp.com
linkdexbot linkdexbot
lwp-request
magpie-crawler
netEstate NE Crawler
page scorer page scorer
plumanalytics pcBrowser
ripz polaris version
s1z.ru
satoristudio.net
scalaj-http scalaj-http
seostar seocompany.store
serpstatbot
siteripz
spyfu
sysscan sysscan
trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
voyagerx.com
webmeup-crawler webmeup-crawler
webprosbot
x09Mozilla
xpymep1.exe xpymep1.exe
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
3-letter-domains.net 2x2fan.ru
agecheckadult.com animal-drawings.com
alta-realestate.com ilikevitaly.com
geilehausfrauen.net lionking-1994.blogspot.com
goldadpremium.com lostaruban.com
gotcher.us misandesign.se
hunthillfarmtrust.org torrent-newgames.com
php-market.ru wrona.it
steebook.com xn--d1acah0c.xn--p1ai
teenforporn.com zawyna.ua

936
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
101lesbian.xyz 100dollars-seo.com
1314dh.com 12u.info
1688.com 2015god.org
21h2o.com 2nt.ru
3dracergames.com 3weekdiet.com
3th.co.in 45en.ru
3xforum.ro 7yue.org
4kpics.rocks
5-steps-to-start-business.com
888.com
8xv8.com
991mostfm.co.id 991mostfm.co.id
above.com aadroid.net
absugars.com aasoldes.fr
abwa.tk absolutelyanalog.com
activepr.ru actulite.com
adconscious.com addblueoff.com.ua
adhome.biz adexprts.com
adrenalin-stk.ru adultwebhosting.info
adultfullhd.com adviceforum.info
adultnet.in agapovdg.ru
adzerg.com alevinefesleri.com
aero2.ru alfa-medosmotr.ru
agardomains.com alkoravto.ru
alert-fjg.xyz all4wap.ru
alert.scansafe.net allcredits.su
alfa-img.com alloysteel.ru
alldubai.biz aludecor.info
allstatesugarbowl.org amanda-porn.ga
am15.net amoi.tn
anal-acrobats.hol.es ampower.me
anaksma.info
anaseracresar.tk anaseracresar.tk
angry-fermi-7633.arukascloud.io animal-drawings.com
anlimebel.kiev.ua animal36.com
apimossnetinfo-a.akamaihd.net apifasterlightin-a.akamaihd.net
appearance-cool.com apxeo.info
apptester.tk arenda-yeisk.ru
arendatora.ru artavenuegardenstudios.com
arewater.com artpicso.com
arquapetrarca.info arvut.org
atmagroup.ru
atovh.local-finders.com atovh.local-finders.com
auto-moto-elektronika.cz auto-zapchasti.org
autobrennero.it auto.rusvile.lt
autowebmarket.com.ua autoua.top
aviapanda.ru
aviav.eu
avtochehli.by
avtocredit-legko.ru avtocredit-legko.ru
ayerbo.xhost.ro backlink4u.net
azarttoto.com badavit.com.ua
azbaseimages.net bala.getenjoyment.net
baikaleminer.com balkanfarma.ru
baladur.ru bamo.xsl.pt
balois.worldbreak.com
bankofthewext.com bankofthewext.com
beauty-bracelet.com bard-real.com.ua
best-games.today bazaronline24.ru
bestbuy.ca bestmobilityscooterstoday.com
bestcoin.cc besttorrentknifta.weebly.com
bestofferswalkmydogouteveryday.gq bfz.biz
bestplacetobuyeriacta.jw.lt
bestwebsiteawards.com
betonka.pro
bget.ru
bildsuche.ru
billiard-classic.com.ua
bio-optomarket.ru bio-optomarket.ru
bizfly.info bizcheapjerseyswholesalechina.com
biznesluxe.ru bjgugu.net.ua
biznesschat.net bjorkbacken.nu
blog.remote-computer.de blog.f00kclan.de
blog.yam.com bloggedporn.com
bloglag.com bloggen.be
blogos.kz
bolitgorlo.net bolitgorlo.net
bombla.org book-bets.com
bonzibuddi.com bosman.pluto.ro
brainboosting.club brillianty.info
brandimensions.com bristolhotel.com.ua
buketeg.xyz brk-rti.ru
bukleteg.xyz buydissertation.net
buyclomidonlaine.com
buyhoverboard.com
buypanicdisorderpill.com buypanicdisorderpill.com
buypuppies.ca bvlgariring.xyz
bwlx.prepedu.cn bwlx.prepedu.cn
callawaygolfoutlet.online calstaterealty.us
camdenmemorials.com canrioloadm.gq
cand.jp carstrends2015.com
canrole.com
capsderfudd.tk
cartierlovestore.xyz
cartierreplica.win cartierreplica.win
cartiertrinity.xyz cbcseward.com
casinopinup-wins.com cdnanalytics.xyz
cbb1smartlist12.click centrdebut.ru
chinese-amezon.com cheap-trusted-backlinks.com
chocolateslim-u-srbiji.com checktext.org
cityadspix.com chlooe.com
click2pawn.com christianlouboutinshoes.xyz
clickaider.com cialgenisrx.com
coderstate.com cineacademy.ru
com-onlinesupport.host citetick.com
compliance-andrew.xyz cmd.kz
converse.ddsoldes.fr coin-hive.com
cookie-law-enforcement-ff.xyz collegerentals.com
cosmetic.donna7753191.ru communisave.co.za
crd.clan.su compliance-alexa.top
cubs-jerseys.us compliance-barak.xyz
customerpromos-a.akamaihd.net cookie-law-enforcement-hh.xyz
daisye.top counter.bloke.com
daymusam.com covid-schutzmasken.de
decorationspcs.com cracksplanet.com
digital-video-processing.com cranly.net
diminishedvalueoforegon.com creatives.adbetclickin.pink
docstoc.com credit-respect.ru
doeco.ru cryptoswap.biz
doublepimp.com cyclobenzaprinevstizanidine.blogspot.com
e-collantes.com daretodonate.co
e-commerce-seo1.com data-ox.com
e90post.com data.vtc.pw
ebonyporn.site ddpills.com
ecommerce-seo.com dear-diary.ru
ecookna.com.ua decenttools.com
ed-shop01.ru deda-moroza-zakaz.ru
ekaterinburg.xrus.org deplim.com
electronicadirect.com detmebel.su
diegolopezcastan.com
dieta-personalna.pl
digest-project.ru
dimfour.com
divku.ru
dkvorota.ru
doctormakarova.ru
documentbase.net
domainroam.win
dood.live
dorratex.tn
dosugrostov.site
downloaddy.net
e-buyeasy.com
eandsgallery.com
eas-seo.com
easync.io
ebooknovel.club
ednorxmed.com
education-cz.ru
ekosmetyki.net.pl
electro-prom.com
eleimgo.pw
elektrischezigarette1.blog.pl
elektrischezigarette2.onsugar.com elektrischezigarette2.onsugar.com
elektroniksigaraankara.info elektrischezigarettekaufen2.cowblog.fr
elite-sex-finders.com elektronischezi.livejournal.com
energydiet-info.ru elidelcream.weebly.com
eng-lyrics.com elmifarhangi.com
enpolis.ru enge-fotzen.info
etenininrade.ga enginebay.ru
europages.com.ru euronis-free.com
evaashop.ru evogarage.com
everypony.ru extremepornos.net
expertblog.info eyessurgery.ru
expolicenciaslatam.co ezigaretteshop2.mywapblog.com
extads.net fanoboi.com
extener.org filerockstar298.weebly.com
extlabs.io filesdatabase.net
ezaz.info filmgo.ru
ezigarettenshop2.myblog.de finejewelryshop.ru
falllow.gq firstsiteguide.com
familienzahnaerzte.com fivedwld.cf
fantasticpixcool.com flauntyoursite.com
fearcrow.com florida-tourism.net
feel-planet.com fodelsedagspresenter.nu
fertilitetsradgivningen.se forex.osobye.ru
fitnesspiks.com forex21.ru
flashbannernow.com forum.doctissimo.fr
foodcrafts.website fotosfotos.eu
footbalive.org franch.info
forum.tvmir.org free-share-buttons.top
fototravel.eu freeuploader.com
foxinsocks.ru front.to
free-deals.faith fugarif.ga
freegamesplay.online gamedayassist.com
freemags.cc gamesprite.me
front.ru gardene.ru
game300.ru geckoandfly.com
games.kolossale.ru gemgrab-a.akamaihd.net
gavuer.ru get-free-social-traffic.com
gazoblok.net.ua gg.zzyjxs.com
gcup.ru gk-atlant.info
ge0ip.com glass-msk.ru
getresponse.com glcomputers.ru
gg-arena.ru golden-catalog.pro
ggiaro.com gsbs.com.ua
gifspics.com gyffu.com
girlsatgames.ru hackers-crackers.tk
glall.ru hausfrauensex18.com
gloverid.site havepussy.com
goforexvps.com hawaiielectriclight.com
gojiberriess.apishops.ru hdpixion.com
goldadpremium.com helicalpile.us
goodwinmetals.co heroz.fr
google-liar.ru
googleseo.com.tr
gov.yanao.ru
graphicwe.org
greenidesign.co
groupmoney.ru
gsasearchengineranker.site
gstatey.net
hamptonoaks.ca
hanink.biz.ly
hannasolution.ru
hatdc.org
hermesbelts.xyz
hermesreplica.win
histock.info histock.info
hobild.net hobild.net
hoholikik.club home.myplaycity.com
homelygarden.com homedo.fabpage.com
hostnow.men house.sieraddns.com
hotenergy.ru hoverboard360.es
hoverboard360.nl howmuchdoestizanidinecost.blogspot.com
huhn.altervista.org htmlcorner.com
ideashome.id hunterboots.online
ideoworld.org ideoworld.org
imk.com.ua ihc2015.info
impotentik.com iinstalll-fii1leis.jus0wil.pp.ua
incep.imagine-ex.co ilbe.club
increasewwwtraffic.info imgarit.pw
infoupdate.org incolors.club
insta-add.pro indetiske.ya.ru
integritylandscapeservices.com infobanks.ru
intellekt21.ru infodocsportal.com
investzalog.ru inmate-locator.us
invitefashion.com instabid.tech
int.search.mywebsearch.com
internetproviderstucson.com
iradiology.ru
istizanidineanopiate.blogspot.com
itservicesthatworkforyou.com
izhevsk.zrus.org izhevsk.zrus.org
jav-fetish.com jennyfire.ru
jerseywholesalechinabiz.com jeremyeaton.co
jetsli.de jerseywholesaleelitestore.com
jose.mulinohouse.co johannesburgsingles.co.za
jrpmakati.com johnnyhaley.top
karaganda.xkaz.org joy-penguin.com
kartiniresto.com jumptap.com
kevblog.top jyvopys.com
keyhantercume.com kabbalah-red-bracelets.com
keyword-suggestions.com kalb.ru
kicknights.gq kazka.ru
kihi.gdn kerei.ru
kinobest.pl khafre.us
kinotorka.ru kinoflux.net
kliksaya.com klitimg.pw
kndxbkdx.bloger.index.hr knigonosha.net
konkursowo-24.pl kochanelli.com
konturkrasoty.ru komputernaya-pomosh-moscow.ru
kosova.de koptims.tiu.ru
krasivye-devushki.net kovesszucs.atw.hu
ks1234.com kovrenok.ru
ladov.ru krasnodar.ru
lalalove.ru ktotut.net
lanasshop.ru labafydjxa.su
landinez.co larutti.ru
lankarns.com le-clos-des-alouettes.com
law-check-eight.xyz legionalpha.com
law-check-seven.xyz lesbian.xyz
leboard.ru libertybilisim.com
lexixxx.com likrot.com
life-instyle.com linkwithin.com
lion.snapmap.us listiseltemournlan.gq
littlecity.ch locatellicorretor.com.br
lombia.com locationdesaison.com
longgreen.info lookingglassemb.com
lordzfilmz.me louboutinshoes.xyz
ltvperf.com lovelycraftyhome.com
lunamedia.co lubetube.com
mansiondelrio.co lutherstable.org
manyresultshub-a.akamaihd.net luxup.ru
mapquestz.us mac-shield.com
marketingtechniques.info magnetic-bracelets.ru
malls.com
marcogrup.com
marktforschung-stuttgart.com marktforschung-stuttgart.com
mathgym.com.au mebel-vstroika.ru
mcar.in.ua medmajor.ru
medanestesia.ru meduza-consult.ru
meddesk.ru meendo-free-traffic.ga
mega-bony2017.pl
members.ghanaweb.com
menetie.ru
metascephe.com metascephe.com
microsoftportal.net mezaruk.info
mielec.pl micasainvest.com
minet.club mikozstop.com
mir-business-24.ru mil-stak.com
mixed-wrestling.ru millionare.com
mjchamonix.org minyetki.ru
modnie-futbolki.net mir-betting.ru
mitrasound.ru
mlvc4zzw.space
mmgq.ru
mobile.ok.ru
mobstarr.com
monclerjacketsoutlet.pw monclerjacketsoutlet.pw
mp3ringtone.info moncleronline.xyz
muabancantho.info moyaterapiya.ru
mymobilemoneypages.com musicas.baixar-musicas-gratis.com
musicpro.monster
mydownlodablefiles.com
myftpupload.com
na15.ru na15.ru
netanalytics.xyz nacap.ru
netcheckcdn.xyz namecrumilchlet.tk
netoil.no naturalpharm.com.ua
networkcheck.xyz news-readers.ru
new7ob.com nextconseil.com
newhairstylesformen2014.com nfljerseys.online
niki-mlt.ru nhl09.ru
nikitabuch.com noscrapleftbehind.co
nlfjjunb5.ru notaria-desalas.com
nowtorrents.com noumeda.com
nudejapan.net obiavo.in
o.light.d0t.ru ofanda.com
o00.in
obesidadealgarve.com
oconto.ru
offergroup.info offergroup.info
offtime.ru offtopic.biz
onclickpredictiv.com onemactrckr.com
onlineku.com onlyporno.ru
onlinemeetingnow.com papasdelivery.ru
orgasmatrix.com pcads.ru
oxford-book.com.ua pdn-4.com
oynat.info phrcialiled.com
paceform.com
paparazzistudios.com.au
pathwhelp.org
patol01.pw
photo-clip.ru
picsfair.com picsfair.com
picture-group.com pictures-and-images.com
pisanieprac.info pijoto.net
pivka.xyz pinwallpaper.top
pl-vouchers.com pio.polytopesexempt.com
pix24x7.com
pizda.lol
pk-services.ru
plastweb.ru
play.leadzupc.com play.leadzupc.com
pooleroadmedicalcentre.co.uk pochemychka.net
popads.net poem-paying.gq
popugaychiki.com pontiacsolstice.info
popup-fdm.xyz popcash.net
pornohd1080.online poptool.net
pornosemki.info pornoforadult.com
pornpost.in potoideas.us
porto.abuilder.net praisong.net
pp-budpostach.com.ua princeadvantagesales.com
prefersurvey.net privatefx.all4invest.info
privacyassistant.net proekt-gaz.ru
prokotov.com professionalwritingservices15.blogspot.ru
prostitutki-odessa.org projefrio.com.br
prostitutki-tolyatti.org proposal-engine.com
prpops.com prostitutki-yaroslavlya.org
pshare.biz psa48.ru
pussyspace.net pufip.com
raisedseo.com pushdata.sendpulse.com
ranking2017.ga q-moto.ru
qpypcx.com
questionmarque.ch
rank-checker.online
rankia.com
rankingchart.de rankingchart.de
rankings-analytics.com razvratnoe.org
ranksonic.info recordpage-a.akamaihd.net
reelheroes.net reelheroes.net
remontbiz.ru refererx.com
remote-dba.de regionshop.biz
remybutler.fr rentehno.ru
rennlist.com resellerclub.com
restaurantlescampi.com robotixix.com
retreatia.com rogervivierforsale.com
richinvestmonitor.com rossanasaavedra.net
roll123.com royalcar-ufa.ru
royal-betting.net roznica.com.ua
ruex.org.ua rus-pornuha.com
runtnc.net samara.rosfirm.ru
russia-tao.ru samoiedo.it
sa-live.com santasgift.ml
sabizonline.com satoristudio.net
sashagreyblog.ga scanhub.ru
saveindex.xyz scanner-julianna.top
sbricur.com scanner-walt.top
sbt-aqua.ru scenarii-1-sentyabrya.uroki.org.ua
scanner-don.top searchinterneat-a.akamaihd.net
scanner-george.top seasonvar.ru
scanner-walter.top semaltmedia.com
scanspyware.net seo-smm.kz
schalke04fc.info seocheckupx.com
scoopquest.com serpstat.com
se-welding.ru sevendays.com.ua
se.bnt-team.com sevenstars7.com
seadragonherbery.com sex-dating.co
searchmywindow-a.akamaihd.net sex-sex-sex5.com
securityallianceservices.com sexreliz.com
seksvideoonlain.com sexvporno.ru
seo-traffic-ranking.info shemale-sex.net
seotoolsagency.com shop-electron.ru
service.adtech.us shopsellcardsdumps.com
sex-tracker.com sisiynas.ru
sexkontakte-seite.com siteripz.net
shakhtar-doneck.ru slowmacfaster.trade
shareyards.com smokewithrabbits.com
shijian.ac.cn snegozaderzhatel.ru
shinikiev.com.ua snip.to
shoesonlinebuy.xyz snow.nvr163.com
shoppingmiracles.co.uk soblaznu.net
shtaketniki.kz socas.pluto.ro
sicfor.bcu.cc socialmediasuggest.com
sindragosa.comxa.com soft1.ru
sinel.info softomix.net
skypasss.com soietvousmaime.fr
slujbauborki.ru spanking.to
smartshoppymac.com specialfinanceoffers.com
sobecjvuwa.com.ru spitfiremusic.com
soc-proof.su spy-sts.com
sockshare.net srecorder.com
sodexo.com starwars.wikia.com
softlinesolutions.me staticfs.host
sogimlecal.tk stop-zavisimost.com
solinf.co stroyplus.ru
sortthemesitesby.com
sportbetfair.com
sprttrack.com
squidoo.com
sr-rekneskap.no
sribno.net
sstroy44.ru
steebook.com
stickers-market.ru
streetfire.net
strfls.com
subj.ukr-lit.com
sugarkun.com sugarkun.com
summerlinhomes411.info supermesta.ru
superoboi.com.ua superstats.com
svensk-poesi.com surffoundation.nl
swimpool.ca swinginwithme.ru
tabletkinaodchudzanie.com.pl symbaloo.com
tagil.zrus.org syvertsen-da.no
talant-factory.ru taboola.com
taqywu51.soup.io
teastory.co
telefonsexi.com
telefonsexkostenlos.tk telefonsexkostenlos.tk
telegraf.by
teletype.in
tengohydar.tk tengohydar.tk
thejournal.ru teslathemes.com
thin.me.pn testingads.pro
thruport.com theallgirlarcade.com
tiens2010.ru thebestphotos.eu
titangel-vietnam.com thetardistimes.ovh
tizanidinedrugtest.blogspot.com thexart.club
tizanidineformigraines.blogspot.com timetorelax.biz
tizanidinehcl2mgsideeffects.blogspot.com tizanidineoral.blogspot.com
tizanidineshowupondrugtest.blogspot.com tmearegion26.com
tizanidinesideeffects.blogspot.com tongkatmadura.info
tksn.ru top10-online-games.com
top-deal.com.pl top10-way.com
top-study.work topclickguru.com
top10registrycleaners.com topshef.ru
topflownews.com torrent-newgames.com
touchmods.fr tourcroatia.co.uk
toursmaps.com tourismvictoria.com
track2.shop trackmedia101.com
tubeline.biz trafficshaper.com
trk-4.net
trumpetedextremes.com
tt-ipd.info
tubeoffline.com
tuberkulezanet.ru
tupper-shop.ru
turizm.bz turizm.bz
twelvevisionspartyofcolorado.com twiclub.in
twitlinks.com tx41tclega.ru
typimga.pw txxx.com
udsgame.online ultramart.biz
unrealcommander.biz uptimebot.net
unrealcommander.org valsalud.com
uptime-gamma.net
urldelivery.com
usswrite.com
utrolive.ru
uytmaster.ru
uzporno.mobi
vacances-voyages.info
vacuumscleaner.com
valuado.com
vandrie-ict.nl
vbabule.net
vedomstvo.net
vegan-foods.us vegan-foods.us
velobikestock.com vepad.com
vezuviy.su
vgoloveboli.net vgoloveboli.net
videospornogratisx.net via-energy-order.com
videtubs.pl via-gra.webstarts.com
visa-china.ru videooko.weebly.com
vodkoved.ru viteonlusarezzo.it
vostoktrade.info viveresaniesnelli.it
vrotike.ru vladimir.xrus.org
vseuznaem.com voluumtracker1.com
vvpg.ru vote-up.ru
w3data.co vremya.eu
watchinf.com vuclip.com
wdfdocando.com vut.com.ru
website-speed-up.site vzglyadriv.kg
weddingdresses.xyz warningwar.ru
westermarkanjou.se watracker.net
wetgames.ru webextract.profound.net
wholinkstome.com webinstantservice.com
whos.amung.us weboptimizes.com
windowssearch-exp.com welck.octopis.com
womama.ru wheelchairliftsarea.com
word-vorlagen.net whipme.yopoint.in
worldinternetauthority.com wholesalejerseychinaoutlet.com
wildworld.site
win-spy.com
wineitudes.wordpress.com
womensterritory.ru
woweb.com.ua
wpthemedetector.co.uk
wrona.it wrona.it
www.wohnkabinen-shop.de wtsindia.in
wwwadultcheck.com wurr.voila.net
x-mix.info x-musics.com
x-porno.video x-true.info
xblog.in xblog.in
xitjw.info xlolitka.com
xmronta.com xn----7sbaphztdjeboffeiof6c.xn--p1ai
xn--h1aakne2ba.xn--p1ai xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai
xnxx-n.com xn--1-8sbcpb0bdm8k6a.xn--p1ai
xrus.org xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xvideosporn.biz xn--80aeahghtf8ac5i.xn--p1ai
xxxsiterips.xyz xn--e1aggki3c.xn--80adxhks
yachts-cruise.info xxxrus.org
yoopsie.com xxxtube69.com
you-shall-not-pass.is74.ru yadro.ru
youbrainboost.asia yogamatsexpert.com
youradulthosting.com youdesigner.kz
yoursearch.me youporn-forum.ga
za-music.mymobiplanet.com yourdesires.ru
zastenchivosti.net zaapplesales.blogspot.com
zhorapankratov7.blogspot.com zaloadi.ru
zionstar.net zatjmuzu.info
zojirushi-products.ru zeikopay.com
zona-aqua.ru znakomstva-piter78.ru
zrelaya.pw znaniyapolza.ru
zvuker.net zog.link
zone-kev717.info
zoominfo.com
zootravel.com
zskdla.site

486
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
00author.com
108shot.com
1xbet4you.com 1xbet4you.com
5i2.net 2000k.ru
76brighton.co.uk a.pr-cy.ru
9med.net acmebtn.ml
aaa38852e886ac4af1a3cff9b47cab6272.com actices.com
abusora.com addtoadd.men
adidas.frwebs.fr adman.gr
advocatemsk.ru adservme.com
afslankpillen2017nl.eu advokat-grodno.by
agreda.pluto.ro anaksma.info
alarmobninsk.ru animalrank.com
amazingpic.net apishops.ru
androids-store.com apparel-offer.com
apartment.ru arcadeplayhouse.com
apiappenableinfo-a.akamaihd.net architecturebest.com
appsecurityr.com arendas.net
arabgirls.adultgalls.com arewater.com
arkartex.ru
aruplighting.com
ascat.porn
asrv-a.akamoihd.net asrv-a.akamoihd.net
azte.ch azbukafree.com
bablonow.ru bablonow.ru
baditri.com backgroundpictures.net
bannerads.de battlecarnival.su
battle.net beauty-clinic.ru
bavsac.com bedcapdealers.com
bfz.biz best-seo-solution.com
bioca.org bestbuy.ca
blacktwhite.com bestimagecoollibrary.com
bluesalt.co betshuckclean.com
boazpower.com bimatoprost-careprost.com
bombla.org bitcoinpile.com
bonanza-fortune.men blogs.rediff.com
brainzpod.win boo-arts.com
breastaugmentation.co.za break-the-chains.com
buy-cheap-online.info brianhenry.co
cablecar.us bristolhotel.com
cacheimages.com bukleteg.xyz
camdenmemorials.com businescoop.men
cartierloveringreplica.xyz business-suggested.tk
cbb1smartlist12.click buy-forum.ru
chatroulette.online buyessay3.blogspot.ru
cheap-pills-norx.com cakesplus.com.au
clips.ua.ac.be canrole.com
clodo.ru capture-room.com
com-cleaner.systems carfax.com.ua
cometorussia.net ccjp.eu
compliance-ivan.top cenoval.ru
compliance-josh.top cheapcigarettesc.info
dev33.dioniqlabb.se cheapmarmot.online
digest-project.ru chuckguilford.com
doeco.ru cinemaflix.website
domain-tracker.com clickso.com
domaincdn.xyz cookielawblog.wordpress.com
dustyorate.com coolgramgoods.com
dynainbox.com coolingoods.com
e2click.com cowlmash.com
edge.sharethis.com customsua.com.ua
education-cz.ru daymusam.com
eldiariodeguadalajara.com digital-sale.su
elektrischeziga.livejournal.com dildofotzen.net
embedle.com djihispano.com
epicbrogaming.com docsportal.net
ertelecom.ru doctormakarova.ru
european-torches.ru domination.ml
excaliburfilms.com dostavimvdom.ru
executehosting.com downloadmefiranaratb1972.xpg.com.br
expdom.com e-commerce-seo1.com
ezb.elvenmachine.com ebooknovel.club
f0815.de eksprognoz.ru
farm26.ru elitedollars.com
felizporno.com eloxal.ru
ff30236ddef1465f88547e760973d70a.com enge-fotzen.info
fillmewithhappiness.com ero-advertising.com
finejewelryshop.ru expolicenciaslatam.co
floating-share-buttons.com falopicm.pw
floridamobilebillboards.com fapgon.com
fodelsedagspresenter.nu fast-wordpress-start.com
franecki.net filkhbr.com
freefoto.ca filmi-onlain.info
freshwallpapers.info foto-basa.com
frvo.alptandem.ru fungirlsgames.net
fsalas.com fusoradio.info
gamingspark.com fxtips.ru
gaz-voshod.ru gambarkataku.co
gerhardhealer.com gamevalue7.weebly.com
gfaq.ru gateway.zscalerone.net
gsasearchengineranker.space gaytube.com
gsasearchengineranker.xyz geileweiber.tk
gsbs.com.ua glall.ru
gunsvicceadadebt.tk global-ics.co.za
hifidesign.ru gombita.info
hjaoopoa.top good-mummy.ru
hol.es goodhousekeeping.com
gsasearchengineranker.site
gungamesz.com
hawaiisurf.com
highstairs-a.akamaihd.net
holidaypics.org holidaypics.org
hotenergy.ru hoste.octopis.com
howlongdoestizanidinestayinyoursystem.blogspot.com hotshoppymac.com
iloveitaly.ru humanorightswatch.org
imagerydatabase.com humbmosquina.tk
impisr.edunsk.ru hydropump.su
infostatsvc.com igadgetsworld.com
intellektmedia.at imadedinner.net
interior-stickers.ru imfamous.info
isistaylorporn.info imgata.com
ispac.org interferencer.ru
iyasimasennka.com internetproviderstucson.com
javrip.net iqs.biz.ua
jmat.cn ivoiretechnocom.ci
kamagragelusa.net javlibrary.cc
karaganda.xkaz.org jerseychinabizwholesale.us
katjimej.blog.fc2.com jetsli.de
kazrent.com jo24news.com
keywordbasket.com kinobaks.com
kicknights.gq kinobest.pl
kinocccp.net kol-energo.ru
kmgamex.cf kosmetyki.tm.pl
kndxbkdx.bloger.index.hr landscaping.center
knigonosha.net leboard.ru
kollekcioner.ru linkdebrideur.xyz
konica.kz loftdigital.eu
konoplisemena.com lol-smurfs.com
kvartiry-remont.ucoz.ru louboutinreplica.xyz
lanadelreyfans.us lulea-auktionsverk.se
leadwayau.com lunchrock.co
locationdesaison.com m5home.ru
locksmith.jp magda-gadalka.ru
maranbrinfo.com.br mastershef.club
mearns-tractors.co.uk mebel-alait.ru
megavolt.net.ua meendo-free-traffic.ga
message-warning.net microstatic.pl
mixx.com mixtapetorrent.com
mobioffertrck.com mmofreegames.online
monsterdivx.tv mnogolok.info
mrbitsandbytes.com montredemarque.nl
musicdaddy.net mostantikor.ru
musirc.com mrpornogratis.xxx
mydownloadengine.com msk-diplomat.com
na15.ru multgo.ru
namecrumilchlet.tk musicktab.com
netfacet.net myseoconsultant.com
nhl17coins.exblog.jp noumeda.com
novodigs.com novgorod.xrus.org
nudepatch.net nyfinance.ml
o.light.d0t.ru o-o-11-o-o.com
obesidadealgarve.com onetravelguides.com
olvanto.ru
onclkads.com
onko-24.com onko-24.com
openmultipleurl.com onlineporno.site
origin-my.ru onlythegames.com
orion-v.com oren-cats.ru
pandroid.co orenburg-gsm.ru
partnerads.men ororodnik.goodbb.ru
partnerworkroom.men outshop.ru
pinup-casino1.ru pacificair.com
pixelrz.com pagesense.com
placid-rounded-coast.glitch.me parlament.biz
playboyfiles.xblog.in patterntrader-en.com
plohaya-kreditnaya-istoriya.ru penisvergrotendepillennl.ovh
poems.com.ua picphotos.net
popmarker.com picsforkeywordsuggestion.com
poquoson.org pizdeishn.com
pornstartits.xblog.in poisk-zakona.ru
prostitutki-almata.org pornogad.com
proxyelite.biz pornoklad.net
psoriasis-file.trade pornoxxx.com.mx
pst2017.onlinewebshop.net poshiv-chehol.ru
purplesphere.in prodess.ru
razyboard.com promgirldresses.xyz
relayblog.com prostitutki-kharkova.org
remont-comp-pomosh.ru ranksignals.com
rique.host.sk rapidsites.pro
rusenvironmental.net registratciya-v-moskve.ru
sabizonline.com remont-komputerov-notebook.ru
sarm.tk rossmark.ru
scanner-alexa.top sa-rewards.co.za
scanner-margo.top sbf441.com
scanner-walt.top scanner-elena.top
search.pch.com sdi-pme.com
sel-hoz.com sdrescher.net
seozoom.it
serw.clicksor.com serw.clicksor.com
shinikiev.com.ua sexkontakteao.info
sexobzor.info
sexspornotub.com
shisha-swag.de
shtaketniki.ru shtaketniki.ru
site-analyzer.com siha.de
sitiz.club sisi-go.ru
sklad-24.ru sky-mine.ru
smartshoppymac.com skytraf.xyz
smokewithrabbits.com softtor.com
snomer1.ru spravka130.ru
socialmadesimple.com stackthatbucks.com
sonata-arctica.wz.cz steelmaster.lv
soserfis.com stjamesschool.info
sovinsteel.ru stpicks.com
sportbetfair.com suchenindeutschland.com
ssconstruction.co svtrd.com
stanthonyscatholicchurch.org tacbibirfa.tk
strag-invest.ru tagil.zrus.org
stretchingabuckblog.com taximytishi.ru
strongsignal-a.akamaihd.net teletype.in
sukarame.net tgtclick.com
supermodni.com.ua tizanidinedrugtest.blogspot.com
supers.com.ua tizanidinehydrochloride2mg.blogspot.com
swsociety.se topclickguru.com
td-33.ru topflownews.com
teenforporn.com torture.ml
teguh.info trackzapper.com
telefonsex-ohne0900.net trafficjunky.net
telemetryverification.net trafficmonetize.org
thepornsex.org tvteleport.ru
tizanidine4mg.blogspot.com twu.com.ua
tizanidine4mguses.blogspot.com ugguk.online
tizanidinecost.blogspot.com ukrobstep.com
tizanidineingredients.blogspot.com ukrtvir.com.ua
tizanidinepain.blogspot.com unacittaconte.org
topsy.com uptimebot.net
trion.od.ua uptimechecker.com
try-rx.com v24s.net
tversvet.ru varikozdok.ru
uac.net.au vinylvault.co.uk
urlopener.com vip-file.com
vacuumcleanerguru.com vipcallsgirls.com
videonsk.com voloo.ru
videositename.com vseigru.one
vipps.com.my wallpapers-best.com
visa-pasport.ru wallpapersdesk.info
vote-up.ru winwotgold.pl
vpnhowto.info wma-x.com
waterefficiency.co works.if.ua
webkeyit.com worldhistory.biz
webs.com wygraj-teraz.com
website-analytics.online xgftnlrt.bloger.index.hr
websites-reviews.com xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai
westermarkanjou.se xn--80ak6aa92e.com
while.cheapwebsitehoster.com xn--80aodinpgi.xn--p1ai
whiteelephantwellington.com xxxkaz.org
wholesalecheapjerseysfree.com youdao.com
witherrom55.eklablog.fr youresponsive.com
womens-journal.net zakazfutbolki.com
www.arenda-yeisk.ru zarabiaj-dzis.pl
xep.info zarabotat-na-sajte.ru
xgames-04.com zdesoboi.com
xkaz.org znakomstvaonlain.ru
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xn----8sbarihbihxpxqgaf0g1e.xn--80adxhks
xn----9sbebi2bvzr7h.xn--p1ai
xuki.us
ynymnwbm.bloger.index.hr
yubikk.info
yun56.co
zero1.it
zscalerone.net

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.123 0; 10.1.0.51 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3502 ### Version: V4.2022.12.3503
### Updated: Thu Dec 1 22:01:29 UTC 2022 ### Updated: Fri Dec 2 10:03:12 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.12.3502 ### Version: V4.2022.12.3503
### Updated: Thu Dec 1 22:01:29 UTC 2022 ### Updated: Fri Dec 2 10:03:12 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##