vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 02:59:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.12.2750 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-12-01 22:07:24 +00:00
parent 36b568a553
commit dff249554a
46 changed files with 59859 additions and 59859 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2749 #### Version: V4.2021.12.2750
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.12.2749 #### Version: V4.2021.12.2750
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.12.2749 #### Version: V4.2021.12.2750
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7348
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.12.2749 #### Version: V4.2021.12.2750
#### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7079](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [630](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

7352
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7352
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

7352
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

7352
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 1; 10.1.0.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 0; 10.1.0.170 0;
127.0.0.1 0; 127.0.0.1 0;

7352
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 1; 10.1.0.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 0; 10.1.0.170 0;
127.0.0.1 0; 127.0.0.1 0;

7352
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 1; 10.1.0.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 0; 10.1.0.170 0;
127.0.0.1 0; 127.0.0.1 0;

7352
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2749 2750

7352
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

7352
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.67.190.109 0; 20.115.122.202 0;

7352
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 1; 10.1.0.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 0; 10.1.0.170 0;
127.0.0.1 0; 127.0.0.1 0;

7352
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 1; 10.1.0.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 0; 10.1.0.170 0;
127.0.0.1 0; 127.0.0.1 0;

7352
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 1; 10.1.0.170 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 0; 10.1.0.170 0;
127.0.0.1 0; 127.0.0.1 0;

7352
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.67.190.109 0; 20.115.122.202 0;

7352
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
13.67.190.109 0; 20.115.122.202 0;

7352
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 1; 10.1.0.170 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
80legs Anthill
AwarioRssBot InfoNaviRobot
BuiltWith MeanPath Bot
Fimap
Go!Zilla
Offline Explorer
Offline Navigator Offline Navigator
Searchestate Ripper
Spbot SalesIntelligent
WebCollage Seomoz
Site Sucker
Web Pix
Webshag

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
404enemy 404enemy
80legs
AIBOT
ASPSeek
Abonti
Aboundex Aboundex
Aboundexbot
Acunetix Acunetix
Alexibot AhrefsBot
AllSubmitter AllSubmitter
AlphaBot
Anarchy
Anarchy99 Anarchy99
Ankit Ankit
Aspiegel Asterias
BBBike AwarioRssBot
BDCbot
BDFetch BDFetch
BLEXBot BLEXBot
BackDoorBot BackStreet
Backlink-Ceck
BatchFTP BatchFTP
Battleztar Bazinga Battleztar Bazinga
Blow BlackWidow
Bolt Blackboard
Brandwatch Brandwatch
BunnySlippers BunnySlippers
BuzzSumo CATExplorador
CCBot CensysInspect
Calculon CheeseBot
CazoodleBot
Cegbfeieh
CherryPicker CherryPicker
Chlooe ChinaClaw
Cliqzbot Cloud mapping
Cocolyzebot CopyRightCheck
Cogentbot Crawling at Home Project
Copyscape
Cosmos
CrazyWebCrawler CrazyWebCrawler
DBLBot Custo
DSearch CyotekWebCopy
DIIbot
DTS Agent DTS Agent
DatabaseDriverMysqli
Devil Devil
DigitalPebble Digincore
Disco Discobot
Discoverybot Discoverybot
DnyzBot Dispatch
DomCopBot DittoSpyder
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot
Domains Project
Dotbot Dotbot
ECCP/1.0 Dragonfly
EasyDL EasyDL
Ebingbong Ebingbong
EirGrabber Exabot
Evil ExtLinksBot
ExtractorPro
Extreme Picture Finder
Ezooms Ezooms
FemtosearchBot FDM
Fimap Fimap
Firefox/7.0
Flunky Flunky
Foobot Foobot
G-i-g-a-b-o-t GT::WWW
Genieo GetRight
GermCrawler
GetWeb
Gigabot Gigabot
Gotit Grabber
GrabNet Grafula
GridBot HMView
HEADMasterSEO
HTMLparser HTMLparser
Haansoft HTTP::Lite
HaosouSpider HTTrack
Havij Harvest
Humanlinks Humanlinks
IDBTE4M HybridBot
Id-search IRLbot
IlseBot Iblog
Image Fetch
IndeedBot
InfoTekies InfoTekies
Intelliseek JOC Web Spider
JamesBOT
Jbrofuzz
JikeSpider
Joomla Joomla
Jorgee Jorgee
JustView Jyxobot
Keyword Density Keyword Density
Kinza
Kozmosbot
Lanshanbot Lanshanbot
Larbin Larbin
LeechGet Leap
LexiBot LeechFTP
Lftp Lftp
LibWeb
Libwhisker Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkextractorPro Linkbot
LinksManager Linkdexbot
Litemage_walker LinqiaMetadataDownloaderBot
MIDown tool Lipperhey Spider
MIIxpc Lmspider
MTRobot Ltx71
Magnet MQQBrowser
Majestic-SEO MSIECrawler
Majestic12 Mag-Net
MarkWatch MarkMonitor
Mass Downloader Masscan
Mata Hari MauiBot
MeanPath Bot MeanPath Bot
MicroMessenger MegaIndex.ru
Microsoft Data Access Metauri
Minefield
Mister PiX Mister PiX
Moblie Safari
Mojolicious
Mozlila
Mr.4x3 Mr.4x3
NICErsPRO
Nameprotect
Needle Needle
Nessus Nessus
Net Vampire
NetAnts NetAnts
NetMechanic
NetSpider NetSpider
NextGenSearchBot NetZIP
Netcraft
Netvibes
Nibbler
Niki-bot
NimbleCrawler NimbleCrawler
Nmap Ninja
Not Nuclei
Octopus Nutch
Offline Navigator Offline Navigator
OpenVAS OpenLinkProfiler
POE-Component-Client-HTTP Openvas
PHPCrawl
Page Analyzer Page Analyzer
PageGrabber PageAnalyzer
PageThing.com PageScorer
Pandalytics Pandalytics
Papa Foto Panscient
PeoplePal PeoplePal
Petalbot Petalbot
Picscout Picscout
Picsearch
PictureFinder
Piepmatz
ProPowerBot
ProWebWalker ProWebWalker
Probethenet Pu_iN
Pump Pump
PxBroker PyCurl
RankActive RankFlex
Rankivabot Rankivabot
RankurBot
ReGet
RebelMouse RebelMouse
Recorder
RedesScrapy RedesScrapy
RepoMonkey
RocketCrawler
SEOkicks SEOkicks
SEOkicks-Robot
SEOprofiler SEOprofiler
SMTBot
SalesIntelligent SalesIntelligent
ScanAlert
Scanbot
ScoutJet ScoutJet
SearchmetricsBot SearchmetricsBot
Semrush
SemrushBot SemrushBot
SentiBot SeoSiteCheckup
SiteExplorer Shodan
Sitebeam Sitebeam
Siteimprove Siteimprove
SlySearch SlySearch
SmartDownload SmartDownload
Snake
Snapbot
Snoopy
SocialRankIOBot SocialRankIOBot
Sosospider Sociscraper
Sogou web spider
SpaceBison
Spammen Spammen
Spbot
Sqlworm Sqlworm
Sqworm
Steeler Steeler
Stripper Stripper
Surfbot Sucker
Suzuran SurveyBot
Szukacz TeleportPro
T8Abot Telesphoreo
Teleport Telesphorep
Telesoft TightTwatBot
TheNomad Titan
Thumbor
Toata
Toweyabot Toweyabot
Tracemyfile
Trendiction
Trendictionbot
True_Robot True_Robot
Turingos
Turnitin Turnitin
TwengaBot TwengaBot
Twice UnisterBot
Typhoeus
Upflow Upflow
Vagabondo V-BOT
VelenPublicWebCrawler VB Project
VeriCiteCrawler VeriCiteCrawler
Virusdie
VoidEYE
Voil Voil
WISENutbot WBSearchBot
WEBDAV
WPScan WPScan
WWW-Collector-E WWW-Collector-E
Wallpapers/3.0 WWW-Mechanize
WallpapersHD WWW::Mechanize
WeSEE
Web Auto Web Auto
Web Collage Web Enhancer
Web Fetch Web Fetch
Web Sauger Web Pix
Web Sucker Web Sucker
WebAuto
WebCollage WebCollage
WebEnhancer WebFetch
WebFuck
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebPix WebSucker
Webalta WebWhacker
WebsiteExtractor Webster
WebsiteQuester
Whack
Whacker Whacker
Widow Whatweb
WiseGuys Robot WinHTTrack
Wonderbot Wprecon
Wotbox Xenu
Xaldon WebSpider
YoudaoBot
Zade
Zauba Zauba
ZoominfoBot ZoominfoBot
ZumBot
ZyBorg ZyBorg
arquivo-web-crawler
arquivo.pt
clark-crawler clark-crawler
coccocbot-web coccocbot-web
dataforseo.com crawler.feedback
demandbase-bot demandbase-bot
eCatch
evc-batch
gopher
heritrix heritrix
ips-agent internetVista monitor
isitwp.com lwp-request
meanpathbot mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot
page scorer
probe-image-size
s1z.ru s1z.ru
satoristudio.net satoristudio.net
scalaj-http scalaj-http
seobility scan.lol
seocompany.store seocompany.store
seoscanners
serpstatbot
sexsearcher sexsearcher
siteripz
sogouspider
sp_auditbot
sysscan
trendiction.com
trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
x09Mozilla webprosbot
zauba.io
zgrab

298
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 01h4x.com
360Spider 404checker
404enemy 404enemy
80legs
ADmantX ADmantX
Abonti ASPSeek
Aboundex Aboundexbot
Acunetix
AhrefsBot
AiHitBot
Aipbot Aipbot
Alexibot Alexibot
Alligator Alligator
AlphaBot AlphaBot
Anarchie Anthill
Anarchy Apexoo
Aspiegel Aspiegel
Asterias
Attach
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike
BDCbot BDCbot
BackDoorBot BDFetch
Badass
Bandit Bandit
BatchFTP Barkrowler
Battleztar Bazinga Battleztar Bazinga
BetaBot
Bitacle Bitacle
BlowFish Black Hole
Boardreader
Bolt
BotALot BotALot
Brandwatch BuiltWith
Buck Bullseye
Buddy BuzzSumo
BunnySlippers CSHttp
CCBot Cegbfeieh
Calculon CensysInspect
CazoodleBot
CheTeam CheTeam
CheeseBot
ChinaClaw ChinaClaw
Collector Claritybot
Copyscape Cloud mapping
CrunchBot Cogentbot
Copier
CopyRightCheck
Curious Curious
CyotekWebCopy
DIIbot DIIbot
DSearch
DTS Agent DTS Agent
DataCha0s DigitalPebble
Dirbuster Dirbuster
Disco Discobot
Discoverybot
Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DnyzBot
DomCopBot
DomainAppender
DomainCrawler DomainCrawler
ECCP/1.0 DomainStatsBot
EMail Siphon Domains Project
Dotbot
EasyDL
Ebingbong Ebingbong
Extractor Ecxi
ExtractorPro EirGrabber
Ezooms EroCrawler
Evil
ExtLinksBot
EyeNetIE
FDM FDM
FHscan Fimap
FemtosearchBot
FlashGet
Flunky Flunky
FrontPage FrontPage
Fuzz Fyrebot
GalaxyBot GalaxyBot
GetRight Genieo
Getintent
Go!Zilla Go!Zilla
Grabber GoZilla
Grafula
GrapeFX
GridBot GridBot
HEADMasterSEO
HMView
HTMLparser HTMLparser
HTTP::Lite HTTP::Lite
HybridBot IRLbot
IDBTE4M Iblog
Id-search
Image Fetch Image Fetch
IndeedBot Image Sucker
Indy Library Indy Library
InfoTekies
Internet Ninja Internet Ninja
Iria Iskanie
IstellaBot IstellaBot
JOC Web Spider
JamesBOT
Jbrofuzz
JennyBot
JetCar JetCar
Joomla
Jorgee Jorgee
Keyword Density
Kozmosbot Kozmosbot
LeechFTP LNSpiderguy
LeechGet Leap
LexiBot LibWeb
Lftp LinkScan
Libwhisker Linkbot
Linkdexbot LinkpadBot
LinkextractorPro LinqiaRSSBot
LinqiaMetadataDownloaderBot LinqiaScrapeBot
Lipperhey Spider
Lmspider Lmspider
MSFrontPage
MSIECrawler
MTRobot MTRobot
Mag-Net Mag-Net
Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic-SEO
Mass Downloader Mass Downloader
Mata Hari
MauiBot
Mb2345Browser
MegaIndex.ru
Metauri Metauri
Microsoft URL Control
Minefield Minefield
Mister PiX Mister PiX
Mojeek Moblie Safari
Mozlila
Name Intelligence
Nameprotect Nameprotect
Navroad
NearSite NearSite
Needle
Nessus
Net Vampire Net Vampire
NetAnts NetMechanic
NetSpider
NetZIP
Netcraft Netcraft
Nmap Niki-bot
Offline Navigator Nimbostratus
OnCrawl Nuclei
Openfind Octopus
Openvas OpenLinkProfiler
OrangeSpider OrangeSpider
PECL::HTTP OutclicksBot
PHPCrawl PHPCrawl
POE-Component-Client-HTTP
Page Analyzer Page Analyzer
PageAnalyzer
PageGrabber
PageThing.com PageThing.com
Pandalytics Pandalytics
Panscient Pavuk
PeoplePal PeoplePal
Petalbot
Pi-Monster
Picscout Picscout
Pimonster
Pixray
ProPowerBot ProPowerBot
Psbot Probethenet
Pu_iN Pu_iN
QueryN Metasearch PyCurl
Quick-Crawler RSSingBot
RankActiveLinkBot RankActiveLinkBot
RankFlex RankFlex
RankingBot2
Rankivabot
RankurBot
ReGet ReGet
RealDownload RebelMouse
Reaper
Recorder Recorder
RepoMonkey RepoMonkey
SEOkicks-Robot RocketCrawler
SEOprofiler SEOkicks
SISTRIX SEOstats
Scanbot Scanbot
ScoutJet
Scrapy
Screaming Screaming
ScrepyBot
SearchmetricsBot
SemanticJuice
Semrush Semrush
SemrushBot SentiBot
SeoSiteCheckup SeoSiteCheckup
SeobilityBot Seomoz
SiteSnagger Shodan
Siphon
Site Sucker
Sitebeam Sitebeam
Siteimprove Snake
Snapbot
Snoopy
Sosospider Sosospider
SpaceBison
Spammen Spammen
SpankBot SpankBot
Spinn3r Spbot
Sqlmap
Sqlworm
Sqworm Sqworm
Steeler SuperBot
SuperHTTP
Surfbot
Suzuran
Swiftbot
T8Abot T8Abot
Teleport Teleport
Telesphoreo Telesphoreo
Titan The Intraformant
Toata Toweyabot
Tracemyfile Trendictionbot
Trendiction Turingos
Typhoeus Turnitin
UnisterBot URLy Warning
URLy.Warning
Upflow
VB Project
VCI
Vacuum Vacuum
Vagabondo Vagabondo
VeriCiteCrawler
VidibleScraper VidibleScraper
Voil Voil
Voltron WASALive-Bot
WEBDAV WBSearchBot
WWW-Collector-E WISENutbot
WWWOFFLE WPScan
Wallpapers Wallpapers/3.0
WeSEE Web Auto
Web Collage
Web Enhancer
Web Fetch Web Fetch
WebCopier Web Pix
WebFetch Web Sauger
WebCollage
WebEnhancer
WebFuck WebFuck
WebGo IS WebGo IS
WebLeacher
WebReaper WebReaper
WebStripper WebStripper
WebZIP WebsiteExtractor
Webalta Whack
WebsiteQuester Widow
Whacker WiseGuys Robot
Wonderbot Wonderbot
Woobot
Wotbox
Wprecon Wprecon
Xaldon WebSpider Xaldon WebSpider
Xaldon_WebSpider
Xenu
YoudaoBot YoudaoBot
Zauba Zauba
Zeus Zermelo
Zitebot
ZmEu
ZoomBot
ZoominfoBot
ZumBot
ZyBorg ZyBorg
adscanner adscanner
backlink-check arquivo-web-crawler
crawler4j arquivo.pt
autoemailspider
check1.exe
clark-crawler
crawler.feedback
domainsproject.org domainsproject.org
evc-batch
gopher
internetVista monitor
ips-agent
magpie-crawler magpie-crawler
mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler
oBot oBot
page scorer page scorer
plumanalytics pcBrowser
polaris version
s1z.ru s1z.ru
satoristudio.net
scalaj-http
scan.lol
seoscanners
siteripz siteripz
sogouspider trendiction.de
sp_auditbot ubermetrics-technologies.com
trendiction.com
webpros.com webpros.com
webprosbot x09Mozilla
x22Mozilla
zgrab zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
1xbet4you.com elektronischezi.livejournal.com
ciproandtizanidine.blogspot.com fx-brokers-review.com
go2mike.ru gamerscorps.com
iframe-toloka.com geilehausfrauen.net
nero-us.com movies-in-theaters.net
nofreezingmac.work profitwithalex.info
oranga.host.sk shop.xz618.com
owohho.com stroiminsk.org
parajumpersoutlet.online wellcome2slovenia.ru
thecoolimages.net zhacker.net

924
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
01apple.com 0912701309f8ce.com
08800.top 24videos.tv
1-free-share-buttons.com 2daytrendingnews.com
1008.su 300richmond.co.nz
14b.info 45en.ru
1qingdao.com 4pp13.com
4kepics.com 7search.com
4kplayer.pl 7zap.com
4replicawatch.net 8kisses.eu
7a2rnuey1tw9ar.ru aa8b68101d388c446389283820863176e7.com
7wind.ru adelly.bg
883zy.com admeasures.com
abovetherivernc.com adrunnr.com
academia-nsk.org
adclickservice.com
adidas.frwebs.fr
adloads.com
adnotbad.com
adservme.com
adtech.fr adtech.fr
adultactioncam.com adult3dgames.com
affiliate-fr.com advancedcleaner.com
agardomains.com aerogo.com
airlandsea.info ahmedabadwebs.com
aktivator-windows10.blogspot.com aibolita.com
ald2014.org alkoravto.ru
alert.scansafe.net allboard.xobor.de
all-streaming-media.com alldownload.pw
animebox.com.ua allesohnegirls.net
anjalika.co.in alta-realestate.com
aosheng-tech.com amazon-seo-service.com
ap.senai.br ananumous.ru
apiappenableinfo-a.akamaihd.net anapa-inns.ru
apimountainbikei-a.akamaihd.net anmysite.com
appsaurus.com arquivo.pt
ardimobileinfo.ml asdfz.pro
articlesdirectoryme.info atmagroup.ru
as5000.com
asia-forum.ru
asrvrep-a.akamaihd.net
atofilms.com atofilms.com
atovh.local-finders.com
audiofree.ru
ausergrubhof.info
auto4style.ru auto4style.ru
autotrends.today autoplate.info
aviav.org aviav.ru.com
ayeartoforget.com avitocars.ru
azartclub.org ayanaspa.com
bagcionderlab.com azadnegar.com
balla-boo.se azlex.uz
bank.uz bablonow.ru
bannerpower.com bala.getenjoyment.net
bannerspace.com bamo.xsl.pt
barbourjackets.online battlecarnival.su
bdsmgalls.net bear.gotcher.us
beauby.ru beauty-b0x.pl
beauty-bracelet.com bestmobilityscooterstoday.com
belinvestdom.by beyan.host.sk
bestbrainprod.win
bestplacetobuyeriacta.jw.lt
bestpornuha.com
bestwaystogetpaid.us
betshuckclean.com
betterscooter.com
bg6s0.com bg6s0.com
biographiya.com bgviagrachrx.com
bioscorp.ru bibys.com
bobba.dzaba.com bif-ru.info
bracketsmackdown.com bigpictures.club
browsepulse-a.akamaihd.net bio-japan.net
bscodecs.com blacktwhite.com
btc4u.ru blpmovies.com
burn-fat.ga bluerobot.info
buy-forum.ru boanonihaca.tk
buyessaynow.biz bolezniorganov.ru
bvlgariring.xyz bonanza-fortune.men
callstevens.com bonkers.name
book-bets.com
bookmark4you.com.biz
braindaily.xyz
brandedleadgeneration.com
brianhenry.co
buttons-for-website.com
callcafe.info
camel-beach.com
camsex.xxx-cam.webcam
candcstuccoandstone.com candcstuccoandstone.com
canrole.com candlewooddentalcentre.com.au
cartiertrinity.xyz carapics.com
casablancamanor.co.za carrauterie.be
cashonads.com cartierreplica.pw
casinopinup-wins.com cayado.snn.gr
celebsopics.com cdn.walkme.com
charmstroy.info cdnnetwok.xyz
chatroulette.world cfjump.com
cheap-trusted-backlinks.com cheapjerseysfootballshop.com
chee-by.biz checktext.org
chocolateslim-original.com christianlouboutinreplica.win
cicaki.net ci.ua
citetick.com cialisovercounteratwalmartusa.com
clarithromycin500mg.com cigarpass.com
clickhype.com cinemaenergy-hd.ru
clickintext.net cl.s7.exct.net
clips.ua.ac.be cleanmypc.com
clodo.ru club-lukojl.ru
club-musics.ru coindirect.io
cmhomestayagency.com companies-catalog.com
compliance-brian.top compiko.info
cookielawblog.wordpress.com compliance-elena.xyz
counciltally.com compliance-jess.top
cutt.us compliance-viktor.top
dairyindia.in concordexoticrentals.com
dandiyabeats.in coolgramgoods.com
coolwallpapers4k.info
coprofam.org
cougfan.info
counter.bloke.com
counterbot.com
covetnica.com
cowblog.fr
creativehutindia.com
creditwell.ru
culturevie.info
customsua.com.ua
daydream-studio.ru daydream-studio.ru
deadlinkchecker.com decenttools.com
dealitright.click decoratinghomes.ga
dekorkeramik.ru delayreferat.ru
design-lands.ru dfiles.me
destinationrealestate.com dieta-personalna.pl
detoxmed24.ru dimkino.ru
digital-video-processing.com discountliv.com
dikx.gdn dodlive.mil
dinosaurus.site domainsphoto.com
directrev.com dorratex.tn
discovertreasure-a.akamaihd.net dosugrostov.site
diusyawiga.tk doxysexy.com
divci-hry.info doyouknowtheword-flummox.ml
docs4all.com drev.biz
documentbase.net drugs-no-rx.info
domain2008.com dyt.net
domaincheck.io easyukraine.com
download-walpaperhd.blogspot.com ecommerce-seo.com
downvids.net ekspresihati.info
doxyporno.com election.interferencer.ru
drugspowerstore.com elektronischezi.livejournal.com
dverimegapolis.ru elite-sex-finders.com
dvrlists.com elvenar.com
e-buyeasy.com
eandsgallery.com
easycommerce.cf
econom.co
educatemetv.com
ekspertmed.com
electronicadirect.com
elektroniksigaraankara.info
elstal.com.pl
emergencyneeds.org
emerson-rus.ru emerson-rus.ru
empis.magix.net empathica.com
encodable.com enews.tech
enpolis.ru eng-lyrics.com
erotikchat-24.com englishdictionaryfree.com
eu-cookie-law.blogspot.com epicbrogaming.com
euronis-free.com erot.co
exhibitionplus.eu erotikstories.ru
extrabot.com etenininrade.ga
ezigarettekaufen2.blox.pl etur.ru
ezigarettekaufen2.mpbloggar.se europeanwatches.ru
ezigarettenkaufen1.pagina.gr euroskat.ru
evehemming.blogspot.com.au
excaliburfilms.com
excitacao.com
excitacion.info
extremal-blog.com
extremez.net
f0815.de f0815.de
facecup.top f5mtrack.com
factorynightclub.com falllow.gq
fba-mexico.com faracontrol.ir
fatfasts-4tmz.com
favornews.com
fidelityfunding.com fidelityfunding.com
filmi-v.online filerockstar298.weebly.com
financeloan.us filmfanatic.com
finansov.info finteks.ru
flagcounter.me fitness-video.net
flavors.me fitnesspiks.com
fok.nl flprog.com
focalink.com
foodcrafts.website foodcrafts.website
forensicpsychiatry.ru footbalive.org
forsex.info forpackningsutveckling.se
free-deals.faith foto-telok.net
freshberry.com.ua fotopop.club
fz139.ttk.ru free-traffic.xyz
freelifetimefuckbook.com
freeuploader.ml
fuel-gas.com
g33.org g33.org
g7m.pl gallictures.com
gameonasia.com gambarxkata.co
gaytube.com gateway.zscalerone.net
gcup.ru gavuer.ru
gdebestkupit.ru gazobeton-p.com.ua
ge0ip.org gemara.com
geile-lelly.eu gepezz.info
geoads.com get-your-social-buttons.info
gfaq.ru getlaid-xxxhookupdirect.com
ghernnqr.skyrock.com getridofstretchmarks.org
goforexvps.com golden-praga.ru
good-mummy.ru good-mummy.ru
googlefeud.com
googlepositions.com googlepositions.com
gorodservis.ru gotwebsite1.com
gratis-sexkontakte.com greatgrace.ru
hd-film.pl growmyfunds.ca
hdseriale.pl growshop.es
heartofbeijing.blogspot.com gsasearchengineranker.xyz
helvetia.com.ua gsasearchenginerankerdiscount.com
hermesreplica.pw gwebtools.com
gyffu.com
hawaiielectriclight.com
healgastro.com
hidefiles.org
himazin.info himazin.info
histock.info
hit-kino.ru
hkladys.com hkladys.com
homedo.fabpage.com hoholikik.club
hotchatdate.com hostingclub.lk
houston-vikings.com hotdl.in
howlongdoestizanidinestayinyoursystem.blogspot.com hoterika.com
hulfingtonpost.com howmuchdoestizanidinecost.blogspot.com
i-hobot.ru howtoclean.club
ideibiznesa2015.ru hscsscotland.com
iframe-toloka.com hut1.ru
infobanks.ru i-midias.net.br
igrovyeavtomaty777.ru
igtools.club
iklysha.ml
imagefinder.site
infokonkurs.ru
infostatsvc.com infostatsvc.com
insomniagamingfestival.com infoupdate.org
insta-add.pro inspiring-desperate.tk
invest-pamm.ru integritylandscapeservices.com
intellektmedia.at
intim-uslugi.info
invitefashion.com invitefashion.com
irkutsk.online-podarki.com io9.com
istizanidineacontrolledsubstance.blogspot.com ispac.org
istmira.ru ispaniya-costa-blanca.ru
itrevolution.cf itservicesthatworkforyou.com
j33x.com ivearchenceinflu.cf
janettabridal.com izismile.com
javcoast.com jennyfire.ru
jo24news.com jobmarket.com.ua
joessmogtestonly.com
johnnyhaley.top johnnyhaley.top
jongose.ninja jongose.ninja
kalb.ru jumptap.com
kaz.kz justdating.online
kazan.xrus.org kaliningrad.zrus.org
keki.info kashubadesign.ru
keywesthideaways.co katadhin.co
kino2018.cc keywordsdoctor.com
kmgamex.cf keywordsuggests.com
koleso24.com.ua kino-ecran.ru
kongruan.com kino-key.info
konteiner24.com kino-rf.ru
koopilka.com kinohit1.ru
kinotorka.ru
kit-opt.ru
kiwi237au.tk
kliksaya.com
knowsitall.info
konoplisemena.com
kopihijau.info
koral.se
kozhniebolezni.com kozhniebolezni.com
krasnodar-avtolombards.ru krafte.ru
kupit-adenu.ru kredit-blog.ru
kursy-ege.ru kskjco.club
kvartiry-remont.ucoz.ru kustanay.kz
kwzf.net laexotic.com
labvis.host.sk lanasshop.ru
lapolis.it landreferat.ru
laptoper.net lankarns.com
larutti.ru leboard.ru
leadn.pl letmacwork.world
lechenie-gemorroya.com letto.by
letmacworkfaster.site li-er.ru
letolove.ru lifeinsurancekb.com
lifehacĸer.com liffa.ru
lignofix.ua linkbux.ru
liran-locks.com linkpulse.com
live-sexcam.tk liquimondo.com
live-sexchat.ru livingcanarias.com
lmrauction.com localmatchbook.com
lottospring.com lol-smurfs.com
lonely-mature.com
lonerangergames.com
lostfilm-online.ru
louboutinbooties.xyz
lyrics.home-task.com
lyrster.com lyrster.com
machicon-ueno.info magicdiet.gq
markjaybeefractal.com mainhp.net
maxxtor.eu manifestation.betteroffers.review
mbiologi.ru manipulyator-peterburg.ru
medosmotr-ufa.ru mebel-alait.ru
meendo-free-traffic.ga meduza-consult.ru
menhealed.net meetlocalchicks.com
microsoftportal.net mega-polis.biz.ua
mielec.pl mesto-x.com
millionare.com mexicotravelnet.com
mindeyegames.com mezaruk.info
mirtorrent.net minet.club
mobilierland.com mini.7zap.com
monsterdivx.tv mister-shop.com
moscow.online-podarki.com moesexy.com
mp3downloadhq.com mole.pluto.ro
musezone.ru monthlywinners.com
music7s.me mosrif.ru
muycerdas.xxx moviemail-online.co.uk
muznachas-service.ru mp3films.ru
muztops.ru muztops.ru
my-bc.ru my-aladin.com
mydownlodablefiles.com mydoctorok.ru
myfreemp3.eu nagdak.ru
myghillie.info
mymobilemoneypages.com
mysex21.com
naj-filmy24.pl
namenectar.com namenectar.com
napalm51.nut.cc networkcheck.xyz
naperehresti.info nevansk.ru
naphukete.ru
narkologiya-orel.ru
nastydollars.com
naturalbreakthroughsresearch.com
naughtyconnect.com
needtosellmyhousefast.com
news-readers.ru
newsperuse.com
newtechspb.ru
nicovideo.jp nicovideo.jp
nina.az nordvpn.com
nomuos.it novodigs.com
nuup.info novosti-hi-tech.ru
oakleyglassesonline.us oakridgemo.com
offer.wpsecurity.website ochistka-stokov.ru
okroshki.ru oecnhs.info
onclickpredictiv.com offside2.5v.pl
onclkads.com okmusic.jp
one-gear.com onemantrip.com
online-podarki.com onlinedomains.ru
openmultipleurl.com onlinetvseries.me
openstat.com onlywoman.org
orion-v.com ontargetseo.us
osb.se11.ru ops.picscout.com
otbelivanie-zubov.com orgasmatrix.com
ownshop.cf outshop.ru
page2rss.com paclitor.com
pandarastore.top pateaswing.com
parajumpersoutlet.online pawli.eu
pc-virus-d0l92j2.pw pc-test.net
pcgroup.com.uy pcgroup.com.uy
pharmacyincity.com pearlisland.ru
pekori.to
photokitchendesign.com
pics-group.com
picsearch.com
pihl.se
pillscheap24h.com pillscheap24h.com
pl-vouchers.com pinup-casino1.ru
plastjulgranar.se pinwallpaper.top
play.leadzupc.com piulatte.cz
playboyfiles.xblog.in pkr1hand.com
pochemychka.net pliks.pl
poems.com.ua pod-muzyku.club
popserve.adscpm.net pogosh.com
porno-home365.com polcin.de
porno-raskazy.ru popcash.net
porno-video-chati.ru porno-dojki.net
pornoslive.net pornofiljmi.com
pornosok.ru pornogad.com
portal-eu.ru pornosee.info
posible.net pornpost.in
positive2b.ru professionaldieselcare.com
preg.marketingvici.com promoheads.com
presleycollectibles.com pronorm.fr
profitwithalex.info prostitutki-astana.org
prohoster.info
prom23.ru
prombudpostach.com.ua
proposal-engine.com
prostitutki-kharkova.org prostitutki-kharkova.org
prostitutki-kiev.org psoriasis-file.trade
puteshestvennik.com qcstrtvt.bloger.index.hr
quickbuck.com quick-seeker.com
quickcashlimited.com qwesa.ru
quit-smoking.ga raavidesigns.com
qwarckoine.com rankscanner.com
rabotaetvse.ru rapidsites.pro
rank-checker.online
rankia.com
ranksays.com
ranksignals.com
rarbg.to rarbg.to
raschtextil.com.ua razyboard.com
razamicroelectronics.com rebuildermedical.com
recipedays.com remorcicomerciale.ro
reklama1.ru restaurantlescampi.com
rentaremotecomputer.com rightenergysolutions.com.au
responsive-test.net rimedia.org
retailwith.com rogervivierforsale.com
rocketchange.ru roma-kukareku.livejournal.com
ruex.org.ua rospromtest.ru
rusexy.xyz ruf777.com
sack.net russkoe-zdorovie.ru
rusvideos.su
s1z.ru
sad-torg.com.ua
sady-urala.ru sady-urala.ru
samara.rosfirm.ru saecsa.co
saneitconsulting.com salutmontreal.com
sarahmilne.top sanyuprojects.com
scanner-jane.top saveindex.xyz
sdsjweb.com scanner-julia.top
seadragonherbery.com scanner-viktor.top
searchinterneat-a.akamaihd.net scrnet.biz.ua
seocdvig.ru searchengineranker.email
servethis.com seccioncontrabajo.com
sexgalleries.top segol.tv
sexreliz.com seo-prof1.xyz
sexsearch.com seoboxes.com
sexvideo-sex.com seogadget.ru
shell-pmr.ru seokicks.de
serdcenebolit.com
sexflirtbook.com
share-buttons-for-free.com
shareyards.com
sherlock.se sherlock.se
shitting.pro shisha-swag.de
shivafurnishings.com shoppingmiracles.co.uk
site-auditor.online sierraapps.com
sitevaluation.org sildenafil-tadalafil.info
sizeplus.work silvermature.net
skachat-besplatno-obrazcy.ru simplepooltips.com
skincrate.net siteexpress.co.il
skutecznetabletkinaporostwlosow.pl sitopreferito.it
skytraf.xyz sixcooler.de
slavia.info slavic-magic.ru
sledstvie-veli.net smartadserver.com
smartshoppymac.com smstraf.ru
snip.tw snjatie-geroinovoy-lomki.ru
snowplanes.com soda.media
snyatie-lomki-v-stacionare.ru solinf.co
social-fun.ru soviet-portal.do.am
solitaire-game.ru star61.de
solution4u.com stockspmb.info
sortthemesitesby.com strana-solnca.ru
sovetogorod.ru stroilka.info
sovinsteel.ru studiokamyk.com.pl
spasswelt.net studworks.org
splendorsearch-a.akamaihd.net sugarkun.com
spravka130.ru sumo.com
staff.prairiesouth.ca
stairliftsarea.com
starpages.net
stathat.com
streamin.to
stroicol.net
studentguide.ru
sunflowerdrawingpaintings.blogspot.com sunflowerdrawingpaintings.blogspot.com
svnuppsalaorebro.se sureone.pro
swinger-mobil.net susanholtphotography.com
szucs.ru taqywu51.soup.io
tacbibirfa.tk taximytishi.ru
takethatad.com tecspb.ru
tech4master.com theporndude.com
telefonsexkostenlos.tk
telemetryverification.net
thebitcoincode.com
thebluffs.com
thedownloadfreeonlinegames.blogspot.com
thegioixekhach.com
thexart.club
time-japan.ru time-japan.ru
timhost.ru titangel-vietnam.com
titan-ads.life tizanidineandcipro.blogspot.com
tizanidine4mgstreetvalue.blogspot.com tizanidinehclsideeffects.blogspot.com
tizanidineformigraines.blogspot.com tizanidinehydrochloride4mgstreetvalue.blogspot.com
tizanidinerecreationaluse.blogspot.com tizanidinestreetvalue.blogspot.com
tongkatmadura.info
topquality.cf
torrent-newgames.com
tracklead.net
tradedeals.biz
trafficmonetize.org
trafficreceiver.club trafficreceiver.club
trailer.cinemaflix.website traffique.net
tri-slona.org tri-slona.org
triumf-realty.ru trk-4.net
trucri.me truemfilelj.gq
tryrating.com tryrating.com
tsan.net tuckermktg.com
turbo-suslik.org tvoystartup.ru
tv-spoty.info twincitiescarservice.com
twu.com.ua twittrading.com
typer.one twodollarshows.com
uchil.net tx41tclega.ru
ultimatesetnewfreeallsoftupgradesystems.pw udayavani.com
uggbootsoutletsale.us
ukrtvir.com.ua
unitygame3d.com
unrealcommander.biz
uptimechecker.com uptimechecker.com
urengoy.pro vbabule.net
urlopener.blogspot.com.au video--production.com
uzporno.mobi videonsk.com
vapsy.com
via-energy-acquistare.com
videochat.bz
videokrik.net
vigrx-original.ru
villacoloniale.com
villakohlanta.nu
vip2ch.com vip2ch.com
visa-china.ru vipps.com.my
viteonlusarezzo.it virus-schutzmasken.de
vladhistory.com vod.com.ua
vladimir.xrus.org vodaodessa.com
voloo.ru vostoktrade.info
voronezh.xrus.org vozbujdenie.com
vseigry.fun vroze.com
wareseeker.com vse-pesni.com
warningwar.ru vykupavto-krasnodar.ru
wcb.su warezaccess.com
webinstantservice.com website-speed-up.site
webix.me whereiskentoday.com
website-speed-checker.site wholesalejerseychinaoutlet.com
welcomeauto.ru witherrom55.eklablog.fr
whosonmyserver.com word-vorlagen.net
winwotgold.pl writingservices17.blogspot.ru
wma-x.com www.kabbalah-red-bracelets.com
worldtraveler.world wygraj-teraz.com
ws.ampower.me wzgyyq.com
www.bookmaker-bets.com x-diesel.org
www.get-free-traffic-now.com x-mix.info
x5market.ru x-true.info
xkaz.org xgames-04.com
xmladserver.com xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
xn----ctbigni3aj4h.xn--p1ai xn--1-8sbcpb0bdm8k6a.xn--p1ai
xn--24-glceagatoq7c2a6ioc.xn--p1ai xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xn--b1agm2d.net
xn--c1acygb.xn--p1ai xn--d1aifoe0a9a.top
xn--d1abj0abs9d.in.ua xn--l1aengat.xn--p1ai
y8games-free.com xxxkaz.org
yachts-cruise.info
yebocasino.co.za yebocasino.co.za
yebocasino.com youbloodyripper.com
yellowstonesafaritours.com yuweng.info
yougotanewdomain.com zaidia.xhost.ro
yourporn.com zambini.ru
yun56.co zastenchivosti.net
zarenica.net zeleznobeton.ru
zebramart.ru zeroredirect11.com
zhacker.net zigarettenonl.canalblog.com
zhorapankratov7.blogspot.com zigarettenonlinekaufen2.drupalgardens.com
zoodrawings.com zkjovpdgxivg.ga
zskdla.site znaniyapolza.ru
zrizvtrnpale.tk
zscaler.net
zscalertwo.net

482
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
0912701309f8ce.com 10bet.com
101lesbian.xyz 2015god.org
11235813.webzdarma.cz 6128786.com
1688.com 83net.jp
3rbseyes.com 883zy.com
3xforum.ro aa8b68101d388c446389283820863176e7.com
4webmasters.com adtiger.tk
5kstore.com affrh2015.com
aaa38852e886ac4af1a3cff9b47cab6272.com alienwheel.es
aasoldes.fr animebox.com.ua
abbanreddy.com apispringsmartne-a.akamaihd.net
actices.com appsecurityr.com
adconscious.com audiofree.ru
adprotect.net autolombard-krasnodar.ru
adtech.us avtocredit-legko.ru
adtology.com bannerswap.com
adultactioncam.com beauby.ru
ahrefs.com best-coupon-offer.com
alienwheels.de best-games.today
anal-acrobats.com bestimagecoollibrary.com
analnoeporno.tv binaryoptionscops.info
anatomiy.com blog-hits.com
apibetweenlinesn-a.akamaihd.net blogsfunky672.weebly.com
appfixing.space bochemit.com.ua
artparquet.ru bonky.biz
as5000.com boo-arts.com
atovh.local-finders.com bookmark4you.com.biz
auto.rusvile.lt boole.onlinewebshop.net
autorn.ru brainxs.us
azadnegar.com bristolhostel.com
azte.ch bristolhotel.com
babyguns.ru businescoop.men
back.dog buyclomidonlaine.com
bankcrediti.ru cablecar.us
battle.net callawaygolfstore.online
bdsmgalls.net carabela.com.do
beachpics.com carapics.com
becuo.com carrauterie.be
begalka.xyz carson.getenjoyment.net
bestmobilityscooterstoday.com cdnnetwok.xyz
betterscooter.com chocoslim.pro
bikini-image.com clickprotects.com
bloggerads.net compliance-alex.top
bosman.pluto.ro compliance-jane.top
brianhenry.co compliance-mary.top
buscarfoto.com cookie-law-enforcement-ee.xyz
buzzzg.men counciltally.com
call-of-duty.info crazyboost.pro
calvet.altervista.org cutalltheshit.com
cardsdumps.com damasarenai.info
carmuffler.net daooda.com
cbozhe.com dayibiao.com
chinacheapelitejerseys.com detmebel.su
cialischmrx.com dichvuvesinhhanoi.com
cialischstgerts.com
club-lukojl.ru
club-samodelkin.ru
cocyq.inwtrade.com
compliance-jessica.top
compliance-julianna.top
cscwtalkto.site
customchocolate.business-for-home.com
daneshetabiat.com
data-mining.tk
deadlinkchecker.com
digital-sale.su
digitalfaq.com digitalfaq.com
dikx.gdn divci-hry.info
dollartree.info dptaughtme.com
domainroam.win dumpsmania24.com
domashneeruporno.com edge.sharethis.com
dotmass.top ege-essay.ru
earncash.com.ua elektronischezi.livejournal.com
egvar.net elektronischezigarettekaufen1.beeplog.com
ekosmetyki.net.pl ero-advertising.com
empis.magix.net erotische-geschichten-xxl.com
eropho.net essay-writing.work
european-torches.ru estibot.com
exlarseva.webblog.es ezigarettekaufen.myblog.de
eyelike.com.ua financehint.eu
ezigarettenkaufen2.dreamwidth.org floridamhca.org
ezigarettenshop1.yolasite.com footbalive.org
fingerprintjs.com forex.osobye.ru
fishtauto.ru forzeronly.com
footballfarrago.com free-fbook-traffic.com
fortevidyoze.net fx-brokers-review.com
fotosfotos.eu
fuckingawesome.com
funponsel.com
g.starmoe.xyz g.starmoe.xyz
gembird.com galeria-zdjec.com
ghernnqr.skyrock.com gamebackyard.com
gheus.altervista.org gameonasia.com
goblacked.com gameplexcity.com
gopixdatabase.com gentamicineyedrops.blogspot.com
gotorussia.com gerhardhealer.com
gowreckdiving.com goodprotein.ru
gratis-sexkontakte.com greendream.com.ua
gribkovye-zabolevaniya.com gsasearchengineranker.xyz
grtyi.com hao123.com
gungamesz.com helvetia.com.ua
hccoder.info hollywoodactress.info
hmmm.cz hoterika.com
hobbyhuren24.net ideawheel.com
hobild.net igithab.com
homemade.gq imadedinner.net
hornymatches.com imgata.com
hostingclub.lk indiasourcemart.in
https-legalrc.biz instakink.com
humanelydrew.com investyb.com
hvd-store.com ipchicken.com
ilikevitaly.com joy-penguin.com
incep.imagine-ex.co jwss.cc
infowarcraft.ru kartiniresto.com
insider.pro keyhantercume.com
instabid.tech kiskinhouse.com
intervsem.ru kosova.de
investzalog.ru kraljeva-sutjeska.com
invivo.hu kupit-adenu.ru
istock-mebel.ru kustanay.kz
jagg.info kw21.org
javxxx18.com labplus.ru
joessmogtestonly.com lacloop.info
jpcycles.com
jus0wil.pp.ua
kochanelli.com
koptims.tiu.ru
krasula.pp.ua
kung-fu-ru.com
ladov.ru
lancheck.net lancheck.net
lazy-z.com landoftracking.com
lockerz.com lapitec.eu
longgreen.info lapolis.it
lookingglassemb.com lawyers.cafe
losangeles-ads.com lego4x4.ru
lyngdalhudterapi.no lider-zhaluzi.kiev.ua
lyrics.home-task.com linkwithin.com
magento-crew.net live-sexchat.ru
managerpak204.weebly.com liveu.infoteka.hu
ljusihus.se
longlifelomilomi.info
lsp-awak-perikanan.com
luckyshop.net.ua
lulea-auktionsverk.se
lyrster.com
manuscript.su
marketland.ml
mature.free-websites.com mature.free-websites.com
mb140.ru meccadumps.net
mini.7zap.com mega-polis.biz.ua
morepoweronmac.trade moesexy.com
music.utrolive.ru mustat.com
naphukete.ru muz-shoes.ru
netpics.org noscrapleftbehind.co
news-speaker.com nuup.info
nhl09.ru nvformula.ru
nmrk.ru nzfilecloud.weebly.com
nomuos.it o.light.d0t.ru
norththeface.store o333o.com
nudepatch.net offtopic.biz
oddamzadarmo.eu olvanto.ru
ok-ua.info onemactrckr.com
omsk.xrus.org online-hit.info
osagonline.ru online-zaymy.ru
outclicks.net ons-add.men
outpersonals.com ororodnik.goodbb.ru
owathemes.com outshop.ru
paradontozanet.ru ovirus.ru
pensplan.com oxotl.com
pewit.pw pc-virus-d0l92j2.pw
picscout.com pensplan4u.com
perfection-pleasure.ru
photo.houseofgaga.ru
pictures-and-images.net pictures-and-images.net
pillscheap24h.com pinkduck.ga
pinsdaddy.com piski.top
pizda.lol plastjulgranar.se
playlott.com podshipniki-nsk.ru
pliks.pl pornhubforum.tk
pooleroadmedicalcentre.co.uk
porno.simple-image.com.ua porno.simple-image.com.ua
portnoff.od.ua pornohd1080.online
potolokelekor.ru pornosexrolik.com
pr-ten.de preparevideosafesystem4unow.space
printie.com pro-okis.ru
qor360.com prom23.ru
raavidesigns.com prostitutki-yaroslavlya.org
reining.lovasszovetseg.hu putevka24.ru
raisedseo.com
ranksays.com
ranksonic.info
rczhan.com
redirectme.net
remont-ustanovka-tehniki.ru remont-ustanovka-tehniki.ru
runstocks.com rewardit.com
rusenvironmental.net rockingclicks.com
saecsa.co roleforum.ru
sarm.tk ryetaw.com
scanner-andrew.top scanner-jane.top
scenarii-1-sentyabrya.uroki.org.ua scanner-willy.top
search.1and1.com scanspyware.net
searchencrypt.com seo-platform.com
seo-prof1.xyz seoboxes.com
sex-tracker.com seogadget.ru
sexobzor.info sexuria.net
sexpartygirls.net sharpchallenge.com
sexvideo-sex.com shemale-sex.net
shama-rc.net shisha-swag.de
shop.acim.org sigmund-freud.co.uk
simple-image.com.ua signoredom.com
sitopreferito.it sindragosa.comxa.com
sklad-24.ru site3.free-share-buttons.com
sky-mine.ru sitiz.club
smartadserver.com skylta.com
smsactivator.ru snjatie-geroinovoy-lomki.ru
smstraf.ru sobecjvuwa.com.ru
snowplanes.com sockshares.tv
spb.afora.ru space2019.top
spymac.net spamnuker.com
staff.prairiesouth.ca spylog.com
stromerrealty.com ssconstruction.co
strongholdsb.ru stackthatbucks.com
summerlinhomes411.info steamoff.net
superstats.com stefanbakosab.se
surffoundation.nl stickers-market.ru
swinger-mobil.net stretchingabuckblog.com
t-bygg.com superlist.biz
taaaak.com
taboola.com
takeflyte.com takeflyte.com
tam-gde-more.ru tako3.com
tehngr.ru testbotprocessor44.com
telsis.com
thebitcoincode.com
thiegs.reco.ws
thruport.com
tilido.com
tizanidine4mgstreetvalue.blogspot.com
tizanidineandhydrocodone.blogspot.com tizanidineandhydrocodone.blogspot.com
tizanidinecapsules.blogspot.com tizanidinevsvalium.blogspot.com
tizanidinehcl4mgisitanarcotic.blogspot.com tk-assortiment.ru
tmearegion26.com tmm-kurs.ru
track-rankings.online toloka.hurtom.com
tripper.de tootoo.to
truck-land.life topsy.com
trumpetedextremes.com track112.site
uamtrk.com tradedeals.biz
urlopener.blogspot.com.au traffic-club.info
varikoz24.com trafficfactory.biz
videochat.tv.br trafficjunky.net
videochat.world truck-rece.life
vladhistory.com try-rx.com
vrotike.ru tsstcorpcddvdwshbbdriverfb.aircus.com
vucms.com twodollarshows.com
watracker.net u17795.netangels.ru
while.cheapwebsitehoster.com uac.net.au
wholinkstome.com unece.org
xingzi-vision.com upupa.net
xn----8sbhefaln6acifdaon5c6f4axh.xn--p1ai uytmaster.ru
xn--24-glceagatoq7c2a6ioc.xn--p1ai vashsvet.com
xn--d1aifoe0a9a.top vbtracker.net
yoopsie.com vduplo.ru
zixizop.net.ru video-chat.love
zoompegs.com videochat.mx
vsesubwaysurfers.com
wikes.20fr.com
works.if.ua
xaylapdiendanang.com
xblognetwork.com
xchangetrak.com
xn----7sbaphztdjeboffeiof6c.xn--p1ai
xnxxandxvideos.com
xz618.com
yaminecraft.ru
yellowads.men
youtuhe.com
zaim-pod-zalog-krasnodar.ru
zawyna.ua
znakomstvaonlain.ru
zrizvtrnpale.tk

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.234 0; 10.1.0.170 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2749 ### Version: V4.2021.12.2750
### Updated: Wed Dec 1 10:03:13 UTC 2021 ### Updated: Wed Dec 1 22:01:29 UTC 2021
### Bad Bot Count: 630 ### Bad Bot Count: 630
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.12.2749 ### Version: V4.2021.12.2750
### Updated: Wed Dec 1 10:03:13 UTC 2021 ### Updated: Wed Dec 1 22:01:29 UTC 2021
### Bad Bot Count: 630 ### Bad Bot Count: 630
################################################### ###################################################
### Version Information ## ### Version Information ##