vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.04.3049 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-04-17 10:09:31 +00:00
parent c31dc11bb5
commit 8686e3406e
46 changed files with 96873 additions and 96873 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3048 #### Version: V4.2022.04.3049
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.04.3048 #### Version: V4.2022.04.3049
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.04.3048 #### Version: V4.2022.04.3049
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11974
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.04.3048 #### Version: V4.2022.04.3049
#### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7092](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11978
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11978
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11978
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11978
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.0.73 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.0.73 0;
127.0.0.1 0; 127.0.0.1 0;

11978
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.0.73 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.0.73 0;
127.0.0.1 0; 127.0.0.1 0;

11978
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.0.73 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.0.73 0;
127.0.0.1 0; 127.0.0.1 0;

11978
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3048 3049

11978
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11978
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.42.199.110 0; 20.231.230.15 0;

11978
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.0.73 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.0.73 0;
127.0.0.1 0; 127.0.0.1 0;

11978
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.0.73 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.0.73 0;
127.0.0.1 0; 127.0.0.1 0;

11978
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.0.73 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.0.73 0;
127.0.0.1 0; 127.0.0.1 0;

11978
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.42.199.110 0; 20.231.230.15 0;

11978
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
104.42.199.110 0; 20.231.230.15 0;

11978
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 1; 10.1.0.73 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Anarchy Deusu
Atomseobot DomainStatsBot
CensysInspect Evil
GrabNet JOC Web Spider
Kenjin Spider Masscan
LinkextractorPro Name Intelligence
Voltron Nessus
Whack NetAnts
Zermelo SISTRIX
dataforseo.com check1.exe

300
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404checker ADmantX
404enemy
80legs
ALittle Client
Abonti Abonti
Aboundex Aboundex
AiHitBot AhrefsBot
Alexibot
AlphaBot AlphaBot
Anarchy Apexoo
Anthill Aspiegel
Asterias Atomseobot
Attach Attach
AwarioRssBot BLEXBot
AwarioSmartBot
BackDoorBot
BackWeb BackWeb
Badass BacklinkCrawler
Barkrowler Bigfoot
Battleztar Bazinga
BetaBot
Black Hole Black Hole
BlackWidow
Blackboard Blackboard
BlowFish Brandwatch
Buddy BuiltBotTough
BuiltWith
BunnySlippers BunnySlippers
BuzzSumo CATExplorador
CCBot
CODE87 CODE87
Calculon Calculon
CensysInspect Cegbfeieh
CheTeam
CherryPicker CherryPicker
Cliqzbot Cliqzbot
Cloud mapping
Cocolyzebot Cocolyzebot
Collector
Copier
CopyRightCheck CopyRightCheck
Copyscape Copyscape
Craftbot Crawling at Home Project
CrazyWebCrawler CrazyWebCrawler
Crescent
CrunchBot
Curious Curious
CyotekWebCopy
DIIbot
DSearch DSearch
DataCha0s DigitalPebble
DatabaseDriverMysqli Dirbuster
Digincore
Discobot
Discoverybot
DnyzBot DnyzBot
DomainAppender DomainAppender
DomainStatsBot DomainCrawler
Dragonfly DomainSigmaCrawler
Drip EirGrabber
ECCP/1.0 Exabot
EMail Siphon
Ecxi
EroCrawler
Extractor Extractor
Extreme Picture Finder EyeNetIE
FHscan
FemtosearchBot FemtosearchBot
Fimap
Firefox/7.0 Firefox/7.0
FlashGet
Foobot Foobot
Freeuploader Freeuploader
FrontPage FrontPage
Fyrebot Fuzz
GT::WWW FyberSpider
Genieo G-i-g-a-b-o-t
GalaxyBot
GermCrawler
GetRight GetRight
Getintent
Gigabot Gigabot
Go!Zilla GridBot
Go-Ahead-Got-It HMView
GrabNet
GrapeFX
HTTP::Lite HTTP::Lite
HaosouSpider HaosouSpider
Harvest
Heritrix Heritrix
Hloader
HonoluluBot
Humanlinks
HybridBot
IDBTE4M
IDBot IDBot
IRLbot IRLbot
Iskanie IndeedBot
JamesBOT Internet Ninja
InternetSeer
JOC Web Spider
Jetty
JikeSpider
Jorgee Jorgee
JustView JustView
Kenjin Spider
Keyword Density Keyword Density
Kinza
LWP::Simple LWP::Simple
Lanshanbot Leap
LibWeb LeechGet
Libwhisker LieBaoFast
LinkScan
LinkWalker
Linkbot
LinkextractorPro
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
MFC_Tear_Sample Lipperhey
Lipperhey Spider
MIDown tool MIDown tool
MTRobot MQQBrowser
Magnet MSFrontPage
Majestic12 Mag-Net
MarkMonitor Majestic SEO
Masscan Majestic-SEO
Mediatoolkitbot MarkWatch
MegaIndex.ru Mass Downloader
Metauri Meanpathbot
MicroMessenger
Microsoft Data Access
Minefield
Mister PiX
Mojeek
MolokaiBot MolokaiBot
Mozlila
Mr.4x3 Mr.4x3
Musobot
NICErsPRO
NPbot NPbot
Name Intelligence
NearSite
Needle Needle
NetMechanic Net Vampire
NetZIP
Netcraft
Netvibes Netvibes
NextGenSearchBot
Nibbler
Nikto
NimbleCrawler NimbleCrawler
Nimbostratus Nimbostratus
Nmap Nmap
Nuclei Not
Nutch OnCrawl
Octopus OpenLinkProfiler
Offline Navigator OpenVAS
Openfind Openfind
Openvas
OrangeBot OrangeBot
OrangeSpider
OutclicksBot OutclicksBot
OutfoxBot OutfoxBot
PHPCrawl PHPCrawl
POE-Component-Client-HTTP POE-Component-Client-HTTP
PageAnalyzer Page Analyzer
PageThing.com Papa Foto
Pavuk Petalbot
Pi-Monster Picscout
Picsearch Picsearch
PictureFinder Piepmatz
Pimonster PleaseCrawl
Pixray PyCurl
ProPowerBot QueryN Metasearch
ProWebWalker
RankActive
RankActiveLinkBot
Rankivabot Rankivabot
RankurBot RankurBot
Re-re Re-re
ReGet ReGet
RealDownload RebelMouse
RedesScrapy RocketCrawler
Rogerbot
SEOkicks
SEOlyticsCrawler
SEOprofiler
SEOstats SEOstats
SMTBot SalesIntelligent
Scanbot Screaming
ScreenerBot ScrepyBot
SearchmetricsBot SemanticJuice
Seomoz Seomoz
Site Sucker Shodan
SiteLockSpider SiteExplorer
SiteSucker SiteSnagger
SlySearch SlySearch
Snake Snake
SocialRankIOBot Snoopy
Sociscraper
Sogou web spider
Sottopop Sottopop
SpaceBison SpaceBison
Spammen Spanner
SpankBot
Spbot Spbot
SputnikBot Surfbot
Sqlmap
SuperBot
Swiftbot Swiftbot
T8Abot T0PHackTeam
TeleportPro
Telesphoreo Telesphoreo
The Intraformant Telesphorep
Thumbor
TightTwatBot TightTwatBot
Toata Titan
Toweyabot
Trendiction Trendiction
Trendictionbot Trendictionbot
TurnitinBot Turnitin
TwengaBot TwengaBot
Typhoeus Upflow
URLy.Warning V-BOT
UnisterBot
VCI
Vagabondo
VidibleScraper
Virusdie
VoidEYE VoidEYE
Voil
Voltron Voltron
WEBDAV WASALive-Bot
WWW-Collector-E WBSearchBot
WISENutbot
WWW-Mechanize
WWW::Mechanize WWW::Mechanize
Wallpapers WWWOFFLE
WallpapersHD WallpapersHD
Web Auto Web Auto
Web Fuck Web Fuck
Web Pix Web Sauger
Web Sucker WebAuto
WebBandit WebCollage
WebCopier
WebEnhancer
WebFuck WebFuck
WebImageCollector WebGo IS
WebPix
WebStripper
WebSucker WebSucker
WebWhacker
WebZIP WebZIP
Webalta
WebmasterWorldForumBot WebmasterWorldForumBot
WebsiteQuester Webshag
Whacker Whacker
Whatweb
Widow Widow
Wonderbot WinHTTrack
Woobot WiseGuys Robot
Zade Wotbox
Zauba Xaldon_WebSpider
YoudaoBot
Zermelo Zermelo
Zitebot Zeus
ZmEu ZmEu
ZoominfoBot ZoominfoBot
ZyBorg ZyBorg
arquivo.pt adscanner
check1.exe archive.org_bot
cognitiveseo arquivo-web-crawler
crawler.feedback backlink-check
crawler4j cah.io.community
dataforseo.com crawl.sogou.com
domainsproject.org demandbase-bot
gopher
heritrix heritrix
lwp-trivial instabid
internetVista monitor
iubenda-radar
mediawords
netEstate NE Crawler netEstate NE Crawler
oBot
page scorer page scorer
pcBrowser ripz
probe-image-size satoristudio.net
s1z.ru
scalaj-http
scan.lol scan.lol
seoscanners seoscanners
seostar
serpstatbot serpstatbot
sexsearcher
sitechecker.pro
sogouspider sogouspider
sp_auditbot spyfu
trendiction.com trendiction.com
voyagerx.com voyagerx.com
webmeup-crawler webprosbot
webpros.com
x22Mozilla x22Mozilla
xpymep1.exe xpymep1.exe
zauba.io zauba.io
zgrab

318
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
404enemy
ASPSeek ASPSeek
Abonti AfD-Verbotsverfahren
AiHitBot
Aipbot Aipbot
AllSubmitter Alexibot
Alligator
Anarchy
Anarchy99 Anarchy99
Anthill Apexoo
Aspiegel
Attach Attach
BBBike AwarioRssBot
BDFetch BackStreet
BackDoorBot
Backlink-Ceck Backlink-Ceck
Badass
Bandit
Barkrowler Barkrowler
BlowFish BatchFTP
Boardreader Black Hole
Bolt Bolt
Brandwatch
Buddy
BuiltBotTough BuiltBotTough
BuiltWith
Bullseye
CATExplorador
CCBot CCBot
CODE87
CSHttp CSHttp
Calculon
CazoodleBot CazoodleBot
Cegbfeieh CensysInspect
CheTeam
CheeseBot
ChinaClaw ChinaClaw
Chlooe Citoid
Claritybot
Cliqzbot Cliqzbot
Cocolyzebot Cloud mapping
Collector Cogentbot
Craftbot CopyRightCheck
CrazyWebCrawler Cosmos
CrunchBot Crawling at Home Project
Crescent
CyotekWebCopy
DBLBot
DIIbot DIIbot
DSearch DTS Agent
DataCha0s
Deusu
Digincore Digincore
Disco Disco
Discobot Dispatch
DittoSpyder
DnBCrawler-Analytics DnBCrawler-Analytics
DomCopBot DomCopBot
DomainAppender DomainAppender
DomainCrawler
DomainStatsBot DomainStatsBot
Domains Project
Dotbot Dotbot
Download Wonder EMail Siphon
Drip
Ebingbong
Ecxi Ecxi
EirGrabber EirGrabber
Evil Exabot
ExtLinksBot Extractor
Fyrebot EyeNetIE
Ezooms
FemtosearchBot
Foobot
FrontPage
FyberSpider
G-i-g-a-b-o-t G-i-g-a-b-o-t
GT::WWW
GalaxyBot
Genieo Genieo
GetWeb GermCrawler
GoZilla GetRight
GrabNet Go-Ahead-Got-It
Grabber Grabber
GrapeFX GrapeFX
HEADMasterSEO GridBot
HMView HTMLparser
HTTP::Lite
HTTrack HTTrack
Havij Havij
Hloader Humanlinks
HonoluluBot
IDBot IDBot
IRLbot
Image Fetch Image Fetch
Indy Library InterGET
InfoNaviRobot
Internet Ninja
InternetSeer InternetSeer
Iskanie Iria
IstellaBot IstellaBot
JOC Web Spider JamesBOT
Jbrofuzz
JennyBot JennyBot
JetCar Joomla
Jetty
JikeSpider
Jorgee Jorgee
Kenjin Spider
Keybot Translation-Search-Machine
Keyword Density
Kinza Kinza
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot Lanshanbot
Larbin LibWeb
LeechGet Libwhisker
Lightspeedsystems Likse
LinkScan LinkScan
Linkbot Linkbot
Litemage_walker LinkextractorPro
Lmspider LinkpadBot
Lipperhey Spider
Ltx71 Ltx71
MIDown tool MFC_Tear_Sample
MIIxpc MQQBrowser
Mag-Net MSIECrawler
Mail.RU_Bot Majestic SEO
Majestic-SEO
Majestic12 Majestic12
MarkWatch MarkMonitor
MauiBot Masscan
MicroMessenger Mata Hari
Microsoft Data Access Metauri
Microsoft URL Control Microsoft URL Control
Minefield Minefield
Mister PiX Mojeek
Mojolicious Morfeus Fucking Scanner
MolokaiBot
Mozlila Mozlila
Mr.4x3
Musobot
NICErsPRO NICErsPRO
NPbot
Name Intelligence Name Intelligence
Nameprotect Navroad
Net Vampire NetAnts
NetLyzer NetMechanic
Netcraft NetSpider
NetZIP
Netvibes
NextGenSearchBot NextGenSearchBot
NimbleCrawler Nibbler
Nimbostratus
Ninja Ninja
Octopus Octopus
Offline Explorer Offline Explorer
OpenVAS
Openfind Openfind
OrangeSpider Openvas
PHPCrawl OrangeBot
POE-Component-Client-HTTP OutclicksBot
PageAnalyzer PECL::HTTP
PageGrabber Page Analyzer
PageScorer Papa Foto
Pandalytics Pi-Monster
Pavuk PictureFinder
Petalbot Pixray
Picscout QueryN Metasearch
Picsearch Quick-Crawler
Pimonster RSSingBot
PleaseCrawl RankActiveLinkBot
Probethenet
Pump
RankActive
RankingBot RankingBot
RankurBot RankingBot2
Re-re
ReGet ReGet
RealDownload RealDownload
Reaper Reaper
RepoMonkey RebelMouse
Ripper Recorder
RocketCrawler
SEOkicks SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOlyticsCrawler SEOlyticsCrawler
SISTRIX SISTRIX
SMTBot
ScanAlert ScanAlert
ScoutJet Scanbot
Scrapy
Screaming
ScreenerBot
SearchmetricsBot SearchmetricsBot
Seekport
SemanticJuice
Semrush Semrush
Seomoz
Shodan Shodan
SiteCheckerBotCrawler Siphon
SiteExplorer Site Sucker
SiteLockSpider SiteLockSpider
SlySearch SiteSucker
Sitevigil
SmartDownload
Snapbot Snapbot
Sottopop Sosospider
SpaceBison Spanner
Spammen Sqlworm
SputnikBot Sqworm
Stripper Stripper
Sucker Sucker
Sucuri Sucuri
Suzuran SuperHTTP
Swiftbot Surfbot
T0PHackTeam T0PHackTeam
T8Abot T8Abot
TeleportPro Teleport
The Intraformant Thumbor
TheNomad TightTwatBot
Tracemyfile Tracemyfile
Trendiction Turingos
Turnitin
TurnitinBot
TwengaBot
URLy Warning URLy Warning
UnisterBot V-BOT
VB Project
VCI VCI
VoidEYE Vacuum
VelenPublicWebCrawler
VeriCiteCrawler
Virusdie
Voltron Voltron
WASALive-Bot WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WWW-Mechanize WWWOFFLE
WWW::Mechanize
Wallpapers Wallpapers
WallpapersHD
Web Collage
Web Enhancer
Web Fetch
Web Fuck Web Fuck
Web Pix Web Pix
Web Sauger Web Sucker
WebAuto WebCopier
WebFetch
WebFuck
WebReaper
WebSauger WebSauger
WebStripper
WebSucker WebSucker
WebWhacker WebmasterWorldForumBot
WebsiteExtractor Webshag
WebsiteQuester
Whack Whack
Who.is Bot Whacker
Wonderbot Widow
WinHTTrack
Wotbox
Xaldon WebSpider
Xaldon_WebSpider
Zade Zade
ZoomBot ZmEu
ZoominfoBot arquivo-web-crawler
archive.org_bot
arquivo.pt arquivo.pt
check1.exe autoemailspider
clark-crawler cah.io.community
coccocbot cognitiveseo
com.plumanalytics crawl.sogou.com
crawler.feedback crawler.feedback
dataforseo.com crawler4j
demandbase-bot heritrix
gopher internetVista monitor
instabid
ips-agent
lwp-request
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot
mediawords mediawords
muhstik-scan oBot
netEstate NE Crawler page scorer
ripz pcBrowser
scalaj-http polaris version
scan.lol seocompany.store
seobility seoscanners
seostar serpstatbot
sexsearcher
siteripz siteripz
sp_auditbot trendiction.com
spyfu
trendiction.de trendiction.de
webgains-bot
webmeup-crawler webmeup-crawler
webpros.com zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
03p.info alyeskaresort.com
aktobe.xkaz.org binomo.com
callmd5map.com get-free-traffic-now.com
cocyq.inwtrade.com javidol.site
glastecfilms.com.my ledpolice.ru
ilmexico.com marcowebonyodziezowe.pl
myshopmatemac.com oil-td.ru
progress-upakovka.ru paramountmarble.co.uk
registry-clean-up.net stuffpride.com
vip-dom.in veselokloun.ru

912
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00webcams.com 123kuma.com
100searchengines.com 1j7740kd.website
108shot.com 34.gs
1bet.com 45en.ru
8jn.dating-app.ru 6128786.com
9med.net 777-club.ru
a1security.com.ua aad1f4acb0a373420d9b0c4202d38d94fa.com
aa14ab57a3339c4064bd9ae6fad7495b5f.com abclauncher.com
aa8b68101d388c446389283820863176e7.com abovetherivernc.com
abcguru.xyz addbin.men
above.com adexprts.com
activecampaign.dreamhosters.com adloads.net
actulite.com
add-add.men
addray.pro
adidas.frwebs.fr
adsloads.com
adsref.men adsref.men
adtiger.tk
adtology.com adtology.com
advokat-grodno.by adverster.com
alekseevec.ru affrh2015.com
alert-jdh.xyz afslankpillen2017nl.eu
all4invest.info agahinameh.com
aghanyna.com
alert.scansafe.net
alif-ba-ta.com
allesohnegirls.net allesohnegirls.net
allfinweb.com alphacarolinas.com
alphaforum.ru alpinism.ru
alphavisions.net
amazon-adsystem.com
amazon-seo-service.com
ameblo.top ameblo.top
analytics-ads.xyz amt-k.ru
anaseracresar.tk anime.dougasouko.com
android-style.com animenime.ru
apibatbrowsecom-a.akamaihd.net anjalika.co.in
apimegabrowsebiz-a.akamaihd.net apparel-offer.com
app-ready.xyz arcteryxstore.online
arkartex.ru articlesdirectoryme.info
arvut.org
asdfg.pro
asiengirls.net
asseenontvonline.ru
atovh.local-finders.com atovh.local-finders.com
autoloans.com autovideobroadcast.com
autoseo-traffic.com autowebmarket.com.ua
ayodhya.co avkzarabotok.com
bangla-chat-uk-paralud.ga avtovolop.ru
bannertesting.com bankcrediti.ru
bazaronline24.ru banki76.ru
beauty-b0x.pl bankiem.pl
bedandbreakfast.com bannerads.de
bannerpower.com
bannerspace.com
beachtoday.ru
bedcapdealers.com
begalka.xyz begalka.xyz
bestbrainprod.win best-coupon-offer.com
beta.hotkeys.com beztuberkuleza.ru
betteroffers.review bikini-image.com
bidr.trellian.com bio-japan.net
bif-ru.info blog.f00kclan.de
bigcareer.info blog.koreadaily.com
bioscorp.ru bloggers.nl
bizru.info
blackhatworth.com
blogqpot.com blogqpot.com
bonzibuddi.com bluerobot.info
bookmark4you.com.biz bmsco.co
brainzpod.win brainxs.us
brimstonehillfortress.org brandbucket.com
brillianty.info
bristolhotel.com
bronzeaid-a.akamaihd.net bronzeaid-a.akamaihd.net
bscodecs.com buigas.00it.com
bugshoot.cn bumascloset.com
callawaygolfstore.online burn-fat.ga
callmd5map.com buron.pw
camdolls.com buy-meds24.com
candycrushshop.com buypanicdisorderpill.com
carsplate.com bycontext.com
ccjp.eu call-of-duty.info
cenokos.ru callawaygolfoutlet.online
chatroulette.video candypeople.se
cheapjerseysbizwholesale.us candytiner.com
chelnytruck.ru canrole.com
christianlouboutinoutlet.win canzoni.ru
christianlouboutinsaleoutletonline.us carina-sy.de
cibpenonptib.flu.cc carstrends2015.com
cityadspix.com cashonads.com
clips.ua.ac.be cheapmoncler.xyz
club-samodelkin.ru checktext.org
co.lumb.co choice-credit.ru
compliance-andrew.top cineacademy.ru
compliance-elena.top clean-virus-mac.com
compliance-george.top clickso.com
clicksor.net
coccoc.com
coinsspb.com
collegerentals.com
compliance-irvin.xyz compliance-irvin.xyz
compliance-ivan.top compliance-julianna.top
compliance-walt.top conocer-sanabria.com
compucelunlock.net constantaservice.net
confib.ifmo.ru construmac.com.mx
copblock.org cookie-law-enforcement-aa.xyz
cordstrap.cc counterbot.com
corta.co
credit-respect.ru credit-respect.ru
custom-product-labels.com customchocolate.business-for-home.com
customerguru.in customerguru.in
cymbaltaandtizanidine.blogspot.com customerpromos-a.akamaihd.net
cutt.us
cxpromote.com
cyclobenzaprinevstizanidine.blogspot.com
dailyrank.net dailyrank.net
darkages.info darmebel.com.ua
darodar.com dating-time-now.com
decenttools.com datsun-do.com
decorationspcs.com davebestdeals.com
decorazilla.com ddospanel.com
dekoration.us
delfin-aqua.com.ua
dentfidemountpreach.tk
deplim.com deplim.com
design-lands.ru descargar-musica-gratis.net
dev78.dioniqlabb.se dgfitness.co
digital-sale.su dieta-personalna.pl
discovertreasure-a.akamaihd.net div.as
dividendo.ru dokfilms.net
docstoc.com domainsphoto.com
dotmass.top downloadmefiranaratb1972.xpg.com.br
drugspowerstore.com dvd-famille.com
drunkmoms.net e-kwiaciarz.pl
ebonyporn.site e2click.com
edge.sharethis.com eaplay.ru
editors.choice6912650.hulfingtonpost.com easyukraine.com
ekspresihati.info ecommerce-seo.org
elektrischezigarettekaufen2.cowblog.fr ekn-art.se
elektroniksigaraankara.info elektrischezigaretten1.blogse.nl
elektrozigarettekaufen1.devhub.com elstal.com.pl
elkacentr.ru emediate.eu
elmifarhangi.com empis.magix.net
emailaccountlogin.co energydiet-info.ru
emergencyneeds.org englate.com
empathica.com
enskedesquashclub.se
esopini.com esopini.com
etenininrade.ga espaceinventoristes.com
euromasterclass.ru este-line.com.ua
evehemming.blogspot.com.au european-torches.ru
evidencecleanergold.com expertblog.info
exct.net extremal-blog.com
exdocsfiles.com eywords-monitoring-your-success.com
exportshop.us f07.de
extstat.com fdzone.org
eyes-on-you.ga fikasound.tk
eyessurgery.ru filerockstar298.weebly.com
ezigarettenkaufen1.deviantart.com filkhbr.com
ezigarettenshop2.postbit.com
fable.in.ua
face.hostingx.eu
farm26.ru
fealq.com
fetishinside.com
fetroshok.ru
filmci.pro
filmetricsasia.com filmetricsasia.com
financehint.eu findanysex.com
findclan.org finteks.ru
flagcounter.me flprog.com
formseo.com forsex.info
fotoxxxru.com fototravel.eu
free-share-buttons.top free-deals.faith
froggytube.com free-fbook-traffic.com
fsakhalin.ru freelotto.com
ftns.ru fusoradio.info
fullfileaccess.com futbolkisales.ru
funnypica.com g33.org
fxgallery.com games.kolossale.ru
g.starmoe.xyz gamewrath.com
gagrasector.ru gaming-journal.com
galeon.com gaypornmovie.net
gdzkurokam.ru gdzkurokam.ru
gelezki.com ge0ip.org
generic-pills-online.com gearcraft.us
get-free-social-traffic.com geileweiber.tk
getmiro.com getaclueamerica.com
getlaid-xxxhookupdirect.com
getprismatic.com
gettpromos.com gettpromos.com
ghazel.ru
gheus.altervista.org
gigixo.com gigixo.com
goo.ne.jp gk-atlant.info
goodprotein.ru gkvector.ru
goosefishpost.bid global-ics.co.za
gorabagrata.ru gogalleryawesome.com
gox.com.ua gopixdatabase.com
gratuitxblcodes.com gratuitbaise.com
greatzip.com
gsasearchengineranker.xyz gsasearchengineranker.xyz
guiadeserraazul.com gta-club.ru
gwagka.com guardlink.org
hamacapty.com gungamesz.com
hamilton.ca hackers-crackers.tk
hanwei.us handicapbathtubarea.com
hanink.biz.ly
hannasolution.ru
hdapp1008-a.akamaihd.net
hdpixion.com
heartofpayne.xyz heartofpayne.xyz
hesteel.pl hesteel.pl
hetmanship.xyz histock.info
hikesearch.net hit-men.men
hobild.net hitsbox.info
hop.clickbank.net hkdiiohi.skyrock.com
hotkeys.com host-protection.com
hoverboard360.es hostnow.men
humbmosquina.tk hotblognetwork.com
idegenvezeto.eu hotsocialz.com
ihc2015.info hunterboots.online
ilte.info husky-shop.cz
immobiliaremassaro.com hut1.ru
importchinacoach-teach.com ideibiznesa2015.ru
impresagaia.it iinstalll-fii1leis.jus0wil.pp.ua
inboundlinks.win infobanks.ru
infodocsportal.com interferencer.ru
insta-add.pro irkutsk.zrus.org
ipchicken.com it-max.com.ua
ipostroika.ru japfm.com
iqoption.com javcoast.com
iqs.biz.ua javstock.com
istizanidineanopiate.blogspot.com jbl-charge.info
istmira.ru jerseychinabizwholesale.us
istock-mebel.ru
istripper.com
iwanttodeliver.com
janavibekken.no
jav-fetish.site
jeremyeaton.co
johnnyhaley.top
jose.mulinohouse.co jose.mulinohouse.co
kalb.ru karachev-city.ru
kareliatobacco.ru ketoanhanoi.info
kazinogames.lv keywordsking.com
knigonosha.net kino-fun.ru
konica.kz kinobest.pl
konturkrasoty.ru kladrus.ru
kostenlos-sexvideos.com komp-pomosch.ru
kostenloser-sex.com kretpicf.pw
kotaku.com kuzinsp.ru
kriokomora.info
kursy-ege.ru
kvartira-sutochno.com
laapp.com
labafydjxa.su
lacloop.info
lakomka.com.ua lakomka.com.ua
laudit.ru ldrtrack.com
law-check-seven.xyz lenpipet.ru
leboard.ru linksharingt.com
lecbter-relationships.ga livefixer.com
lechtaczka.net liveshoppersmac.com
legalrc.biz lockerz.com
lesbian.xyz lordzfilmz.me
letslowbefast.today lsp-awak-perikanan.com
lflash.ru m.b00kmarks.com
limads.men macdamaged.tech
littleberry.ru
live-cam6.info
lol-smurfs.com
lombardfinder.ru
love-baby.cz
lyngdalhudterapi.no
machicon-akihabara.info machicon-akihabara.info
macnewtech.com macrotek.ru
magicalfind-a.akamaihd.net mainlinehobby.net
manuscript.su makis.nu
marketingvici.com manifestation.betteroffers.review
master-muznachas.ru manifestyourmillion.com
mansparskats.com
mapquestz.us
marcogrup.com
matsdale.com
maximpartnerspr.com
mcadamssupplyco.com mcadamssupplyco.com
me-ke.com mcnamaratech.com
medicine-4u.org meddesk.ru
medkletki.ru medispainstitute.com.au
meendo-free-traffic.ga meetlocalchicks.com
mega-bony-2017.pl mejoresfotos.eu
megavolt.net.ua meltwater.com
mensandals.xyz mesbuta.info
metaxalonevstizanidine.blogspot.com militarysale.pro
mexicotravelnet.com minet.club
michaelkorsoutletstore.net mlf.hordo.win
mmofreegames.online mncrftpcs.com
mobsfun.net mnogabukaff.net
modabutik.ru mobot.site
moesonce.com mobplayer.net
moivestiy.biz moesen-ficken.com
monclerboots.xyz monclervests.xyz
monthlywinners.com money-for-placing-articles.com
mossmesi.com monitorwebsitespeed.com
moxo.com moomi-daeri.com
mrbojikobi4.biz morepoweronmac.trade
mrlmedia.net mozello.ru
msfsaar.de mrsdalloways.com
mulberryoutletonlineeu.com muizre.ru
mundoaberrante.com mundoaberrante.com
myblogregistercm.tk music.utrolive.ru
mycouponizemac.com mybackgroundlandscape.blogspot.com
mydoctorok.ru mylida.org
mydownloadengine.com
myfreemp3.eu
myftpupload.com
myplaycity.com
mzdish.site
nac-bearings.ru
nagdak.ru nagdak.ru
narkologiya-peterburg.ru nalogovyy-kodeks.ru
nbsproject.ru namecrumilchlet.tk
netcheckcdn.xyz narco24.me
nfvsz.com naturalbreakthroughsresearch.com
ngps1.ru naturalshair.site
nippon-bearings.ru negociosdasha.com
nomuos.it negral.pluto.ro
november-lax.com new-apps.ru
nudo.ca nfljerseyscheapbiz.us
nvssf.com noclegonline.info
o-o-8-o-o.com nofreezingmac.click
obuv-kupit.ru nosecret.com.ua
ochistka-stokov.ru o333o.com
offer.wpsecurity.website obsessionphrases.com
omsk.xrus.org oktube.ru
onetravelguides.com onescreen.cc
online-hit.info openfrost.com
orenburg-gsm.ru ops.picscout.com
otbelivanie-zubov.com optibuymac.com
outclicks.net optom-deshevo.ru
oynat.info oralsexfilme.net
ozas.net orion-code-access.net
pacificair.com orsonet.ru
paidonlinesites.com ozoz.it
paccohichetoti.ml
paighambot.com paighambot.com
palvira.com.ua paleohub.info
panchro.xyz pay2me.pl
parlament.biz paydayloanslocal.com
partner-cdn.men
partnerads.men
partybunny.ru
pcimforum.com
pdfprof.com pdfprof.com
pdns.cz perfection-pleasure.ru
perso.wanadoo.es perfectplanned.com
pic-re.blogspot.com pictures-and-images.net
piccdata.com
piccshare.com
pijoto.net pijoto.net
pistonclasico.com pinwallpaper.top
playmsn.com piratecams.com
porn-w.org piski.top
pornoplen.com pk-pomosch.ru
pornosmola.info pl-top.pl
pornotubexxx.name placid-rounded-coast.glitch.me
plastgranar.nu
playlott.com
pliks.pl
ploenjitmedia.azurewebsites.net
podshipniki-nsk.ru
poisk-zakona.ru
pokerniydom.ru
pons-presse.com
porngalleries.top
pornodojd.ru
pornosee.info
pornosemki.info
pornosok.ru
portal-eu.ru
portside.xyz portside.xyz
preconnubial.usuby.site pr-ten.de
privatov-zapisi.ru privacylocationforloc.com
prizefestival.mobi
prlog.ru
prodess.ru prodess.ru
produkto.net proekt-mos.ru
profit-opportunity.com proprostatit.com
psvita.ru prostitutki-astana.org
pufip.com prostitutki-kiev.org
puteshestvennik.com pst2017.onlinewebshop.net
qexyfu.bugs3.com pulse33.ru
quickchange.cc pulseonclick.com
quit-smoking.ga pussysaga.com
r-e-f-e-r-e-r.com pxhdwsm.com
rada.ru qor360.com
registratciya-v-moskve.ru qpypcx.com
registry-cleaner.net qwertty.net
registrysweeper.com randalljhoward.com
remedyotc.com rapevideosmovies.com
remont-mobile-phones.ru redirectingat.com
renhacklids.tk registrationdomainsite.com
reward-survey.net reklama-i-rabota.ru
rewardpoll.com replica-watch.ru
richinvestmonitor.com replicaclub.ru
rock-cafe.info rethinkwasteni.info
roznica.com.ua rfd-split.hr
runofilms.ru rialp.getenjoyment.net
romhacking.ru
rospromtest.ru
rossanasaavedra.net
rubanners.com
ruspdd.com
russintv.fr russintv.fr
rvi.biz rybalka-opt.ru
samara.rosfirm.ru sa-rewards.co.za
saugatuck.com sack.net
scanner-george.top sajatvelemeny.com
scanner-jess.top sammyweaver.com
schlampen-treffen.com saneyes.com
score-ads.men satoristudio.net
sdsjweb.com scanner-mary.top
search-goo.com se.bnt-team.com
seminarykansas904.ml seasaltwithfood.com
seocdvig.ru seeingmeerkat.com
seocheckupx.com selectads.men
seofirmreviewsus.info seoexperimenty.ru
seonetwizard.com seorankinglinks.xyz
serviporno.com serialsway.ucoz.ru
sex-tracker.com sexblog.pw
sex-znakomstva.online sexobzor.info
sexfreepornoxxx.com sextracker.com
sexy-screen-savers.com
share-buttons-for-free.com
shikiso.info shikiso.info
shopfishing.com.ua sigmund-freud.co.uk
shoppingmiracles.co.uk sim-service.net
shtaketniki.ru simul.co
sinel.info sinel.info
sklad-24.ru sirpornogratis.xxx
snjack.info sitopreferito.it
social-s-iii.xyz smartbalanceworld.com
smartshoppymac.com
snomer1.ru
soblaznu.net
soc-proof.su
social-s-hhh.xyz
social-vestnik.ru
socialmadesimple.com socialmadesimple.com
soda.media sogimlecal.tk
sonnikforme.ru solnplast.ru
sortthemesitesby.com sosdepotdebilan.com
sportbetfair.com space2019.top
spy-sts.com speeddream.xyz
steebook.com spylog.com
stats-public.grammarly.io
staynplay.net
steelmaster.lv
stillmiracle.com
stockspmb.info
stopnarco.ru stopnarco.ru
strangeduckfilms.com strana-krasoty.ru
streetfire.net strfls.com
stuff-about-money.com stroiminsk.com
sudexpert66.ru stroyhelp-dv.ru
suhanpacktech.com
summerlinhomes411.info summerlinhomes411.info
superinterstitial.com superinterstitial.com
taihouse.ru svetodiodoff.ru
t-bygg.com
takethatad.com takethatad.com
taxi-v-eisk.ru teenforporn.com
teastory.co teenfuck.tv
technopellet.gr tehngr.ru
tecspb.ru telemetryverification.net
templates.franklinfire.co tengohydar.tk
tfxiq.com tetracsaudi.com
thebitcoincode.com thetattoohut.com
thecoral.com.br thiegs.reco.ws
thepokertimer.com tizanidine6mg.blogspot.com
theprofitsmaker.net tizanidinecapsules.blogspot.com
thetoiletpaper.com tizanidinehcl4mgtabinfo.blogspot.com
timetorelax.biz todohr.com
titangel-vietnam.com top10-online-games.com
tizanidine4mg.blogspot.com topflownews.com
tizanidine4mguses.blogspot.com track-rankings.online
tizanidineinteractions.blogspot.com trafaret74.ru
tizanidinepills.blogspot.com trafficjunky.com
tizanidinesideeffectsweightloss.blogspot.com trafficjunky.net
tizanidinestreetprice.blogspot.com trafficreceiver.club
tizanidinewithdrawal.blogspot.com trailer.cinemaflix.website
tongkatmadura.info trichizobswiv.agddns.net
tonivedu.it twojebook.pl
topdownloads.ru tyumen.xrus.org
topquality.cf uchebavchehii.ru
toyota.7zap.com
track2.shop
traffic-club.info
traffic2cash.org
treasuretrack-a.akamaihd.net
triumf-realty.ru
tube8.com
tupper-shop.ru
ucoz.ru ucoz.ru
ugogo.info
ukrtextbook.com
ultimateclassicrock.com
umekana.ru umekana.ru
unimodemhalfduplefw.pen.io upproar.com
uptimechecker.com uptimechecker.com
url-img.link urlcut.ru
utrolive.ru urll.eu
vashsvet.com userequip.com
vegan-foods.us uzporno.mobi
vegascosmetics.ru vadimkravtcov.ru
video-chat.love valmetrundan.se
videochat.tv.br valuado.com
vancleefreplica.pw
vclicks.net
veles.shop
video-chat.in
video-production.com
videos-for-your-business.com
videtubs.pl videtubs.pl
vip-dom.in vip2ch.com
vipms.ru voluumtrk.com
vitanail.ru vzglyadriv.kg
vostoktrade.info wallpaperaccess.com
vremya.eu wallpapersist.com
vsesubwaysurfers.com watchdogs-2.ru
waycash.net waterefficiency.co
website-datenbank.de webadvance.club
weburlopener.com weblo.com
weightatraining.com webmasterhome.cn
wemarketing.se websiteexplorer.info
westsextube.com wedding-salon.net
westvilletowingservices.co.za wemedinc.com
whereverdesperate.gq wesharepics.com
wholesalejerseys-cheapest.com westen-z.life
wholinkstome.com whereiskentoday.com
winx-play.ru wildcattube.com
woman-tampon.ru wladimirpayen.com
www.ehscloud.cn wleuaprpxuvr.ga
www.labves.ru word-vorlagen.net
www.pinnacle-bets.com wovis.site
x-diesel.org x-lime.com
x-mix.info x-rates.ru
xaylapdiendanang.com x-stars.ru
xdoza.com xmlinde.com
xerox-douglas.cf xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai
xev.ru xn----7sbifcamovvfggw9d.xn--p1ai
xn----7sbaphztdjeboffeiof6c.xn--p1ai xn----btbdvdh4aafrfciljm6k.xn--p1ai
xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai xn--80ajbshivpvn2i.xn--p1ai
xn--80aaajbdbddwj2alwjieei2afr3v.xn--p1ai xn--h1ahbi.com.ua
xn--90acenikpebbdd4f6d.xn--p1ai xn--q1a.xn--b1aube0e.xn--c1acygb.xn--p1ai
xn--h1aakne2ba.xn--p1ai xtremeeagles.net
xxxmania.top yadro.ru
xzlive.com yougetsignal.com
yaderenergy.ru youporn-ru.com
yeartwit.com zeikopay.com
youradulthosting.com znakom.sibtest.ru
yourporngay.com zooggames.com
ypmuseum.ru zoompegs.com
zahvat.ru zootoplist.com
zaidia.xhost.ro zynax.ua
zarenica.net zzbroya.com.ua
zerocash.msk.ru
znakomstva-piter78.ru
zone-kev717.info
zoodrawings.com
zvezdagedon.ru
zytpirwai.net

480
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
08800.top 0c47f8422d3f.com
4kpics.rocks 12masterov.com
4webmasters.com 2015god.org
5i2.net 202ch.com
888.com 404.mn
8lufu.com 5ws.dating-app.ru
aa81bf391151884adfa3dd677e41f94be1.com 9icmzvn6.website
abovetherivernc.com a96527gi.beget.tech
acads.net aadroid.net
acc.eu.org abcdeg.xyz
addray.pro actulite.com
adobereader-free.ru adbetclickin.pink
adsloads.com afftrack001.com
adverster.com agecheckadult.com
alarmobninsk.ru aktivator-windows10.blogspot.com
all-number.com albumsuper.info
alpinism.ru alibestsale.com
amateurgalls.com amazingninja.com
amazingpic.net amazingpic.net
angel1777.kz apper.de
apiluckyleapnet-a.akamaihd.net apple.com-webbrowsing-security.science
apishops.ru applicationg29.com
arius.tech arcteryxsale.online
arpelsreplica.xyz ardimobileinfo.ml
asscenihotosidea.blogspot.co.za arendas.net
avek.ru arkkivoltti.net
aviva-limoux.com arraty.altervista.org
backiee.com atlasvkusov.ru
bayareaaeroclub.org b00kmarks.com
beauby.ru banan.tv
bedandbreakfast.com beauty-b0x.pl
beta.hotkeys.com benchmarkcommunications.co.uk
bioca.org bestbuy.ca
biocn.dx.am bezsporno.ru
bjgugu.net.ua bg6s0.com
blackplanet.com billiard-classic.com.ua
bloggedporn.com bimatoprost-careprost.com.ua
boltalko.xyz bitcoinpile.com
bonanza-fortune.men blackbabesporn.com
brokergid.ru blockety.co
bum.com.ru bobinoz.com
buyk.host.sk bradleylive.xyz
cactussoft.biz bronzeaid-a.akamaihd.net
candytiner.com buttons-for-website.com
carmuffler.net buyantiviralwp.com
cartierreplica.xyz cablecar.us
cementaresearch.se cartierreplica.top
cfjump.com cejewelry.xyz
chatroulette.online centraletermice.us
cialisovercounteratwalmartusa.com cheapmoncler.win
cicaki.net cheapmoncler.xyz
clips.ua.ac.be christianlouboutinreplica.win
community.allhiphop.com cibpenonptib.flu.cc
compliance-alexa.top ciekawinki.pl
compliance-don.top cityadspix.com
coprofam.org clickhype.com
covadhosting.biz clickprotects.com
crackzplanet.com com-onlinesupport.site
crazyprotocol.com compliance-donald.xyz
dandiyabeats.in compliance-julia.top
datract.com compliance-walt.top
dellalimov.com coolwallpapers4k.info
dev78.dioniqlabb.se creatives.adbetclickin.pink
diminishedvalueoforegon.com cscwtalkto.site
domain.webkeyit.com dairyindia.in
domaineaneblanc.com daydream-studio.ru
domznaniy.ru dildofotzen.net
dostavka-v-ukrainu.ru discovertreasure-a.akamaihd.net
elektrischezigaretten2.beeplog.com doska-vsem.ru
enginebay.ru dvd-famille.com
enpolis.ru eduserver.net
eralph.tk ege-russian.ru
essay-writing.work elite-sex-finders.com
etur.ru elvenar.com
euromasterclass.ru empathica.com
evening-dating-club.info epngo.bz
exonline.info eropho.com
eyemagination.com errorfixing.space
ezofest.sk eugenevaultstorage.com
f-loaded.de
farmingworm.com
fashionavenuegame.com fashionavenuegame.com
fba-mexico.com felizporno.com
fertilitetsradgivningen.se filmi-v.online
fillmewithhappiness.com finstroy.net
filmgo.ru florida-tourism.net
findthe.pet formulaantiuban.com
floridamobilebillboards.com
forum.tvmir.org
foxinsocks.ru
fr-bearings.ru fr-bearings.ru
frcls.fr
fsalas.com fsalas.com
futbolkisales.ru fuckmill.com
gameonasia.com game-mmorpg.net
globalsurfari.com gay-site.store
goodly.pro gaypornmovie.net
hayate.biz geckoandfly.com
hotgirlhdwallpaper.com get-free-social-traffic.com
hybrid.ru get-your-social-buttons.info
iccornacircri.cf gettpromos.com
ie.57883.net ghernnqr.skyrock.com
ilmen.net gigapeta.com
ilmexico.com golden-catalog.pro
ilovevitaly.ru googlsucks.com
imagui.eu goosefishpost.bid
imperiafilm.ru grtyi.com
indianmedicaltourismshop.com gwhwpxbw.bloger.index.hr
internetproviderstucson.com healbio.ru
jav-way.site heartofpayne.xyz
javrip.net heliko.no
job.icivil.ir hit-kino.ru
joessmogtestonly.com hitsbox.info
kandidos.com homedo.fabpage.com
homelygarden.com
hoztorg-opt.ru
humanelydrew.com
hystersister.com
igrovyeavtomaty777.ru
inmate-locator.us
interfucks.net
invitefashion.com
iyasimasennka.com
jav-idol.com
javatex.co.id
joingames.org
jpcycles.com
kellyonline.xyz
kinoflux.net
kollekcioner.ru
komputernaya-pomosh-moscow.ru
kurbappeal.info kurbappeal.info
lancheck.net languagecode.com
layola.biz.tc laxob.com
lazymanyoga.com liveinternet.ru
leadn.pl loftdigital.eu
letmacwork.world lost-alpha.ru
lineshops.biz lostfilm-online.ru
loadingpages.me mafa-free.com
lomb.co mafcards.ru
lowephotos.info magento-crew.net
lutherstable.org maju.bluesalt.co
meendo-free-traffic.ga makenahartlin.com
megakino.net master-muznachas.ru
mensandals.xyz maxthon.com
meduza-consult.ru
mezaruk.info mezaruk.info
michaelkorsoutletstore.net millionare.com
mil-stak.com monclerboots.xyz
minet.club moomi-daeri.com
monitorwebsitespeed.com myperiod.club
monsterdivx.com myprintscreen.com
moscow.xrus.org naughtyconnect.com
mosrif.ru nextrent-crimea.ru
my-bc.ru nofreezingmac.work
myonigroup.com obnal.org
nacap.ru oil-td.ru
nationalbreakdown.com online-zaymy.ru
negral.pluto.ro ooomeru.ru
nikhilbahl.com opinionreelle.com
nippon-bearings.ru orion-code-access.net
norththeface.store outshop.ru
o333o.com oxotl.com
oakleyglassesonline.us panamaforbeginners.com
ok.ru panchro.xyz
oklogistic.ru par-fallen.ga
online247.ml partner-trustworthy.men
onlinemegax.com perosan.com
oranga.host.sk photokitchendesign.com
oren-cats.ru photorepair.ru
ossmalta.com
paleohub.info
palocco.it
parfusale.se
partner-high.men
partner-host.men
pearlisland.ru
pila.pl
pistonclasico.com
platinumdeals.gr platinumdeals.gr
poligon.com porndairy.in
polimga.pw pornowarp.info
pony-business.com positive2b.ru
portal-eu.ru privatamateure.com
preconnubial.usuby.site prize44.com
privatefx-in.ru prostitutki-odessa.org
privetsochi.ru prostitutki-yaroslavlya.org
prodvigator.ua ptr.ruvds.com
rczhan.com pxhdwsm.com
realmonte.net qualitymarketzone.com
redbottomheels.xyz quickbuck.com
regdefense.com rangoman.date
rockingclicks.com rasteniya-vs-zombi.ru
royalads.net rcb101.ru
rusvideos.su realizmobi.com
salmonfishingsacramentoriver.com revistaindustria.com
sarm.tk sadaholding.com
sdelatmebel.ru satellite.maps.ilovevitaly.com
search.alot.com sbprabooks.com
searchinquire.com scanner-margo.top
selectads.men seksvideoonlain.com
seminarykansas904.ml semaltmedia.com
senger.atspace.co.uk senger.atspace.co.uk
seofied.com servingnotice.com
seozoom.it sexkontakteao.info
sexblog.pw sexspornotub.com
sexsaoy.com shodanhq.com
sexxdate.net si-unique.com
shemale-sex.net sindragosa.comxa.com
shoppytoolmac.com slim.sellany.ru
skincrate.net snjatie-geroinovoy-lomki.ru
small-games.biz sockshare.net
sodexo.com softomix.net
soft-program.com solinf.co
softtor.com sotkal.lark.ru
songoo.wz.cz stackthatbucks.com
special-porn.com tako3.com
sptslmtrafms.com tampabaywatch.org
spy-app.info tattomedia.com
start.myplaycity.com telesvoboda.ru
staynplay.net thaismartloan.com
steelmaster.lv the-trader.net
stmassage.ru thelottosecrets.com
surflinksmedical.com ticketsys.inetwd.com
takeprofitsystem.com tizanidinehcl4mgtabinfo.blogspot.com
tech4master.com tizanidinesleepaid.blogspot.com
theendivechronicles.com tksn.ru
thefarmergame.com toondinsey.com
thegalerie.eu toonfamilies.net
thepornsex.org topads.men
tivolibasket.it torrent-newgames.com
tizanidineandhydrocodone.blogspot.com toxicwap.com
tizanidinehcl2mgsideeffects.blogspot.com track2.shop
track-rankings.online trackzapper.com
track.deriv.com tupper-shop.ru
traffmonster.info u555u.info
truebeauty.cc unrealcommander.com
tula.mdverey.ru upstore.me
usswrite.com uptime.com
v720hd.ru
vavilone.com
veerotech.com veerotech.com
video-chat.cn velobikestock.com
videochat.life via-energy-acquistare.com
vintontech.info videokrik.net
viktoria-center.ru
vipromoffers.com
vladimir.xrus.org vladimir.xrus.org
voloomoney.com warningwar.ru
web.cvut.cz wesharepics.info
weburlopener.com whatsupinfoley.com
white-truck.life wingsoffury2.com
whiteproduct.com woman-orgasm.ru
wineitudes.wordpress.com world-mmo.com
withstandingheartwarming.com www.wohnkabinen-shop.de
wnoz.de x-lime.com
woodyguthrie.se xdoza.com
ww2awards.info xn-----6kccaibs5cb8afhjrfmix2n.xn--p1ai
xboxster.ru xn----dtbndd4ae7eub.top
xmlinde.com xxxtubesafari.com
xmronta.com yellowproxy.net
xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai yopoint.in
xn--80ahvj9e.xn--p1ai zebradudka.com
xolodremont.ru zed21.net
xxxdatinglocal.us
yaoguangdj.com
yellowstonevisitortours.com
yugo-star.ru
zaloadi.ru
zeg-distribution.com
zigzog.ru
znaturaloriginal.com
zynax.ua

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.78 0; 10.1.0.73 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3048 ### Version: V4.2022.04.3049
### Updated: Sat Apr 16 22:01:23 UTC 2022 ### Updated: Sun Apr 17 10:03:40 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.04.3048 ### Version: V4.2022.04.3049
### Updated: Sat Apr 16 22:01:23 UTC 2022 ### Updated: Sun Apr 17 10:03:40 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##