vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-04 03:29:57 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2021.09.2606 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2021-09-22 10:09:11 +00:00
parent 5379422e2f
commit 1fa8c015ae
46 changed files with 87313 additions and 87313 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2605 #### Version: V4.2021.09.2606
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2021.09.2605 #### Version: V4.2021.09.2606
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -11,7 +11,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2021.09.2605 #### Version: V4.2021.09.2606
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10776
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2021.09.2605 #### Version: V4.2021.09.2606
#### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7078](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [623](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10780
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10780
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10780
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10780
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 1; 10.1.0.128 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 0; 10.1.0.128 0;
127.0.0.1 0; 127.0.0.1 0;

10780
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 1; 10.1.0.128 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 0; 10.1.0.128 0;
127.0.0.1 0; 127.0.0.1 0;

10780
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 1; 10.1.0.128 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 0; 10.1.0.128 0;
127.0.0.1 0; 127.0.0.1 0;

10780
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
2605 2606

10780
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10780
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.76.82.171 0; 52.176.104.23 0;

10780
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 1; 10.1.0.128 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 0; 10.1.0.128 0;
127.0.0.1 0; 127.0.0.1 0;

10780
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 1; 10.1.0.128 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 0; 10.1.0.128 0;
127.0.0.1 0; 127.0.0.1 0;

10780
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 1; 10.1.0.128 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 0; 10.1.0.128 0;
127.0.0.1 0; 127.0.0.1 0;

10780
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.76.82.171 0; 52.176.104.23 0;

10780
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.76.82.171 0; 52.176.104.23 0;

10780
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 1; 10.1.0.128 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Blow 404enemy
BotALot Cliqzbot
Deusu CrunchBot
Mata Hari Go-Ahead-Got-It
Mister PiX HaosouSpider
NimbleCrawler LNSpiderguy
SocialRankIOBot LeechFTP
WBSearchBot RealDownload
Whatweb Scanbot
probe-image-size ZumBot

324
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider 360Spider
404enemy
ADmantX ADmantX
AIBOT AIBOT
Abonti Acunetix
Aboundex Alexibot
Aboundexbot
AiHitBot
Aipbot
AllSubmitter AllSubmitter
Alligator
Anarchie
Ankit
Aspiegel
Asterias Asterias
Attach
AwarioSmartBot
BBBike
BDCbot
BDFetch BDFetch
BackDoorBot BackDoorBot
Backlink-Ceck BackWeb
Bandit Badass
Battleztar Bazinga Barkrowler
BatchFTP
BetaBot BetaBot
Bitacle
Black Hole
BlackWidow
Blow Blow
BlowFish BuiltWith
Boardreader Bullseye
Brandprotect
Brandwatch
Buddy
CCBot CCBot
CODE87 CODE87
Calculon
CazoodleBot CazoodleBot
Cliqzbot ChinaClaw
Cloud mapping Claritybot
Copier Cocolyzebot
Crawling at Home Project Cogentbot
CopyRightCheck
Copyscape
Craftbot
Crescent
CrunchBot CrunchBot
DSearch DTS Agent
DataCha0s Demon
Deusu
Devil Devil
Disco Dispatch
Discobot DittoSpyder
DnyzBot
DomCopBot DomCopBot
DomainAppender
DomainCrawler
DomainStatsBot DomainStatsBot
Dotbot Domains Project
Drip Download Wonder
EMail Siphon ECCP/1.0
EirGrabber EMail Wolf
Evil Ebingbong
Extractor EroCrawler
ExtractorPro Extreme Picture Finder
EyeNetIE FHscan
Ezooms
FemtosearchBot FemtosearchBot
FyberSpider Fimap
Flunky
Fuzz
Fyrebot Fyrebot
GalaxyBot
GetWeb
Getintent
Gigabot Gigabot
GrapeshotCrawler GoZilla
GridBot GrabNet
HTMLparser HEADMasterSEO
Haansoft HaosouSpider
Harvest Havij
Heritrix Hloader
Humanlinks
HybridBot
IDBTE4M
IDBot
IRLbot IRLbot
IndeedBot Iblog
InterGET Id-search
Image Fetch
Image Sucker
Indy Library
Internet Ninja Internet Ninja
InternetSeer InternetSeer
Iria IstellaBot
JOC Web Spider
Jbrofuzz
JennyBot JennyBot
JetCar
JikeSpider JikeSpider
Joomla Joomla
Kenjin Spider Jorgee
Keyword Density JustView
Kinza Jyxobot
LNSpiderguy
LWP::Simple
Lanshanbot Lanshanbot
Leap Larbin
LeechFTP LeechGet
LibWeb Lftp
Libwhisker
LieBaoFast LieBaoFast
Linkdexbot Lightspeedsystems
LinkWalker
LinkextractorPro
LinksManager
LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
Litemage_walker Lipperhey
Ltx71 Lipperhey Spider
MFC_Tear_Sample MFC_Tear_Sample
MJ12bot MIDown tool
Mail.RU_Bot MIIxpc
Majestic SEO MQQBrowser
Majestic12 MSIECrawler
MarkMonitor Magnet
MarkWatch Mass Downloader
Masscan Mb2345Browser
MauiBot MeanPath Bot
Metauri
Microsoft URL Control Microsoft URL Control
Moblie Safari
Mojeek Mojeek
Mojolicious
Mozlila Mozlila
Mr.4x3 Musobot
NPbot Name Intelligence
Nameprotect Nameprotect
NearSite Nessus
Needle
Net Vampire Net Vampire
NetAnts NetLyzer
NetSpider Netcraft
Nettrack
Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler Niki-bot
Nikto Nikto
NimbleCrawler
Nimbostratus Nimbostratus
Nuclei
Octopus Octopus
Offline Explorer OnCrawl
OpenLinkProfiler OpenLinkProfiler
OrangeBot
OrangeSpider
OutclicksBot
OutfoxBot
PECL::HTTP
Page Analyzer
PageAnalyzer
PageScorer
PageThing.com PageThing.com
Papa Foto Panscient
Pavuk Pavuk
Pi-Monster PeoplePal
Picsearch Picsearch
Piepmatz PictureFinder
Pockey PleaseCrawl
ProPowerBot
ProWebWalker ProWebWalker
Pu_iN Probethenet
PxBroker Psbot
PyCurl
QueryN Metasearch QueryN Metasearch
RSSingBot
RankActive RankActive
RankActiveLinkBot RankActiveLinkBot
RankFlex
RankingBot RankingBot
RedesScrapy RankingBot2
RankurBot
Re-re
RebelMouse
Recorder
RepoMonkey RepoMonkey
Rogerbot Rogerbot
SEOkicks SEOkicks
SISTRIX
SalesIntelligent SalesIntelligent
ScanAlert ScoutJet
SearchmetricsBot Scrapy
SentiBot Screaming
Seomoz SemrushBot
SeoSiteCheckup
SeobilityBot
Shodan
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteLockSpider
Sitevigil Sitevigil
Snake
Snapbot
Snoopy
SocialRankIOBot SocialRankIOBot
Sociscraper Sociscraper
Spanner SpaceBison
Spinn3r Spbot
Sqlworm
Sqworm
Steeler Steeler
SuperHTTP Sucker
Swiftbot Swiftbot
Szukacz Szukacz
T8Abot T0PHackTeam
Telesoft Teleport
The Intraformant TeleportPro
Telesphoreo
Telesphorep
TheNomad TheNomad
Thumbor Thumbor
TightTwatBot Titan
Toata Turingos
Trendictionbot Turnitin
TwengaBot TwengaBot
URLy Warning
Upflow Upflow
VCI V-BOT
VB Project
Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VidibleScraper
Virusdie Virusdie
VoidEYE
Voil
WBSearchBot
WEBDAV WEBDAV
WWW::Mechanize WWW-Mechanize
WWWOFFLE WeSEE
WallpapersHD Web Auto
Web Collage
Web Fetch
Web Fuck
Web Pix
Web Sucker
WebAuto WebAuto
WebImageCollector WebCollage
WebLeacher WebCopier
WebEnhancer
WebFetch
WebPix WebPix
WebZIP
Webalta Webalta
WebmasterWorldForumBot
Website Quester Website Quester
WebsiteExtractor
Webster Webster
Whatweb Whatweb
WinHTTrack
WiseGuys Robot
Wonderbot Wonderbot
Woobot Woobot
Wotbox
Wprecon
Xaldon WebSpider Xaldon WebSpider
Xenu Xaldon_WebSpider
YoudaoBot YoudaoBot
Zade Zeus
Zermelo Zitebot
ZoominfoBot ZmEu
ZoomBot
ZumBot ZumBot
ZyBorg
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
autoemailspider autoemailspider
backlink-check backlink-check
clark-crawler clark-crawler
com.plumanalytics coccocbot-web
crawl.sogou.com cognitiveseo
crawler4j crawler.feedback
dataforseo.com eCatch
domainsproject.org heritrix
instabid internetVista monitor
ips-agent
isitwp.com
lwp-request
lwp-trivial lwp-trivial
magpie-crawler
meanpathbot meanpathbot
mediawords
muhstik-scan
netEstate NE Crawler
oBot oBot
page scorer
plumanalytics plumanalytics
satoristudio.net probe-image-size
serpstatbot s1z.ru
scan.lol
seobility
seoscanners
seostar
sitechecker.pro sitechecker.pro
sogouspider siteripz
trendiction.com webmeup-crawler
trendiction.de webpros.com
webprosbot webprosbot
x09Mozilla
x22Mozilla x22Mozilla
zauba.io zauba.io
zgrab

314
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404checker 404checker
80legs 404enemy
ADmantX AhrefsBot
ALittle Client
Abonti
Acunetix
AiHitBot
Aipbot
Alexibot
AllSubmitter AllSubmitter
Alligator
AlphaBot
Anarchy
Anarchy99 Anarchy99
Ankit Ankit
Asterias Apexoo
AwarioRssBot
AwarioSmartBot AwarioSmartBot
BBBike BDCbot
BDFetch
BackDoorBot BackDoorBot
BackStreet BackStreet
Backlink-Ceck BackWeb
BacklinkCrawler BacklinkCrawler
Badass Badass
Bandit
Barkrowler
BatchFTP
Battleztar Bazinga Battleztar Bazinga
Bigfoot Black Hole
Bitacle
Blackboard
Blow Blow
Boardreader Boardreader
BotALot BotALot
Brandprotect
Buddy Buddy
Bullseye BuiltBotTough
BunnySlippers CODE87
BuzzSumo Calculon
CSHttp
CazoodleBot CazoodleBot
CheTeam
CherryPicker CherryPicker
Claritybot ChinaClaw
Chlooe
Cliqzbot Cliqzbot
Cocolyzebot
Cogentbot Cogentbot
Copier Crawling at Home Project
Copyscape
Cosmos
Crescent
Curious Curious
Custo
CyotekWebCopy
DBLBot
DTS Agent DTS Agent
Dirbuster DataCha0s
Discobot DatabaseDriverMysqli
Discoverybot Devil
DomainStatsBot DittoSpyder
Domains Project DnyzBot
ECCP/1.0 DomCopBot
Download Wonder
Drip
EMail Siphon EMail Siphon
Ebingbong
Ecxi
EroCrawler EroCrawler
Evil ExtLinksBot
Extractor ExtractorPro
Extreme Picture Finder
EyeNetIE EyeNetIE
Flunky Ezooms
FDM
FemtosearchBot
Fimap
Foobot
Freeuploader
FyberSpider
Fyrebot Fyrebot
Getintent G-i-g-a-b-o-t
Go!Zilla GalaxyBot
Go-Ahead-Got-It Genieo
Grafula GermCrawler
GrapeFX Gigabot
GrapeshotCrawler GoZilla
HTTP::Lite HTTP::Lite
HTTrack
Haansoft
Heritrix
Humanlinks
HybridBot HybridBot
IDBot IDBot
IRLbot
Iblog Iblog
InternetSeer Image Fetch
IstellaBot Indy Library
JustView InfoNaviRobot
Jyxobot Intelliseek
Keyword Density JOC Web Spider
Kozmosbot JamesBOT
JennyBot
Jetty
JikeSpider
Jorgee
LNSpiderguy
LWP::Simple LWP::Simple
Leap Larbin
LexiBot LeechFTP
Lftp
LibWeb LibWeb
Libwhisker
LieBaoFast
Likse
LinkScan
LinkWalker
Linkbot
Linkdexbot
LinkextractorPro LinkextractorPro
LinqiaMetadataDownloaderBot LinqiaMetadataDownloaderBot
LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey Spider
Lmspider Lmspider
MSIECrawler Ltx71
MTRobot MIDown tool
MIIxpc
MQQBrowser
Mag-Net Mag-Net
MarkWatch Majestic SEO
MarkMonitor
Mata Hari Mata Hari
MicroMessenger MauiBot
Microsoft URL Control Mb2345Browser
Mister PiX Mediatoolkitbot
Moblie Safari MegaIndex.ru
Mojeek Microsoft Data Access
Morfeus Fucking Scanner Musobot
Mozlila Name Intelligence
Msrabot Nameprotect
NPbot
Needle
Nessus Nessus
NetAnts
NetLyzer
NetSpider NetSpider
Netcraft
Nettrack Nettrack
Netvibes
NextGenSearchBot NextGenSearchBot
Niki-bot Nikto
Ninja
Offline Explorer Offline Explorer
Offline Navigator OpenVAS
OnCrawl Openvas
OpenLinkProfiler
Openfind
OrangeSpider
PECL::HTTP PECL::HTTP
PHPCrawl PHPCrawl
POE-Component-Client-HTTP Page Analyzer
Pavuk PageScorer
Papa Foto
Petalbot
Pi-Monster Pi-Monster
Picscout
Picsearch Picsearch
Pimonster PictureFinder
PleaseCrawl Piepmatz
Pockey
Probethenet
Psbot Psbot
PxBroker
RSSingBot RSSingBot
RankActive RankActive
RankActiveLinkBot
RankFlex
RankingBot RankingBot
RankingBot2 RedesScrapy
RankurBot RepoMonkey
RealDownload
RebelMouse
Recorder
Ripper
RocketCrawler RocketCrawler
SBIder SEOkicks-Robot
SEOlyticsCrawler Screaming
SEOstats Searchestate
SalesIntelligent
Scanbot
ScoutJet
SemrushBot SemrushBot
SeoSiteCheckup SeoSiteCheckup
Shodan Seomoz
Site Sucker Site Sucker
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger
SiteSucker
Sitebeam Sitebeam
Siteimprove SlySearch
SmartDownload SmartDownload
Snake Snake
Snoopy Sociscraper
Sogou web spider Sottopop
Sosospider
SpaceBison SpaceBison
Sqlmap Spammen
Sqlworm SpankBot
Stripper Spbot
Sucker Sqworm
Sucuri
Surfbot
SurveyBot SurveyBot
Suzuran
T8Abot
Teleport Teleport
Telesphoreo
Telesphorep
The Intraformant The Intraformant
Toweyabot Titan
Trendictionbot Tracemyfile
True_Robot True_Robot
TwengaBot Turingos
Typhoeus
URLy Warning URLy Warning
URLy.Warning UnisterBot
Upflow Upflow
V-BOT V-BOT
Vacuum
Vagabondo Vagabondo
VeriCiteCrawler VeriCiteCrawler
VidibleScraper Virusdie
Voil
WBSearchBot WBSearchBot
WEBDAV WEBDAV
WISENutbot
WPScan WPScan
WWW-Mechanize WWW-Mechanize
Wallpapers
Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE
Web Enhancer
Web Fuck Web Fuck
Web Pix
Web Sauger
WebAuto WebAuto
WebBandit
WebCollage
WebCopier WebCopier
WebEnhancer WebFetch
WebFuck WebFuck
WebImageCollector WebImageCollector
WebReaper WebStripper
WebSauger WebSucker
WebWhacker WebWhacker
Webshag WebmasterWorldForumBot
Website Quester Website Quester
WebsiteExtractor WebsiteExtractor
Webster Webster
Whacker
Widow
Wonderbot Wonderbot
Wprecon Woobot
Xenu Wotbox
Xaldon WebSpider
YoudaoBot
Zade
Zermelo
Zeus Zeus
Zitebot
ZmEu ZmEu
ZoominfoBot ZoomBot
ZumBot
ZyBorg ZyBorg
clark-crawler archive.org_bot
crawl.sogou.com arquivo.pt
backlink-check
cah.io.community
check1.exe
crawler.feedback
dataforseo.com dataforseo.com
demandbase-bot eCatch
domainsproject.org evc-batch
heritrix facebookscraper
instabid instabid
ips-agent
isitwp.com isitwp.com
lwp-request
lwp-trivial lwp-trivial
magpie-crawler muhstik-scan
meanpathbot netEstate NE Crawler
mediawords polaris version
oBot probe-image-size
page scorer
pcBrowser
satoristudio.net
scan.lol scan.lol
sexsearcher seobility
sitechecker.pro seoscanners
sogouspider
sp_auditbot sp_auditbot
tAkeOut
trendiction.com
trendiction.de trendiction.de
ubermetrics-technologies.com ubermetrics-technologies.com
webprosbot webpros.com
x09Mozilla x22Mozilla
xpymep1.exe
zauba.io
zgrab

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
apifasterlightin-a.akamaihd.net adbetclickin.pink
brainxs.us bag77.ru
ci.ua board.f00d.de
ege-essay.ru hasfun.com
egovaleo.it kimcurlrvsms.com
freshsuperbloop.com narkologiya-peterburg.ru
pigrafix.at page2rss.com
rus-teh.narod.ru runtnc.net
thelottosecrets.com stardevine.com
yeartwit.com startwp.org

942
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1001desert.com 00author.com
123cha.com 03e.info
08800.top
101billion.com
101lesbian.xyz
123locker.com 123locker.com
1314dh.com 178evakuator178.ru
1xbet4you.com 1kinobig.ru
20pascals.nl
214jaluwobapef.cf 214jaluwobapef.cf
2itech.ru 256bit.by
2kata.ru 2x2fan.ru
4rent.ru 3xforum.ro
7fon.ru 5kstore.com
7wind.ru
8kisses.eu
aab94f698f36684c5a852a2ef272e031bb.com
aac500b7a15b2646968f6bd8c6305869d7.com
aadroid.net aadroid.net
aarbur.com accessoires-mode.in
acads.net adexprts.com
acmebtn.ml adktrailmap.com
adssafeprotected.com admitad.com
adult-shop.com.ua adtiger.tk
adultfriendfinder.com
adultmeetonline.info adultmeetonline.info
adultwebhosting.info
advancedcleaner.com advancedcleaner.com
affordablewebsitesandmobileapps.com adventureparkcostarica.com
ald2014.org adviceforum.info
alienwheels.de advokateg.ru
alloysteel.ru afmuseum.com
allwomen.info agapovdg.ru
am15.net alert.scansafe.net
amateurgalls.com all-streaming-media.com
amateurmatch.com allnews24.in
anal-acrobats.hol.es alphavisions.net
aosheng-tech.com anal-acrobats.com
apibrowsesmartne-a.akamaihd.net andpolice.com
apimossnetinfo-a.akamaihd.net apiappenableinfo-a.akamaihd.net
apiqualitinknet-a.akamaihd.net apper.de
aramaicmedia.org apple.com-webbrowsing-security.review
arenda-avtoprokat-krasnodar.ru appleid-verification.com
arendakvartir.kz artdeko.info
artdestshop.eu as5000.com
asociatia-tipografilor-transilvania.ro asrv-a.akamaihd.net
asscenihotosidea.blogspot.com asrvvv-a.akamaihd.net
astana.xxxkaz.org atleticpharm.org
astrochicks.com australia-opening-times.com
atmovs.com auto4style.ru
atovh.local-finders.com autoua.top
audiobangout.com autovideobroadcast.com
ausergrubhof.info avramstroy.ru
autogrep.ru
aviav.eu
aviav.ru.com
avtochehli.by
azinoofficial777.ru azinoofficial777.ru
azte.ch badavit.com.ua
b-styles.xyz bagcionderlab.com
balkanfarma.org balla-boo.se
balook.com bankiem.pl
bbtec.net battle.net
beachdriveblog.com bayareaaeroclub.org
beremenyashka.com begalka.xyz
berrymall.ru bestwrinklecreamnow.com
best-businessman.ru bet365.com
best-deals-products.com bget.ru
bestssaker.com bigcities.org
biglistofwebsites.com biglistofwebsites.com
biographiya.com biocn.dx.am
bioscorp.ru blackcurranthumidifiers.site
bitcoinpile.com boleznikogi.com
bizlist.com.de bonky.biz
blog-hits.com boole.onlinewebshop.net
bloggerads.net braip.com.br
blue-square.biz brendbutik.ru
boc.kz brus.city
boostmyppc.com
bosman.pluto.ro
brainboostingsupplements.org
brk-rti.ru
brokergid.ru
btc4u.ru btc4u.ru
btvn.ru budpost.com.ua
butstrap.space buigas.00it.com
buy-cheap-online.info buqayy0.livejournal.com
buy-meds24.com burn-fat.ga
buyk.host.sk bvlgaribracelet.xyz
buypuppies.ca callawaygolfstore.online
buyscabiescream.com callmd5map.com
buzz-porno.info camel-beach.com
bzero1jewelry.net cand.jp
callawaygolfoutlet.online cars-modification.net
callejondelpozo.es cartierjusteunclou.xyz
camdolls.com cellfun.mobi
carstrends2015.com cercacamion.it
cartierbraceletsreplica.pw certifywebsite.win
cartierlovebracelet.xyz chastnoeporno.com
cascadelink.org chatroulette.life
celejihad.info cheapjerseysbizwholesale.us
cementaresearch.se cheapjerseysfootballshop.com
cerev.info check-host.net
charmstroy.info checkhit.com
cheapbelstaff.online choice-credit.ru
checkpagerank.net cialischmrx.com
christianlouboutinsaleoutletonline.us cialisovercounteratwalmartusa.com
christianlouboutinshoes.xyz cigarpass.com
citizenclsdriveri7.pen.io cleanpcnow.com
clickprotects.com clickpapa.com
clipartnew.com clubfashionista.com
codq.info cmd.kz
collegeessay19.blogspot.ru coaching-netz.info
com-secure.download com-supportcenter.website
compliance-elena.xyz compliance-olga.top
connectionstrenth.com compucelunlock.net
cosmetic.donna7753191.ru connectingsingles.com
counter.yadro.ru constantaservice.net
courtshipgift.com coolbar.pro
covadhosting.biz coolgramgoods.com
covid-schutzmasken.de coolyarddecorations.com
cs-passion.pl copypaste.traderzplanet.in
dailystrength.org cottageofgrace.com
darkages.info country-chic.ru
dating-app.ru cpajunkies.com
dayibiao.com creams.makeforum.eu
dealwifi.com daisye.top
diamond-necklace.info damasarenai.info
digest-project.ru deanmoore.ie
decoratinghomes.ga
defenderxtactical.com
dentalpearls.com.au
dev-seo.blog
dfwu1013.info
diarioaconcagua.com
digitalassetmanagement.site
dignitasdata.se dignitasdata.se
dikqlyremy.info dikx.gdn
dinosaurus.site diminishedvalueoforegon.com
directivepub.com dispo.de
dirtpics.pw djstools.com
discovertreasure-a.akamaihd.net doctormakarova.ru
distver.ru dogclothing.store
djihispano.com dojki-hd.com
dnepropetrovsk.mistr-x.org dojki365.online
docsportal.net domaincheck.io
doctissimo.fr domainsphoto.com
domainroam.win domcran.net
domznaniy.ru dotmass.top
droidlook.net dpihatinh.gov.vn
drpornogratisx.xxx drivotracker.com
dynainbox.com drunkenstepfather.com
e-collantes.com dtm-spain.com
easyshoppermac.com e-commerce-seo1.com
ecookna.com.ua eas-seo.com
ekto.ee econom.co
election.interferencer.ru editors.choice6912650.hulfingtonpost.com
electro-prom.com ekaterinburg.xrus.org
eleimgo.pw elektrischezigarettekaufen2.cowblog.fr
elektrischeziga.livejournal.com
elektrischezigaretten2.beeplog.com elektrischezigaretten2.beeplog.com
emerson-rus.ru elidelcream.weebly.com
en.altezza.travel enews.tech
enge-fotzen.info enginebay.ru
es5.com essay-writing.work
espaceinventoristes.com euroskat.ru
eupornstar.info event-tracking.com
europeanwatches.ru exoneration-project.us
exct.net extrabot.com
expolicenciaslatam.co eyes-on-you.ga
extremez.net ezb.elvenmachine.com
extstat.com f-online.de
ezigarettekaufen2.mpbloggar.se falcon-images.blogspot.com
fandlr.com falcoware.com
fearcrow.com family1st.ca
fickenbumsen.net faracontrol.ir
fil.ru fashion-mk.net
fillmewithhappiness.com fatfasts-4tmz.com
filmania-x.ru fatmaelgarny.com
findanysex.com feeriaclub.ru
finejewelryshop.ru fettefrauen.net
firesub.pl fighrofacciufreesig.ga
fitness-video.net fikasound.tk
flashlarevista.com findpik.com
floating-share-buttons.com finteks.ru
food.dtu.dk firstsiteguide.com
footbalive.org fitfloponline.store
fordsonmajbor.cf fivedwld.cf
fortwosmartcar.pw free-share-buttons.blogspot.com
frameimage.org free-social-buttons.xyz
frankofficial.ru front.ru
free-today.com fullgirl.ru
freemaintenancesysforpcandmac.top g.starmoe.xyz
freewebs.com gabeshop.ru
fuckingawesome.com gamebackyard.com
galleryawesome.com gameonasia.com
gambarkataku.co getmiro.com
gamesprite.me ghostvisitor.com
gazporno.com gigixo.com
getlaid-xxxhookupdirect.com goldadpremium.com
ghernnqr.skyrock.com golden-catalog.pro
glastecfilms.com.my gonextmedia.com
globalscam.ga gorabagrata.ru
go2jump.org gta-club.ru
googglet.com hahashka.ru
gpirate.com haveinc.xyz
gpms.org.my hccoder.info
greenzaim.ru hdapp1008-a.akamaihd.net
growshop.es hdimagegallery.net
guardlink.org hidemyass.com
hacktougroup.ru holidaypics.org
handicapvansarea.com hop.clickbank.net
hanwei.us hornymatches.com
hatedriveapart.com hotelcrocenzi.sm
hayate.biz hotloans.ru
headpress.ru hotsocialz.com
hentai-manga.porn house.sieraddns.com
histats.com hoverboard360.es
hkladys.com hplaserjetpdriver8y.pen.io
homedecoguide.info hscsscotland.com
homegardenlova.com ie.57883.net
hotkeys.com ihtec2019.org
housediz.com ilovevitaly.xyz
hspline.com imagerydatabase.com
huhn.altervista.org imediadesk.com
humbmosquina.tk imicrovision.com
hustoon.over-blog.com immobiliaremassaro.com
ibmdatamanagement.co importchinacoach-teach.com
idc.com.ua impresagaia.it
ideawheel.com inmate-locator.us
imagui.eu investsuccess.org
inboundlinks.win ipostroika.ru
infobanks.ru isitpaleo.info
innodgfdriverhm.aircus.com
interior-stickers.ru
iron-age.info
ispac.org
istanbulit.com
itronics.ca
izismile.com izismile.com
jewelryandfiligree.com j33x.com
jrcigars.com jacago.com
junketjuice.blogspot.com janerikholst.se
kabbalah-red-bracelets.com je7.us
kanimage.com jikoman.info
kenaba.su kandidos.com
keywordblocks.com keyhantercume.com
kinocccp.net keywordglobal.co.za
kit-opt.ru keywordhouse.com
khafre.us
kinofak.net
kinoplen.ru
kiskinhouse.com
kletkimehan.ru kletkimehan.ru
knigonosha.net kollekcioner.ru
kochanelli.com komputernaya-pomosh-moscow.ru
kreativperlen.ch kupit-adenu.ru
kvartir-remont.biz kvartir-remont.biz
kvartiry-remont.ucoz.ru kvartiry-remont.ucoz.ru
kw21.org larger.io
larutti.ru larutti.ru
laserpen.club laudit.ru
law-check-seven.xyz lesbian.xyz
letmacworkfaster.site lescinq.com
likesuccess.com lexixxx.com
liquimondo.com linkdebrideur.xyz
liupis.com lion.misba.us
liver-chintai.org livejasmin.com
liverpool.gsofootball.com ljusihus.se
lkbennettoutlet.online lmrauction.com
lolnex.us louisvuittonoutletstore.net
long-beach-air-conditioning.com luckyday.world
lost-alpha.ru mac-shield.com
luciddiagnostics.in make-money-online.com
macdamaged.tech makenahartlin.com
macotool.com manimpotence.com
magento-crew.net manipulyator-peterburg.ru
magicdiet.gq manualterap.roleforum.ru
makedo.ru manuscript.su
mapquestz.us markjaybeefractal.com
marinetraffic.com mathgym.com.au
maxxtor.eu mebel-alait.ru
mebel-ekb.com med-zdorovie.com.ua
mebeldekor.com.ua medicineseasybuy.com
medicinacom.ru medkritika.ru
menetie.ru meetlocalchicks.com
merryhouse.co.uk mex-annushka.ru
michellblog.online milblueprint.com
militarysale.pro miniature.io
misandesign.se mirmedinfo.ru
mmostrike.ru mirobuvi.com.ua
mobstarr.com mockupui.com
moesexy.com modabutik.ru
mojaocena.com modenamebel.ru
mompussy.net moesen-ficken.com
monclervests.xyz mokrayakiska.com
mosdverka.ru morf.snn.gr
mundoaberrante.com morlat.altervista.org
mvpicton.co.uk mrinsidesales.com
my-big-family.com mybackgroundlandscape.blogspot.com
my-cash-bot.co myghillie.info
myprintscreen.com mylida.org
mystats.xyz myperiod.club
myyour.eu
mzdish.site
na-telefon.biz
na15.ru
naphukete.ru naphukete.ru
nastroyke.net
nastydollars.com
natali-forex.com
nationalbreakdown.com nationalbreakdown.com
naturalbreakthroughsresearch.com naturalpharm.com.ua
naughtyconnect.com negral.pluto.ro
net-radar.com netanalytics.xyz
netfacet.net newfilmsonline.ru
new-apps.ru nikitsyringedrivelg.pen.io
nina.az nosecret.com.ua
novodigs.com notebook-pro.ru
nsatc.net novgorod.xrus.org
nuup.info nrv.co.za
odesproperty.com nzfilecloud.weebly.com
oneshotdate.com obiavo.uz
orgasmatrix.com odoratus.net
owathemes.com offer.camp
ownshop.cf officedocuments.net
offside2.5v.pl
okeinfo.online
olgacvetmet.com
onescreen.cc
outclicks.net
outshop.ru
oz-offers.com
p-business.ru
paintingplanet.ru
palocco.it palocco.it
partner-pop.men palvira.com.ua
paphoselectricianandplumber.com
partner-host.men
partner-stop.men
patol01.pw
patterntrader-en.com
pb-dv.ru
pc-virus-d0l92j2.pw pc-virus-d0l92j2.pw
pelfind.me pekori.to
perm-profnastil.ru pendelprognos.se
pharmacyincity.com percin.biz.ly
phormchina.com pertlocogasilk.tk
photostudiolightings.com phobia.us
php-market.ru pills24h.com
picphotos.net pintattoos.com
picture-group.com piter.xrus.org
pictures-and-images.net pizza-imperia.com
pillscheap24h.com plugingeorgia.com
piluli.info porn-w.org
pinstake.com porndairy.in
piski.top pornhub-forum.ga
pizdopletka.club pornofiljmi.com
pizza-tycoon.com pornoklad.ru
playtap.us pornolook.net
pornoklad.net pornoted.com
pornosexrolik.com portnoff.od.ua
pornstartits.xblog.in portside.xyz
pornzone.tv preventheadacheguide.info
potolokelekor.ru prizesbook.online
powitania.pl prodess.ru
preparevideosafesystem4unow.site profitwithalex.info
prize44.com prostitutki-odessa.org
prizefestival.mobi purplestats.com
produkto.net puteshestvennik.com
profit-opportunity.com putitin.me
progonrumarket.ru q-moto.ru
prokotov.com qexyfu.bugs3.com
prostitutki-belgoroda.org rating-casino2021.ru
prostitutki-tyumeni.org
purplesphere.in
quickbuck.com
qwarckoine.com
raavidesigns.com
raisedseo.com
ranksonic.org
rapidgator-porn.ga
rarbg.to
razorweb-a.akamaihd.net razorweb-a.akamaihd.net
remmling.de rebelmouse.com
remont-fridge-tv.ru registratciya-v-moskve.ru
remont-ustanovka-tehniki.ru registrationdomainsite.com
remontbiz.ru remontbiz.ru
resant.ru remontgruzovik.ru
ring4rhino.com rewardit.com
rocketchange.ru rospromtest.ru
roma-kukareku.livejournal.com royalvegascasino.com
royal-betting.net roznica.com.ua
rp9.ru s-iwantyou.com
rumamba.com saecsa.co
rustic-quiver.win safe-app.net
sandhillsonline.com salut-camp.ru
sardinie.us sammlungfotos.online
saugatuck.com sammyweaver.com
savememoney.co.za saneitconsulting.com
sdi-pme.com sashagreyblog.ga
searchlock.com saveriopiazza.it
securesmrt-dt.com sax-sex.com
selectads.men sdsjweb.com
semaltmedia.com secret.xn--oogle-wmc.com
seo18.su semalt.com
seocheckupx.com semrush.com
serdcenebolit.com seoprofiler.com
sfj-ror.no servethis.com
shell-pmr.ru sex-tracker.com
shopfishing.com.ua sex-znakomstva.online
sibecoprom.ru sexobzor.info
simpoed.ufop.br sharebutton.net
sitopreferito.it shariki-zuma-lines.ru
slowmac.tech sherlock.se
smadihome.com shikiso.info
smart-scripts.com silvercash.com
sobecjvuwa.com.ru siteimprove.com
socialsignals24.com sms2x2.ru
softtor.com snip.tw
space2019.top songplanet.ru
spammen.de soundfrost.org
souvenir.cc
spasswelt.net
spitfiremusic.com
spravka130.ru
stanthonyscatholicchurch.org
stariy-baku.com stariy-baku.com
statustroll.com
steame.ru
steebook.com steebook.com
stefanbakosab.se
stevemonsen.com stevemonsen.com
superfish.com store-rx.com
superkanpo.com stroymonolit.su
supernew.org subj.ukr-lit.com
tackletarts.co sucsesofinspiration.com
td-l-market.ru suralink.com
tdsing.ru svbur.ru
techart24.com tam-gde-more.ru
tecspb.ru
tedy.su tedy.su
tetracsaudi.com teenforporn.com
thaismartloan.com teenfuck.tv
theautoprofit.ml telesvoboda.ru
thebestphotos.eu testbotprocessor44.com
themeforest.net thegalerie.eu
themestotal.com thetoiletpaper.com
thruport.com thexart.club
tizanidinemusclerelaxant.blogspot.com timdreby.com
tizanidineshowupondrugtest.blogspot.com timeallnews.ru
tmearegion26.com tmm-kurs.ru
toolsky.com tnctrx.com
toon-families.com tomatis.gospartner.com
topflownews.com top-karkas.ru
torrentdownloadhub.com top10registrycleaners.com
torrents.cd torrents.life
track.deriv.com tpu.ru
tracfone.com
track-rankings.online
track112.site track112.site
tracklead.net
trackmedia101.com
tracksz.co
traderzplanet.in traderzplanet.in
traxdom.ru trafficnetzwerk.de
trucri.me trafficreceiver.club
truebeauty.cc traffmonster.info
tuberkuleznik.ru traffpartners.com
twittrading.com trubywriting.com
uasb.ru ttrraacckkrr.com
uk-zheu20.ru tupper-posuda.ru
ukrtextbook.com tvnewsclips.info
unitygame3d.com twiclub.in
url-extractor.xyz uchil.net
usadacha.net ucsol.ru
usdx.us uggsale.online
vbabule.net unrealcommander.com
vduplo.ru uroffer.link
veerotech.com usiad.net
video-chat.in v-doc.co
videochat.life vacances-voyages.info
vids18.site vchulkax.com
villakohlanta.nu vedomstvo.net
visa-china.ru vereo.eu
visionwell.com.cn viagra.pp.ua
viteonlusarezzo.it viddyoze.com
vkak.ru videochat.world
vksex.ru videogamesecrets.com
vsesubwaysurfers.com vipsiterip.org
vtc.pw vkontaktemusic.ru
vuclip.com voloo.ru
vucms.com vonradio.com
w7s.ru vote-up.ru
wallpapersist.com vpnhowto.info
vut.com.ru
walpaperlist.com walpaperlist.com
watchmygf.net wardreapptokone.tk
website-audit.com.ua warningzscaler.heraeus.com
weddingdresses.xyz waysbetter.cn
welck.octopis.com webmasters.stackexchange.com
welcomeauto.ru webshoppermac.com
wemedinc.com website-speed-check.site
westermarkanjou.se website-speed-up.top
whipme.yopoint.in weburok.com
whiteproduct.com wesharepics.site
wholesalejerseysgaa.com westvilletowingservices.co.za
wholinkstome.com wetgames.ru
wildnatureimages.com wfb.hatedriveapart.com
win-spy.com whatistizanidinehclusedfor.blogspot.com
wiosenny-bon-1500.pl wholesalejerseychinaoutlet.com
wonderfulflowers.biz wingsoffury2.com
workle.website womensterritory.ru
wovis.site word-vorlagen.net
wrona.it word-vorlagen.xyz
wtsindia.in worldhistory.biz
wygraj-skiny.win worldmusicfests.com
x-diesel.org worst-sites.online
x-stars.ru xedserver.com
x69ty.ru xn-----8kcatubaocd1bneepefojs1h2e.xn--p1ai
xev.ru xn----8sbdbjgb1ap7a9c4czbh.xn--p1acf
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn----btbdvdh4aafrfciljm6k.xn--p1ai xn--e1afanlbnfckd7c3d.xn--p1ai
xn--80aanaardaperhcem4a6i.com xnxxandxvideos.com
xn--80ahvj9e.xn--p1ai xportvusbdriver8i.snack.ws
xolodremont.ru xxxguitars.com
xxxhdvideo.site yaderenergy.ru
xxxkaz.org yeartwit.com
xxxtubesafari.com youdesigner.kz
youjizz.vc youjizz.vc
your-bearings.com yourporn.com
yourmovies.pl youtubedownload.org
zakon-ob-obrazovanii.ru zaim-pod-zalog-krasnodar.ru
zarabotat-na-sajte.ru zeleznobeton.ru
zarenica.net zhacker.net
zawyna.ua zigarettenonlinekaufen2.bloog.pl
zhuravlev.info znakomstvaonlain.ru
zigarettenonlinekaufen.tumblr.com zocaparj.kz
zootoplist.com zoodrawings.com
zootravel.com zoosexart.com
zskdla.site zynax.ua
zvezdagedon.ru

478
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
000free.us 123any.com
100dollars-seo.com 1qingdao.com
108shot.com 5-steps-to-start-business.com
2345.com
2daytrendingnews.com
34.gs
4kplayer.pl
7fon.ru 7fon.ru
aa9bd78f328a6a41279d0fad0a88df1901.com 8si.ru
accessoires-mode.in aa8b68101d388c446389283820863176e7.com
adhome.biz aanapa.ru
ads-seo.men abclauncher.com
algarveglobal.com adobereader-free.ru
allblogroll.com adprotect.net
alldownload.pw adsloads.com
alloysteel.ru agysya.ru
alphavisions.net alekseevec.ru
animalrank.com andpolice.com
anonymousfox.co apartamentwroclaw.eu
aosheng-tech.com apisecretsaucebi-a.akamaihd.net
apelsinnik.website arendovalka.xyz
apiappenableinfo-a.akamaihd.net astrochicks.com
applyneedy.xyz atout-energie-69.com
astana.xxxkaz.org audiobangout.com
automatic-seo.com b2b-lounge.com
autotop.com.ua bablonow.ru
babyguns.ru babyguns.ru
beta.hotkeys.com bangla-chat-uk-paralud.ga
beyan.host.sk becuo.com
biteg.xyz bidr.trellian.com
bizru.info big-cash.net
blackle.com bigcities.org
blog.yam.com bikini-image.com
bobinoz.com bingo8888.com
boeuklad.com bio-optomarket.ru
bongacams.com blockety.co
bookmark4you.com.biz bmsco.co
brains2.biz bobba.dzaba.com
bumascloset.com buypanicdisorderpill.com
cand.jp buyviagraa.blogspot.com
candiceloves.us c-gainsbourg.com
carabela.com.do cablecar.us
cartierjusteunclou.xyz callmd5map.com
caulderwoodkennels.com carstrends2015.com
cheap-trusted-backlinks.com checkpagerank.net
cheapmoncler.pw chidporn.com
cleansearch.net chocolateslim-original.com
clubfashionista.com cialgenisrx.com
coin-hive.com cialischmrx.com
compliance-barak.xyz clickpapa.com
coop-gamers.ru clickzzs.nl
corta.co clippingphotoindia.com
covetnica.com clips.ua.ac.be
crazyprotocol.com clmforexeu.com
dailystrength.org commoncrawl.org
damasarenai.info compliance-irvin.xyz
dataloading.net coprofam.org
deda-moroza-zakaz.ru customcollegeessays.net
dengi-pod-zalog-nedvizhimosti.ru dairyindia.in
derevesendeco.com danceuniverse.ru
dktr.ru dear-diary.ru
dentfidemountpreach.tk
dieswaene.com
docstoc.com
documentbase.net
dodlive.mil dodlive.mil
domainsatcost.com dolohen.com
dosugrostov.site domaincdn.xyz
dprtb.com dtm-spain.com
drugs-no-rx.info dustyorate.com
drunkenstepfather.com
e-kwiaciarz.pl
ecommerce-seo.org
ecookna.com.ua
ednorxmed.com
ege-essay.ru
elektrischeziga.livejournal.com
elektrozigaretten2.yn.lt
elidelcream.weebly.com
elmifarhangi.com elmifarhangi.com
englate.com empirepoker.com
erectile.bid enskedesquashclub.se
erotik-kostenlos.net epicbrogaming.com
essayassist.com esnm.ru
extrabot.com exchangeit.gq
ezigarettenkaufen1.pagina.gr expolicenciaslatam.co
family1st.ca ezigarettekaufen2.mpbloggar.se
filmi-v.online ezrvrentals.com
flooringinstallation-edmonton.com f-online.de
folowsite.com familyholiday.ml
forodvd.com fast-torrent.ru
fickluder69.com
financehint.eu
firstdrugmall.ru
footbalive.org
forum69.info
fotopop.club
foxtechfpv.com foxtechfpv.com
ghernnqr.skyrock.com free411games.com
googlefeud.com freeuploader.com
gunsvicceadadebt.tk freeuploader.ml
hd-filmy.net freewareseek.com
headpress.ru galaxys6manual.info
hebr.myddns-flir.com gambarkataku.co
helpmymacfaster.trade gamerextra.com
hikesearch.net gay-file.com
homik.pw geilehausfrauen.net
host-protection.com get-seo-domain.com
host-tracker.com goldadpremium.com
ie.57883.net gox.com.ua
img.wallpaperstock.net gsasearchengineranker.xyz
gungamesz.com
gwhwpxbw.bloger.index.hr
handsandlegs.ru
hermesbelts.xyz
hotelcrocenzi.sm
hotenergy.ru
hspline.com
idc.com.ua
igithab.com
ihtec2019.org
ilovevitaly.com
imagefinder.site
indiakino.net
inet-traffic.com
int.search.mywebsearch.com int.search.mywebsearch.com
intelhdgraphicsgtdrive6w.metroblog.com investingclub.ru
jillepille.com irkutsk.zrus.org
jimmychoosale.online itis4you.com
junketjuice.blogspot.com itronics.ca
karadene.com itsdp3.com
kazinogames.lv ivanstroi.ru
kazka.ru jerseywholesaleelitestore.com
kichenaid.ru jewelryandfiligree.com
krasivoe-hd.net jolic2.com
lawyersinfo.org journalhome.com
legionalpha.com krynica.info
letto.by kutikomi.net
linkarena.com lankarns.com
live-cam6.info leboard.ru
liverpool.gsofootball.com lechenie-gemorroya.com
livingroomdecor.info legalrc.biz
ljusihus.se loadopia.com
locksmith.jp locimge.pw
lonerangergames.com longlifelomilomi.info
lost-alpha.ru m-google.xyz
louboutinbooties.xyz makemoneyonline.com
love-baby.cz manifestyourmillion.com
map028.com max-eclat.men
memberty.com med-zdorovie.com.ua
milblueprint.com medi-fitt.hu
moesexy.com medizinreisen.de
monsterdivx.com mega-bony2017.pl
moomi-daeri.com miniature.io
mirobuvi.com.ua
mojpregled.com
morpicert.pw
mp3ringtone.info mp3ringtone.info
muschisexbilder.com mrpornogratis.xxx
mxgetcode.com mug-na-chas-moscow.ru
mylovelibrabry.com myra.top
mystats.xyz myshopmatemac.com
nac-bearings.ru nfljerseyscheapbiz.us
nagdak.ru noumeda.com
namecrumilchlet.tk nowtorrents.com
naval.jislaaik.com nvformula.ru
notfastfood.ru obesidadealgarve.com
obiavo.kz offf.info
obiavo.ru ooo-gotovie.ru
oktube.ru openmultipleurl.com
onlineslotmaschine.com parlament.biz
osagonline.ru partner-host.men
paramountmarble.co.uk partner-print.men
partner-cdn.men parvezmia.xyz
pateaswing.com
pc4download.co
pechikamini.ru pechikamini.ru
perfection-pleasure.ru pharmacyincity.com
photosaga.info pierrehardysale.online
piluli.info pinkduck.ga
piratecams.com
pirateday.ru pirateday.ru
pizza-imperia.com piuminiita.com
pons-presse.com platesauto.com
pornmania.pl
pornoelita.info
pornofeuer.com
pornofiljmi.com
pornoklad.net
pornoted.com pornoted.com
pornozhara.com pornoxxx.com.mx
preparevideosafesystem4unow.site porto.abuilder.net
presleycollectibles.com pozdrawleniya.com
pro-okis.ru prize44.com
prostitutki-rostova.org prizeestates.cricket
ranksignals.com rabotaetvse.ru
rasteniya-vs-zombi.ru ranking2017.ga
rating-bestcasino.com rating-casino2021.ru
registry-cleaner.net referencemoi.com
relayblog.com relax.ru
remybutler.fr ritlweb.com
renecaovillasale.online ruf777.com
restaurantlescampi.com rybalka-opt.ru
rossanasaavedra.net safe-app.net
runstocks.com sale-japan.com
s1z.ru samara.rosfirm.ru
sack.net sanidumps.com
sampleletters.net scanner-jane.top
savingsslider-a.akamaihd.net sdsjweb.com
scanner-mark.top serviporno.com
search.1and1.com seryeznie-znakomstva.ru
seomarketings.online sex-sex-sex5.com
sexs-foto.com shoesonlinebuy.cn
sextracker.de shoplvlv.us
sexystrippe.info sildenafil-tadalafil.info
shakhtar-doneck.ru site-speed-checker.site
sherlock.se slowmac.tech
shisha-swag.de slutloadlive.com
shop-electron.ru smailik.org
signal03.ru smart-scripts.com
sim-service.net snegozaderzhatel.ru
sitevaluation.com soc-econom-problems.ru
smartpet.ru soietvousmaime.fr
socas.pluto.ro sovetogorod.ru
socialsignals24.com srgwebmail.nl
softomix.net store-rx.com
softxaker.ru superkanpo.com
sosdepotdebilan.com support.nopeas.sk
soundfrost.org
space-worry.ml
sports-supplements.us
startwp.org
stroydetali.ru
superoboi.com.ua
svbur.ru
swinginwithme.ru
tacbelarus.ru
tagil.zrus.org tagil.zrus.org
taranerymagesswa.blogspot.com takeprofitsystem.com
tasteidea.com taraz.xkaz.org
tattooha.com tattoo33.ru
tedxrj.com teastory.co
textads.men thetardistimes.ovh
thegameriders.com titelhelden.eu
timer4web.com tizanidinewithdrawalsymptoms.blogspot.com
tizanidinehcl4mgtabinfo.blogspot.com tizanidinezanaflex.blogspot.com
tizanidinevssoma.blogspot.com topvidos.ru
top-study.work trafficmp.com
top1-seo-service.com traffictrade.life
torrent-newgames.com trkdf.com
track112.site tryrating.com
turn-up-life.life unitexindia.com
tv-spoty.info usacasino.com
twojebook.pl ussearche.cf
uamtrk.com venta-prom.ru
unacittaconte.org verymes.xyz
underthesite.com vezuviy.su
uzungil.com vrnelectro.ru
videogamesecrets.com webextract.profound.net
videojam.tv wingsoffury2.com
visa-pasport.ru wjgony.com
vkmusics.ru xchangetrak.com
vodkoved.ru xn----7sbahjd3btneuw1joc.xn--p1ai
vut.com.ru xn--80aaafbn2bc2ahdfrfkln6l.xn--p1ai
w3data.co xxxtubesafari.com
watchdogs-2.ru youjizz.com
watracker.net yourmovies.pl
wdfdocando.com yoursite.com
weblibrary.win zawyna.ua
wesharepics.site zeleznobeton.ru
westermarkanjou.se zirondelli.it
woman-h.ru zootravel.com
woman-orgasm.ru zytpirwai.net
wosik-dach.service-for-web.de
x-lime.com
x-true.info
xxxkaz.org
yopoint.in
yourtemplatefinder.com
youtubedownload.org
zambini.ru
znakom.sibtest.ru
znakomstva-piter78.ru
zoogdiesney.com

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.96 0; 10.1.0.128 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2605 ### Version: V4.2021.09.2606
### Updated: Tue Sep 21 22:01:08 UTC 2021 ### Updated: Wed Sep 22 10:03:19 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2021.09.2605 ### Version: V4.2021.09.2606
### Updated: Tue Sep 21 22:01:08 UTC 2021 ### Updated: Wed Sep 22 10:03:19 UTC 2021
### Bad Bot Count: 623 ### Bad Bot Count: 623
################################################### ###################################################
### Version Information ## ### Version Information ##