vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-01 18:19:55 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2023.02.3589 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2023-02-25 22:06:55 +00:00
parent 96106474ed
commit 5672a003f9
46 changed files with 88504 additions and 88504 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3588 #### Version: V4.2023.02.3589
#### Bad Referrers Blocked: [7099](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7099](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2023.02.3588 #### Version: V4.2023.02.3589
#### Bad Referrers Blocked: [7099](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7099](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2023.02.3588 #### Version: V4.2023.02.3589
#### Bad Referrers Blocked: [7099](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7099](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10930
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2023.02.3588 #### Version: V4.2023.02.3589
#### Bad Referrers Blocked: [7099](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7099](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

10934
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10934
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

10934
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

10934
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10934
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10934
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10934
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3588 3589

10934
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

10934
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.84.174.128 0; 20.42.13.48 0;

10934
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10934
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10934
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

10934
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.84.174.128 0; 20.42.13.48 0;

10934
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.84.174.128 0; 20.42.13.48 0;

10934
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 1; 10.1.0.36 1;
127.0.0.1 1; 127.0.0.1 1;

20
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
AlphaBot Majestic SEO
Anthill Mata Hari
Atomseobot PageAnalyzer
CrazyWebCrawler Quick-Crawler
DatabaseDriverMysqli RankurBot
EirGrabber SurveyBot
Sitebeam Swiftbot
Turnitin WISENutbot
VB Project cah.io.community
VCI seostar

286
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
01h4x.com 360Spider
404checker 404checker
404enemy 404enemy
ADmantX AIBOT
Abonti ALittle Client
Aboundex Aboundexbot
Acunetix Acunetix
AfD-Verbotsverfahren
AhrefsBot
AiHitBot AiHitBot
Alexibot Aipbot
Alligator
AlphaBot AlphaBot
Anthill
Apexoo Apexoo
Aspiegel
Atomseobot Atomseobot
BDCbot Attach
AwarioRssBot
BackWeb BackWeb
Backlink-Ceck BacklinkCrawler
Bandit Bandit
BatchFTP Barkrowler
BetaBot Bigfoot
Blow Black Hole
BlackWidow
BlowFish
Bolt Bolt
Brandprotect Brandprotect
BuiltBotTough Buck
BunnySlippers Buddy
BuzzSumo
CCBot
CODE87 CODE87
CSHttp
CazoodleBot CazoodleBot
CensysInspect CensysInspect
CheTeam CherryPicker
CheeseBot Claritybot
ChinaClaw CopyRightCheck
Cogentbot Cosmos
Copyscape
Craftbot
Crawling at Home Project Crawling at Home Project
CrazyWebCrawler Crescent
DBLBot Curious
CyotekWebCopy
DSearch
DTS Agent DTS Agent
DataCha0s
Digincore
Disco Disco
Dispatch Discoverybot
DittoSpyder DittoSpyder
DomainAppender DomCopBot
DomainStatsBot DomainStatsBot
Domains Project
Dotbot Dotbot
Drip
ECCP/1.0 ECCP/1.0
EasyDL EMail Siphon
EMail Wolf
Ecxi Ecxi
Evil
Express WebPictures Express WebPictures
ExtLinksBot
Extractor
ExtractorPro ExtractorPro
Extreme Picture Finder Extreme Picture Finder
FemtosearchBot EyeNetIE
Foobot Ezooms
Freeuploader Flunky
Fuzz Fyrebot
Genieo
GermCrawler GermCrawler
GoZilla Getintent
Gigabot
Go!Zilla
Go-Ahead-Got-It
Gotit
Grabber Grabber
Grafula HMView
GrapeFX HTTrack
HTMLparser Havij
HTTP::Lite Hloader
Haansoft HybridBot
HaosouSpider
Heritrix
Humanlinks
IRLbot
Id-search
IlseBot IlseBot
Image Fetch
InfoTekies
Internet Ninja
InternetSeer InternetSeer
Iskanie Iskanie
JamesBOT IstellaBot
JetCar Jetty
JikeSpider JikeSpider
Jyxobot Jyxobot
Keyword Density
Kinza
LNSpiderguy
LWP::Simple LWP::Simple
Larbin Lanshanbot
Leap
LeechFTP
LibWeb LibWeb
Libwhisker Libwhisker
LieBaoFast Lightspeedsystems
LinkWalker LinkWalker
LinkextractorPro Linkbot
LinkpadBot LinkpadBot
LinksManager LinksManager
LinqiaMetadataDownloaderBot
Litemage_walker Litemage_walker
Ltx71 Ltx71
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool MIDown tool
MIIxpc MQQBrowser
MJ12bot MSFrontPage
MSIECrawler
Mag-Net
Magnet
Majestic SEO Majestic SEO
Majestic-SEO Masscan
Mb2345Browser
Meanpathbot
Mediatoolkitbot Mediatoolkitbot
MegaIndex.ru MegaIndex.ru
Metauri Microsoft Data Access
Microsoft URL Control
Minefield Minefield
Mojeek MolokaiBot
Msrabot
NPbot NPbot
Nameprotect
Navroad Navroad
NetAnts Needle
Net Vampire
NetLyzer NetLyzer
NetMechanic
NetSpider NetSpider
Netcraft NetZIP
Nettrack Nikto
Niki-bot Ninja
Nmap Nmap
Nutch Not
Offline Explorer Offline Explorer
OpenLinkProfiler Offline Navigator
OpenVAS OpenVAS
OrangeSpider Openvas
OrangeBot
OutclicksBot OutclicksBot
PHPCrawl PECL::HTTP
POE-Component-Client-HTTP
PageAnalyzer PageAnalyzer
PageThing.com PageScorer
Pandalytics Pandalytics
Papa Foto Pavuk
Pimonster PeoplePal
PleaseCrawl PictureFinder
Pockey Pockey
Quick-Crawler ProPowerBot
ProWebWalker
QueryN Metasearch
RankActiveLinkBot RankActiveLinkBot
RankFlex Rankivabot
RankingBot2 RankurBot
Re-re Re-re
ReGet RebelMouse
RepoMonkey RepoMonkey
Ripper Ripper
SEOkicks-Robot Rogerbot
SBIder
SEOlyticsCrawler SEOlyticsCrawler
SEOprofiler SEOstats
SISTRIX SMTBot
SalesIntelligent
ScanAlert ScanAlert
Scanbot Scanbot
ScoutJet ScoutJet
ScreenerBot Searchestate
SemanticJuice SearchmetricsBot
SemrushBot SentiBot
SeoSiteCheckup Seomoz
SeobilityBot Siphon
Site Sucker
SiteCheckerBotCrawler SiteCheckerBotCrawler
SiteExplorer
SiteLockSpider SiteLockSpider
SiteSnagger SiteSucker
Siteimprove
Sitevigil Sitevigil
SlySearch
SmartDownload SmartDownload
Snake Snapbot
Snoopy
SocialRankIOBot
Sogou web spider Sogou web spider
Sosospider Sosospider
SpaceBison Sottopop
SpankBot Spbot
Spanner Spinn3r
SputnikBot
Sqlmap Sqlmap
Steeler
Sucker Sucker
Sucuri
SuperHTTP SuperHTTP
Surfbot
SurveyBot SurveyBot
Swiftbot Swiftbot
Szukacz
T8Abot T8Abot
Telesoft Teleport
Telesphoreo Telesphoreo
TightTwatBot Titan
Toata Toata
Turingos Trendiction
Trendictionbot
Turnitin Turnitin
TwengaBot TurnitinBot
Typhoeus
URLy.Warning
Upflow
Vacuum Vacuum
VelenPublicWebCrawler VelenPublicWebCrawler
VeriCiteCrawler Virusdie
Voltron VoidEYE
WEBDAV Voil
WWW::Mechanize WPScan
WWWOFFLE WWW-Mechanize
WeSEE Wallpapers
Web Collage Wallpapers/3.0
WallpapersHD
Web Enhancer
Web Fuck Web Fuck
Web Pix
Web Sauger Web Sauger
Web Sucker Web Sucker
WebCollage WebCollage
WebEnhancer WebEnhancer
WebFetch WebFetch
WebFuck WebImageCollector
WebLeacher WebLeacher
WebPix
WebReaper WebReaper
WebSauger WebSauger
WebSucker WebWhacker
Whack Website Quester
Whacker WebsiteExtractor
Whatweb
Who.is Bot Who.is Bot
Widow Widow
Wonderbot WinHTTrack
Xenu Woobot
Zade Zeus
Zitebot Zitebot
ZmEu
ZoominfoBot ZoominfoBot
arquivo-web-crawler arquivo-web-crawler
arquivo.pt
autoemailspider autoemailspider
backlink-check backlink-check
check1.exe
clark-crawler
coccocbot
crawl.sogou.com crawl.sogou.com
dataforseo.com crawler4j
eCatch
evc-batch
facebookscraper facebookscraper
heritrix gopher
instabid instabid
internetVista monitor iubenda-radar
linkdexbot
lwp-trivial
magpie-crawler
meanpathbot
mediawords
muhstik-scan muhstik-scan
netEstate NE Crawler oBot
ripz plumanalytics
s1z.ru probe-image-size
seocompany.store scalaj-http
seostar
sexsearcher
siteripz
spyfu
sysscan sysscan
ubermetrics-technologies.com ubermetrics-technologies.com
webgains-bot webgains-bot
webpros.com webpros.com
x09Mozilla x09Mozilla
xpymep1.exe zauba.io

268
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
360Spider 360Spider
404checker
404enemy
ADmantX ADmantX
AIBOT ASPSeek
Abonti Abonti
Aboundexbot Aboundex
Anarchie AiHitBot
Alligator
AlphaBot
Anarchy99 Anarchy99
Asterias Anthill
AwarioRssBot Aspiegel
AwarioSmartBot
BBBike BBBike
BackDoorBot
Backlink-Ceck Backlink-Ceck
Badass Badass
Bandit Bandit
Barkrowler BatchFTP
Black Hole Blow
BlackWidow BlowFish
Boardreader Boardreader
Bolt
BotALot
Brandprotect Brandprotect
Brandwatch Buck
Bullseye
BunnySlippers BunnySlippers
CSHttp CSHttp
Calculon Cegbfeieh
CazoodleBot CheeseBot
CensysInspect CherryPicker
CheTeam ChinaClaw
Chlooe
Citoid Citoid
Claritybot Copier
Cloud mapping
Cocolyzebot
Cogentbot
Crawling at Home Project
CrazyWebCrawler
Crescent Crescent
CrunchBot
Curious Curious
DSearch CyotekWebCopy
DTS Agent DTS Agent
Deusu Deusu
DigitalPebble
Disco
Discobot Discobot
Discoverybot
DittoSpyder
DomCopBot
DomainCrawler DomainCrawler
Dragonfly DomainStatsBot
Download Wonder
ECCP/1.0 ECCP/1.0
EMail Siphon
Ebingbong Ebingbong
Ecxi
EroCrawler
Exabot Exabot
ExtLinksBot ExtLinksBot
ExtractorPro
EyeNetIE EyeNetIE
Ezooms Ezooms
FHscan
FemtosearchBot FemtosearchBot
FlashGet Fuzz
Fyrebot FyberSpider
G-i-g-a-b-o-t Genieo
GermCrawler
GetRight GetRight
Getintent Go!Zilla
Gigabot Grafula
Go-Ahead-Got-It
Grabber
GrapeFX GrapeFX
HTMLparser
HTTP::Lite HTTP::Lite
Harvest HTTrack
Havij
Hloader Hloader
HonoluluBot HonoluluBot
Id-search HybridBot
IstellaBot IDBTE4M
IDBot
IlseBot
Image Fetch
Image Sucker
Intelliseek
JOC Web Spider JOC Web Spider
JamesBOT JamesBOT
Joomla Jbrofuzz
Keybot Translation-Search-Machine JennyBot
LNSpiderguy Jetty
Jorgee
JustView
Jyxobot
Keyword Density
Kinza
Lanshanbot Lanshanbot
Larbin
Leap Leap
LeechFTP LeechFTP
Libwhisker LeechGet
Likse
LinkScan
Linkbot Linkbot
LinkpadBot
LinksManager LinksManager
LinqiaRSSBot LinqiaMetadataDownloaderBot
Lipperhey Spider Lipperhey Spider
Lmspider
MFC_Tear_Sample
MIDown tool
MIIxpc MIIxpc
MSIECrawler MSFrontPage
Magnet MTRobot
Mail.RU_Bot Majestic SEO
MauiBot Majestic12
MegaIndex.ru MarkWatch
Mass Downloader
Mata Hari
Mb2345Browser
Mediatoolkitbot
Metauri Metauri
MicroMessenger
Microsoft Data Access Microsoft Data Access
Moblie Safari Microsoft URL Control
Mister PiX
Mojeek
Morfeus Fucking Scanner
Mr.4x3
Msrabot
Musobot Musobot
NICErsPRO NICErsPRO
NPbot NPbot
Nameprotect Nameprotect
Nessus Nessus
Net Vampire
NetLyzer
NetMechanic NetMechanic
NetZIP
Netvibes Netvibes
NextGenSearchBot NextGenSearchBot
Nibbler
Niki-bot
Nikto
Nimbostratus Nimbostratus
Ninja
Not Not
Nutch Nutch
Offline Explorer
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
Openvas
OrangeSpider OrangeSpider
OutclicksBot
OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
PageAnalyzer
PageGrabber PageGrabber
PageThing.com PageThing.com
Pandalytics
Panscient Panscient
Pavuk Pavuk
PeoplePal PeoplePal
Petalbot
Pi-Monster Pi-Monster
PictureFinder Picsearch
Pimonster
PleaseCrawl PleaseCrawl
Pockey
ProWebWalker
Probethenet Probethenet
Psbot
Pump Pump
RankFlex QueryN Metasearch
RankActive
RankActiveLinkBot
RankingBot2
RankurBot
Re-re Re-re
ReGet
RealDownload RealDownload
RepoMonkey Recorder
RocketCrawler RedesScrapy
Rogerbot Rogerbot
SBIder SBIder
SEOkicks-Robot
SEOprofiler
SEOstats SEOstats
Scanbot SISTRIX
ScanAlert
Scrapy
Screaming
Searchestate
SearchmetricsBot SearchmetricsBot
Semrush Semrush
SeoSiteCheckup
SeobilityBot SeobilityBot
Seomoz Site Sucker
SiteCheckerBotCrawler SiteLockSpider
SiteSucker
Sitebeam Sitebeam
Sitevigil SlySearch
Snake Snake
Snapbot Sogou web spider
Sociscraper SpaceBison
Sosospider
SpankBot SpankBot
Spanner Spanner
Spinn3r
SputnikBot SputnikBot
Stripper Sqworm
Sucuri Steeler
SuperBot
Surfbot Surfbot
Suzuran SurveyBot
Szukacz Szukacz
TeleportPro
Telesoft Telesoft
Thumbor True_Robot
Toata Turingos
Tracemyfile
Trendictionbot
Turnitin Turnitin
TurnitinBot
Typhoeus
URLy Warning URLy Warning
URLy.Warning URLy.Warning
UnisterBot UnisterBot
Upflow V-BOT
VB Project VB Project
Vacuum Vacuum
Vagabondo Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler VeriCiteCrawler
WISENutbot Voltron
WWW-Collector-E WWW-Collector-E
WWW::Mechanize WeSEE
WWWOFFLE
WallpapersHD
Web Enhancer Web Enhancer
Web Fetch Web Sauger
Web Fuck
WebAuto
WebEnhancer WebEnhancer
WebFetch
WebFuck WebFuck
WebGo IS WebGo IS
WebPix WebSauger
WebReaper WebSucker
WebZIP WebZIP
WebmasterWorldForumBot
Webshag Webshag
Website Quester Website Quester
WebsiteQuester
Whack Whack
Whacker Whacker
Whatweb
Who.is Bot
Widow Widow
WiseGuys Robot WiseGuys Robot
Wonderbot Woobot
Xaldon WebSpider Wotbox
YoudaoBot Wprecon
Zade
Zermelo
Zeus Zeus
ZoomBot Zitebot
ZoominfoBot ZmEu
adscanner adscanner
autoemailspider arquivo-web-crawler
backlink-check arquivo.pt
check1.exe dataforseo.com
clark-crawler domainsproject.org
coccocbot eCatch
com.plumanalytics
crawler4j
facebookscraper
gopher
iubenda-radar iubenda-radar
lwp-request
lwp-trivial lwp-trivial
magpie-crawler
mediawords mediawords
page scorer muhstik-scan
pcBrowser oBot
plumanalytics plumanalytics
polaris version polaris version
ripz ripz
s1z.ru s1z.ru
seobility seobility
seocompany.store serpstatbot
seostar
sexsearcher sexsearcher
siteripz
sogouspider sogouspider
sp_auditbot
spyfu spyfu
trendiction.com tAkeOut
voyagerx.com voyagerx.com
webprosbot webgains-bot
x09Mozilla x22Mozilla
xpymep1.exe

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
12bet.com 7fon.ru
doska-vsem.ru eu-cookie-law.info
emerson-rus.ru izhevsk.xrus.org
montazhnic.ru laparfumotec.com
pozdravleniya-c.ru lenvred.org
scenarii-1-sentyabrya.uroki.org.ua oneshotdate.com
share-buttons-for-free.com placid-rounded-coast.glitch.me
v24s.net toursmaps.com
xn--b1addnj3cah.xn--p1ai xaylapdiendanang.com
zeroredirect6.com xlolitka.com

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
00go.com 00it.com
1000n1.ru 00webcams.com
101billion.com 0c47f8422d3f.com
108shot.com 0n-line.tv
2nt.ru 123any.com
40cg.com 1bet.com
55wmz.ru 1pamm.ru
7zap.com 1xbet4you.com
8jn.dating-app.ru 24videos.tv
8kisses.eu 3th.co.in
991mostfm.co.id 45en.ru
aanapa.ru 4inn.ru
absugars.com 999webdesign.com
acunetix-referrer.com aarbur.com
adbetclickin.pink abcdefh.xyz
adserver-e7.com academiacsmendoza.org
adultmeetonline.info actulite.com
adverster.com add-add.men
ahhjf.com addtoadd.men
aksonural.ru adexprts.com
aktivator-windows10.blogspot.com adf.ly
akusherok.ru ads-seo.men
ads.gold
adultactioncam.com
adviceforum.info
advocatemsk.ru
aero2.ru
albamargroup.com albamargroup.com
alert-hgd.xyz alfa-img.com
alf-img.com alfa9.com
alive-ua.com alfabot.xyz
ananumous.ru alienwheel.es
anapa-inns.ru all4invest.info
android-style.com all4wap.ru
animali.deagostinipassion.it allwidewallpapers.com
animebox.com.ua allwomen.info
anniemation.com amehdaily.com
apimountainbikei-a.akamaihd.net anabolics.shop
arvut.org anaseracresar.tk
asrvvv-a.akamaihd.net angel1777.kz
asscenihotosidea.blogspot.com anime.dougasouko.com
atout-energie-69.com anonymizeme.pro
atovh.local-finders.com anti-virus-removal.info
aproposde.com
aquarium-pro.ru
arcadepages.com
arenanews.com.ua
arkkivoltti.net
asrv-a.akamaihd.net
ausergrubhof.info
auspolice.com auspolice.com
autoplate.info autogrep.ru
avkzarabotok.info autolombard-krasnodar.ru
avon-ukraine.com
avtolombard-krasnodar.com avtolombard-krasnodar.com
azartmsl.com azlex.uz
badavit.com.ua badavit.com.ua
bazaronline24.ru balla-boo.se
bestwebsitesawards.com baltgem.com
bezlimitko.xyz batanga.net
beztuberkuleza.ru berlininsl.com
bget.ru bestofupload.info
bigpenisguide.com big-cash.net
bildsuche.ru biocn.dx.am
bimmerpost.com birzha-truda.eu
bingoporno.com bitcoins-live.ru
bizru.info bizcheapjerseyswholesalechina.com
bizzliving.com blog.koorg.ru
blackcurranthumidifiers.site blog.yam.com
bluesalt.co boltalko.xyz
bnt-team.com braip.com.br
boanonihaca.tk brillianty.info
bonzibuddi.com bryansk.zrus.org
boole.onlinewebshop.net buy-cheap-online.info
brandimensions.com buytizanidineonline.blogspot.com
brimstonehillfortress.org c1.onedmp.com
bristolhotel.com.ua callejondelpozo.es
budilneg.xyz callmd5map.com
bumble.cheapwebsitehoster.com calstaterealty.us
buntube.net cartierlove2u.xyz
callcafe.info cb.iphantom.com
candlewooddentalcentre.com.au ccbill.com
carina-sy.de cejewelry.xyz
carmuffler.net cenokos.ru
carsdined.org censys.io
catalogs-parts.com centre-indigo.org.ua
cbb1smartlist12.click cfjump.com
cbox.ws checkhit.com
cctva.tv children-learningreading.info
centraletermice.us chinaelitecheapjerseys.com
cfsrating.sonicwall.com chinese-amezon.com
cheapjerseysa.com chocolateslim-original.com
cheapjerseysfootballshop.com cienum.fr
check-host.net cleanersoft.com
chimiver.info clipartnew.com
chinajerseyswholesalecoupons.com
christianlouboutinreplica.win
christianlouboutinshoes.xyz
cibpenonptib.flu.cc
clarithromycin500mg.com
cleanmypc.com
clickintext.net
clothesforcash.com
club-lukojl.ru
club-musics.ru club-musics.ru
cmhomestayagency.com cmhomestayagency.com
codysbbq.com codq.info
compiko.info coldfilm.ru
compliance-alexa.top compliance-brian.top
compliance-george.xyz compliance-jack.top
compliance-ivan.top compliance-julia.top
compliance-john.top compliance-margo.top
compliance-julianna.top connectingsingles.com
compliance-willy.top conocer-sanabria.com
constantaservice.net constantaservice.net
coolwallpapers4k.info contentsexpress.com
copyrightclaims.org copblock.org
cornerstone-countertops.com costablue.xyz
cosmeticswomens-womensports.rhcloud.com counter.bloke.com
countercrazy.com crafthubs.com
crackzplanet.com cranly.net
credit-respect.ru creditwell.ru
cubs-jerseys.us czeshop.info
cvety24.by dailystrength.org
cymbaltaandtizanidine.blogspot.com dame-ns.kz
dangphoto.trade dbmkfhqk.bloger.index.hr
daptravel.com dealwifi.com
decenttools.com decoratinghomes.ga
delfin-aqua.com.ua detsky-nabytek.info
design-lands.ru dfwu1019.info
deutschehobbyhuren.net diarioaconcagua.com
digilander.libero.it dichvuvesinhhanoi.com
dnsrsearch.com diegolopezcastan.com
docstoc.com diffbot.com
dogsrun.net discovertreasure-a.akamaihd.net
domainsatcost.com divisioncore.com
dogbreedspicture.net
dogoimage.com
domaincrawler.com
dorratex.tn dorratex.tn
dotnetdotcom.org dosugrostov.site
dtm-spain.com downloads-whatsapp.com
e-poker-2005.com downvids.net
e90post.com drpornogratisx.xxx
eandsgallery.com dstroy.su
eas-seo.com dumuelave.xyz
ebooknovel.club efkt.jp
ekosmetyki.net.pl ege-essay.ru
elektrischeziga.livejournal.com electrik-avenue.com
elektrozigarettekaufen2.blogse.nl eleimgo.pw
elvenmachine.com elektrischezigarette1.blog.pl
empirepoker.com eloconcream.blogspot.com
enginebay.ru emediate.eu
enthuse.computernetworksonline.com erectile.bid
eropho.com eropornosex.ru
etur.ru es5.com
ewebarticle.info exoneration-project.us
excitacion.info extlabs.io
expertblog.info eyelike.com.ua
ezaz.info f-online.de
ezigarettenkaufen2.dreamwidth.org fable.in.ua
ezigaretteshop.webs.com face.hostingx.eu
ezrvrentals.com facialporntube.com
f-loaded.de fakehandbags.xyz
f00kclan.de fantasticpixcool.com
farmprofi.net fefo.gdn
fashion.stellaconstance.co femdom.twiclub.in
fatfasts-4tmz.com fetroshok.ru
fba-mexico.com filmetricsasia.com
feeriaclub.ru findanysex.com
fikasound.tk flauntyoursite.com
filerockstar298.weebly.com floridamhca.org
filesclub.net forex-indextop20.ru
firma-legion.ru fortunejack.com
fitnesspiks.com fr.netlog.com
flashahead.info free-fbook-traffic.com
fok.nl
footbalive.org
formulaantiuban.com
fotoxxxru.com
fotzen-ficken.com
franklinfire.co
free-share-buttons.blogspot.com
freecamdollars.com freecamdollars.com
freemaintenancesysforpcandmac.top freelinkbuilding.website.tk
freewareseek.com freenom.link
frighteningremain.cf fres-news.com
fuel-gas.com freshberry.com.ua
fym.com.ru frvo.alptandem.ru
fyxabomiw.ru fun-mobi.pl
g.starmoe.xyz funcrushgames.com
gamebackyard.com galaxy-family.ru
gallerylisting.com
gameplexcity.com
ge0ip.org ge0ip.org
girlsatgames.ru gemara.com
get-your-social-buttons.info
gkvector.ru gkvector.ru
goldadpremium.com golaya.pw
gombita.info gomusix.com
goodly.pro
goodwinmetals.co goodwinmetals.co
goosefishpost.bid goodwriterssales.com
gorodservis.ru googlsucks.com
gotomontenegro.net gopro-online.info
greatgrace.ru graphid.com
gsasearchengineranker.xyz gstatey.net
gsmtlf.ru gtopstats.com
gwebtools.com guardlink.com
hamacapty.com gyffu.com
hcate.com gymi.name
hccoder.info hanink.biz.ly
hdfreeporno.net hannasolution.ru
hdseriale.pl havepussy.com
helvetia.com.ua hazardky.net
homemature.net hebr.myddns-flir.com
hoste.octopis.com hermesreplica.pw
hoterika.com herokuapp.com
housekuba.org hidefiles.org
houseofrose.com historichometeam.com
houston-vikings.com homedecoguide.info
hully.altervista.org hostingclub.lk
hothor.se
hotxnights.info
housediz.com
humanorightswatch.org
hundejo.com
hyhj.info hyhj.info
iboss.com hystersister.com
ideawheel.com idegenvezeto.eu
ifmo.ru iframe-toloka.com
ihtec2019.org imagerydatabase.com
ikritikimou.gr imagui.eu
ilte.info
impresagaia.it
inboxdollars.com inboxdollars.com
indetiske.ya.ru infodocsportal.com
inome.com.ua infokonkurs.ru
int.search.tb.ask.com infowarcraft.ru
intim-uslugi.info inmate-locator.us
iopeninghours.co.uk innodgfdriverhm.aircus.com
ipostroika.ru insta-add.pro
iqoption-bin.com investzalog.ru
irunfar.com iphantom.com
jackwolfskinoutlet.online iusstf.org
japfm.com javatex.co.id
javrip.net je7.us
jennyfire.ru jennyfire.ru
jerseyswholesalechinalimited.com joyceblog.top
jestr.org junketjuice.blogspot.com
jetsli.de kambasoft.com
johannesburgsingles.co.za kartiniresto.com
johnnyhaley.top keenoutlet.online
juliadiets.com keywords-monitoring-success.com
juliaworld.net
justprofit.xyz
kalb.ru
kandidos.com
kasino-money.pw
katjimej.blog.fc2.com
kemerovo.zrus.org
keywordhouse.com
keywordsuggest.org keywordsuggest.org
kimcurlrvsms.com kimcurlrvsms.com
kinozapas.com kinoflux.net
kmd-pto.ru kiwe-analytics.com
konoplisemena.com kletkimehan.ru
kotaku.com kmgamex.cf
kupit-adenu.ru knigonosha.net
kursy-ege.ru krasivoe-hd.net
la-fa.ru lanadelreyfans.us
lacapilla.info landmania.ru
landreferat.ru landscaping.center
lawyersinfo.org lashstudia.ru
legalrc.biz law-check-nine.xyz
lezbiyanki.net leboard.ru
lirunet.ru lexixxx.com
littlecity.ch li-er.ru
loginduepunti.it limads.men
lonfon.xyz lincolntheatre.com
lulea-auktionsverk.se lisque.batcave.net
lunamedia.co littleberry.ru
macdamaged.tech livingcanarias.com
magicdiet.gq lob.com.ru
managerpak204.weebly.com localflirtbuddies.com
mature.free-websites.com lockerz.com
me-ke.com lodki-pvh.dp.ua
mearns-tractors.co.uk long-beach-air-conditioning.com
menetie.ru louboutinbooties.xyz
message-warning.net lovelycraftyhome.com
militarysale.pro lunchrock.co
miniature.io machicon-akihabara.info
minneapoliscopiers.com mailemedicinals.com
missclub.info mainhp.net
mmofreegames.online makedo.ru
mnogolok.info manuscript.su
mobile.ok.ru marsgatan.com
moesonce.com maslenka.kz
money-for-placing-articles.com mebel-vstroika.ru
morpicert.pw med-zdorovie.com.ua
moskva.nodup.ru mediafresh.online
most-kerch.org medtherapy.ru
mowser.com meetlocalchicks.com
mp3downloadhq.com mega-bony-2017.pl
mulberryoutletonlineeu.com megaindex.ru
musezone.su merryhouse.co.uk
musicktab.com mhi-systems.ru
musicstock.me mini.7zap.com
myfreemp3.eu misslike.ru
mymobilemoneypages.com mobicover.com.ua
myperiod.club modabutik.ru
narkologiya-penza.ru mompussy.net
nodding-passion.tk moncleroutletonline.xyz
novatech.vn most.gov.iq
novodigs.com moxo.com
o-o-6-o-o.ru mrbojikobi4.biz
obiavo.by musicas.baixar-musicas-gratis.com
obiavo.com mykings.pw
ochistka-stokov.ru myseoconsultant.com
odywpjtw.bloger.index.hr naturalbreakthroughsresearch.com
offer.wpsecurity.website nelc.edu.eg
offers.bycontext.com netcheckcdn.xyz
netoil.no
networkad.net
nightvision746.weebly.com
njkmznnb.ru
obiavo.uz
okayimage.com
okmusic.jp
oktube.ru
omgtnc.com omgtnc.com
onclickpredictiv.com onlinemeetingnow.com
online-podarki.com oops-cinema.ru
online-x.ru ops.picscout.com
onlywoman.org optibuymac.com
onthemarch.co orsonet.ru
orgasmus-virtual.com ossmalta.com
osagonline.ru outshop.ru
panicatack.com ozas.net
partnerline.men painting-planet.com
pepperstyle.ru pandarastore.top
pewit.pw park.above.com
phobia.us partner-trustworthy.men
pc4download.co
penisvergrotendepillennl.ovh
pensplan4u.com
perl.dp.ua
peugeot-club.org
photokitchendesign.com
photorepair.ru
picture-group.com
pierrehardysale.online pierrehardysale.online
pihl.se pinwallpaper.top
pipki.r.acdnpro.com pirateday.ru
poems.com.ua piulatte.cz
pooleroadmedicalcentre.co.uk pizdeishn.com
popmarker.com pl-vouchers.com
poptool.net plastjulgranar.se
porn555.com playtap.us
porngalleries.top pobeiranie.pl
porno-raskazy.ru popcash.net
pornokajf.com poquoson.org
pornpost.in porn-w.org
portadd.men porndairy.in
positive2b.ru pornofeuer.com
pornoplen.com
potoideas.us potoideas.us
powc.r.ca.d.sendibm2.com
predominant-invent.tk predominant-invent.tk
preventheadacheguide.info pro-okis.ru
prizeestates.cricket profolan.pl
prohoster.info prom23.ru
projefrio.com.br prostitutki-tyumeni.org
pronekut.com proxyradar.com
proposal-engine.com pushdata.sendpulse.com
prosmibank.ru
prostitutki-kharkova.org
pshare.biz
putevka24.ru
pwwysydh.com pwwysydh.com
pyrodesigns.com.au quelle.ru
quick-seeker.com
rankings-analytics.com rankings-analytics.com
ratemodels.net ranksignals.com
rating-bestcasino.com
realtytimes.com
recipedays.ru recipedays.ru
recordpage-a.akamaihd.net rednise.com
remont-comp-pomosh.ru
remont-mobile-phones.ru remont-mobile-phones.ru
rentaremotecomputer.com retreatia.com
rfd-split.hr ringporno.com
rfserial.net rossanasaavedra.net
rusvideos.su sajatvelemeny.com
rybalka-opt.ru sbetodiodnye-lampy.ru
sape.top scanner-alex.top
saveriopiazza.it scanner-brian.top
scottbywater.com scanner-jane.top
seinterface.com scanspyware.net
seoboxes.com scenicmissouri.us
seorank.info sdrescher.net
seotoolsagency.com search.alot.com
servisural.ru see-your-website-here.com
sethrollins.net seeingmeerkat.com
sexobzor.info sel-hoz.com
seminarygeorgia59.ga
seolab.top
serpstat.com
sexkontakteao.info
sexs-foto.top
sextracker.be
sexytrend.ru sexytrend.ru
shama-rc.net shakhtar-doneck.ru
share-buttons-for-free.com share-buttons-for-free.com
sharebutton.net sharebutton.to
sheerseo.com shikiso.info
shemalegalls.blogporn.in shoesonlinebuy.cn
shisha-swag.de sideeffectsoftizanidine.blogspot.com
sinestesia.host.sk sim-service.net
sims-sims.ru
sirpornogratis.xxx sirpornogratis.xxx
sispe.com.br sitopreferito.it
skuteczna-dieta.co.pl skylta.com
sms2x2.ru sms2x2.ru
snworks.com smstraf.ru
softomix.net snip.tw
softtor.com snyatie-lomki-v-stacionare.ru
sogimlecal.tk soc-proof.su
soochi.co socialmadesimple.com
spabali.org solicita.info
space4updating.win sonnikforme.ru
spanking.to soserfis.com
speechfoodie.com spacash.com
speeddream.xyz spinazdrav.ru
speedup-my.site spravka-medosmotr.ru
spywarenuker.com stairliftsarea.com
stair.registrydomainservices.com stevemonsen.com
stal-rulon.ru stillmiracle.com
stanthonyscatholicchurch.org stoki.ru
star61.de streamin.to
statistici.ro subj.ukr-lit.com
stats-public.grammarly.io sudexpert66.ru
steebook.com sugarkun.com
stefanbakosab.se suhanpacktech.com
stockquotes.wooeb.com sukirgenk.dvrlists.com
stpicks.com superoboi.com.ua
stpolice.com superstats.com
strana-solnca.ru sureone.pro
strigkaomsk.ru swiped.su
strongsignal-a.akamaihd.net tamada69.com
styro.ru tastyfoodideas.com
summerlinhomes411.info taximytishi.ru
szucs.ru tech4master.com
telegraf.by thedownloadfreeonlinegames.blogspot.com
templates.radiodigital.co theguardlan.com
thefds.net theheroes.ru
tizanidineandgabapentin.blogspot.com threecolumnblogger.com
tizanidinehclsideeffects.blogspot.com titan-cloud.life
tjkckpytpnje.com titslove.yopoint.in
tk-assortiment.ru tizanidinesleepaid.blogspot.com
tobeyouday.win tizanidineusedfor.blogspot.com
tonerbox.kz tizanidinevssoma.blogspot.com
top10-way.com tn811.us
topanasex.com toloka.hurtom.com
tracksz.co tonivedu.it
traffic2money.com toolsky.com
trahic.ru top10registrycleaners.com
topsiteminecraft.com
torrentred.games
torrents.life
treasuretrack-a.akamaihd.net
triplepanda.xyz triplepanda.xyz
triumf-realty.ru tubeoffline.com
truck-addzilla.life turizmus.us
trustaffs.com
trustl.life
ttrraacckkrr.com
ttsq.fr
tvoystartup.ru tvoystartup.ru
u-cheats.ru twiclub.in
udayavani.com twitlinks.com
ukrtextbook.com u555u.info
undergroundcityphoto.com ua-company.ru
unpredictable.ga uhdtv.website
unrealcommander.biz ukrtvory.in.ua
urengoy.pro unacittaconte.org
uroffer.link unisexjewelry.org
uytmaster.ru uptimebot.net
valsalud.com urccvfmc.bloger.index.hr
vapomnoncri.tk urdoot.win
vedomstvo.net urll.eu
vekzdorov.ru uroki.net
vepad.com usadacha.net
viagraneggrx.com ussearche.cf
videotuber.ru vavilone.com
viel.su vchulkax.com
vilingstore.net vduplo.ru
vip-file.com viagengrarx.com
vipms.ru videochat.cafe
vipsexfinders.com videochat.world
vizag.kharkov.ua virus-respirators.com
vksaver-all.ru vk-mus.ru
vladhistory.com volume-pills.biz
vladimir.zrus.org vseuznaem.com
vrnelectro.ru vzlomtw.com
vrotike.ru wanker.us
vseigry.fun warezaccess.com
vykup-avto-krasnodar.ru webmasterhome.cn
w3data.co website-speed-checker.site
walpaperlist.com weburlopener.com
wareseeker.com whatsupinfoley.com
warezkeeper.com whereverdesperate.gq
wcb.su wholesalejerseysgaa.com
webcamdevochka.com wholinkstome.com
weclipart.com winterclassichockeyjerseys.com
wedding-salon.net wnoz.de
wedding0venues.tk wpsecurity.website
westermarkanjou.se x-mix.info
whatistizanidine2mg.blogspot.com x69ty.ru
wheelchairliftsarea.com xaylapdiendanang.com
wholesalecheapjerseysfree.com xn--hxazdsfy.blogspot.com
wingsofrefuge.net xsfetish.org
wiosenny-bon-1500.pl xxart.ru
womens-journal.net xxxguitars.com
word-vorlagen.xyz y8games-free.com
works.if.ua yaaknaa.info
wormix-cheats.ru yaminecraft.ru
ww2awards.info
x-rates.ru
xchangetrak.com
xlovecam.com
xmronta.com
xn--80aaagvmjabrs1aoc9luc.xn--p1ai
xn--80aaaks3bbhabgbigamdr2h.xn--p1ai
xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai
xn--80aeahghtf8ac5i.xn--p1ai
xn--80ajjbdhgmudixfjc8c5a9df8b.xn--p1ai
xn--e1aggki3c.xn--80adxhks
xn--lifehacer-1rb.com
xn--oogle-wmc.com
xrus.org
xtubeporno.net
yatrk.xyz
youjizz.vc youjizz.vc
yourserverisdown.com yourads.website
yousense.info youtuhe.com
yugk.net ytmnd.com
yugo-star.ru yun56.co
zapatosenventa.info zapatosenventa.info
zarabotok--doma.ru zebradudka.com
zastroyka.org zeroredirect.com
zeroredirect7.com zoodrawings.com
zone-kev717.info zoogdinsney.com
zootravel.com
zscalertwo.net

488
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
18ps.ru 2000k.ru
1kdailyprofit.me
202ch.com
2nt.ru 2nt.ru
34.gs 6128786.com
7minuteworkout.com 8558l.jobs.net
7wind.ru 8lufu.com
7xc4n.com aad1f4acb0a373420d9b0c4202d38d94fa.com
83net.jp abctoppictures.net
DomainStatsBot abovetherivernc.com
aa8780bb28a1de4eb5bff33c28a218a930.com
abcdefh.xyz
acmebtn.ml acmebtn.ml
addblueoff.com.ua adexprts.com
adservme.com admeasures.com
agahinameh.com albuteroli.com
albumsuper.info alert-hgd.xyz
allboard.xobor.de alice.it
anal-acrobats.hol.es allstatesugarbowl.org
angry-fermi-7633.arukascloud.io anapa-inns.ru
appleid-verification.com appearance-cool.com
architecturebest.com apxeo.info
artavenuegardenstudios.com arenda-yeisk.ru
atout-energie-69.com asociatia-tipografilor-transilvania.ro
aversis.net asrvrep-a.akamaihd.net
azbukafree.com atovh.local-finders.com
backlink4u.net autobrennero.it
beauby.ru autorn.ru
best-seo-solution.com avon-severozapad.ru
bestssaker.com avon-ukraine.com
betterscooter.com azbuka-mo.ru
bmwhighperformers.com babyguns.ru
bolezniorganov.ru battleforupdating.site
btvn.ru beremenyashka.com
buyessay3.blogspot.ru bildsuche.ru
buyfriend.ru bimatoprost-careprost.com.ua
buytizanidineonline.blogspot.com biographiya.com
cactussoft.biz bitcoin-ua.top
canacopegdl.com blog-hits.com
cartierbracelet.xyz bloggedporn.com
cbb1smartlist12.click boazpower.com
clipartnew.com burn-fat.ga
coccoc.com buscarfoto.com
cocolyze.com cartierjusteunclou.xyz
compliance-don.xyz christianlouboutinsaleoutletonline.us
compliance-josh.top cialgenisrx.com
compliance-nelson.top clickaider.com
cookingmeat.ru clicksor.net
coolgramgoods.com coolyarddecorations.com
counter.bloke.com crazyprotocol.com
craftinsta.ru creditnation.ru
creditmoney.com.ua crest-poloski.ru
customsua.com.ua cryptoswap.biz
cyclobenzaprinevstizanidine.blogspot.com customboxes4less.com
darkages.info cwetochki.ru
date-withme.com cxpromote.com
davidovic.info cymbaltaandtizanidine.blogspot.com
delitime.info cypernhuset.se
cyse.tk
dailystrength.org
daooda.com
datadepths.com
dealighted.com
deanmoore.ie
deniven.1bb.ru deniven.1bb.ru
discovertreasurenow.com deplim.com
dosugrostov.site detki-opt.ru
drivotracker.com deutschland123.de
drugs-no-rx.info digitalassetmanagement.site
easync.io dispo.de
divisioncore.com
djstools.com
domain2008.com
dorratex.tn
e-collantes.com
elektronischezi.livejournal.com elektronischezi.livejournal.com
elektrozigaretten1.tumblr.com energy-ua.com
elektrozigaretten1.webs.com englishgamer.com
eluxer.net erotische-geschichten-xxl.com
eurosamodelki.ru estelight.ru
evrotekhservis.ru eu-cookie-law.info
ezigaretteshop2.vefblog.net europeanwatches.ru
facebook-mobile.xyz excitacion.info
failingmarriege.blogspot.com faz99.com
fetroshok.ru filmania-x.ru
filmfanatic.com financehint.eu
foodcrafts.website forex-indextop20.ru
foodgid.net foto-weinberger.at
formulaf1results.blogspot.com fres-news.com
formularz-konkurs.tk freshdz.com
forum-engineering.ru fuel-gas.com
free-stock-illustration.com gambarkatabaru.com
freecamdollars.com gay-site.store
freshberry.com.ua ge0ip.com
freza-sverlo.ru gelezki.com
galleryawesome.com generousdeal-a.akamaihd.net
game-top.su genetworx.com
get-free-social-traffic.com globatur.ru
get-free-traffic-now.com globetrotting-culture.ru
getlaid-xxxhookupdirect.com gogps.me
gne8.com golmau.host.sk
goblacked.com goodly.pro
goodnightjournal.com gsasearchengineranker.pw
gotomontenegro.net hamptonoaks.ca
h2monline.com hdimagelib.com
habermetre.com hdpixion.com
hahashka.ru hdwalls.xyz
handsandlegs.ru hem.passagen.se
hausfrauensex18.com heygidday.biz
haveinc.xyz homedecoguide.info
hchha.com homemade.gq
hdxnxxtube.mobi houdom.net
hjaoopoa.top houston-vikings.com
hledejvshopech.cz hoverboardforsaledirect.com
homedo.fabpage.com htmlcorner.com
hubbble.com ikearugs.xyz
i-midias.net.br ilmexico.com
igtools.club ilovevitaly.ru
iinstalll-fii1leis.jus0wil.pp.ua infobanks.ru
imagecoolpub.com infowarcraft.ru
imagine-ex.co irkutsk.zrus.org
imperia31.ru iusstf.org
incest-ru.com jackwolfskinoutlet.online
intim-uslugi.info jolic2.com
javspace.net keywordspay.com
jyrxd.com kirov.zrus.org
kaac.ru lashstudia.ru
kellyonline.xyz laudit.ru
kletkimehan.ru lazymanyoga.com
krasnodar-avtolombards.ru lida-ru.com
krynica.info lineavabit.it
labvis.host.sk lordzfilmz.me
ladsup.com mafa-free.com
landscaping.center mararoom.ru
lasvegaslockandsafe.com marketingtechniques.info
letsart.ru meddesk.ru
linkbux.ru meendo-free-traffic.ga
liveinternet.ru megaapteka.ru
livingcanarias.com menetie.ru
loadopia.com middlerush-a.akamaihd.net
lyrster.com
manipulyator-peterburg.ru
medosmotr-ufa.ru
meet-flirt-dating.com
megaindex.ru
merryhouse.co.uk
missclub.info
mmoguider.ru
mobsfun.net mobsfun.net
moncleroutletonline.pw mojaocena.com
myra.top mompussy.net
mzdish.site moscow.xrus.org
nalogovyy-kodeks.ru mossmesi.com
narutonaruto.ru music.utrolive.ru
naughtyconnect.com muzaporn.com
navalwiki.info myiptest.com
netcheckcdn.xyz myplaycity.com
newsperuse.com niki-mlt.ru
newtechspb.ru oakridgemo.com
nina.az oconto.ru
nzfilecloud.weebly.com offer.wpsecurity.website
offtime.ru onlainbesplatno.ru
oneminutesite.it onlinemegax.com
onetravelguides.com openmultipleurl.com
onlineku.com osb.se11.ru
ordernorxx.com ostrovtaxi.ru
ororodnik.goodbb.ru palvira.com.ua
ourville.info partners-ship.pro
park.above.com pekori.to
patterntrader-en.com pic2fly.com
photokitchendesign.com piratecams.com
picquery.com piulatte.cz
pills24h.com platesauto.com
plohaya-kreditnaya-istoriya.ru pliks.pl
poppen-nw.net porndairy.in
pornhubforum.tk porno-video-chati.ru
pornotubs.com pornosexrolik.com
potolokelekor.ru pornowarp.info
profitkode.com pravoholding.ru
pskcijdc.bloger.index.hr prefersurvey.net
rankexperience.com preg.marketingvici.com
regdefense.com privat-girl.net
renecaovillasale.online randki-sex.com
responsinator.com ranksonic.org
rostov.xrus.org rating-bestcasino.com
runtnc.net recipedays.com
s-iwantyou.com reeyanaturopathy.com
sajatvelemeny.com roleforum.ru
scanner-andrew.top scenarii-1-sentyabrya.uroki.org.ua
scanner-willy.top scenicmissouri.us
sdsjweb.com schoolfiles.net
search-goo.com se-welding.ru
search.1and1.com searchengineranker.email
seomarketings.online searchinterneat-a.akamaihd.net
serviporno.com searchtooknow-a.akamaihd.net
sims-sims.ru selectads.men
site-speed-checker.site sell-fb-group-here.com
skincrate.net seo-smm.kz
smallseotools.com seoprofiler.com
spabali.org sfj-ror.no
spin2016.cf shohanb.com
sportbetfair.com sibvitr.ru
sr-rekneskap.no sindragosa.comxa.com
stauga.altervista.org site-analyzer.com
sukarame.net sofit-dmd.ru
superinterstitial.com spidtest.space
supermesta.ru stal-rulon.ru
supernew.org stoki.ru
szamponrevita.pl streha-metalko.si
telefonsexkostenlos.tk subj.ukr-lit.com
tfxiq.com superfish.com
thebestweightlosspills.ovh surfbuyermac.com
thegolfclub.info surintech.ac.th
thiegs.reco.ws svnuppsalaorebro.se
time-japan.ru swinger-mobil.net
timetorelax.biz td-l-market.ru
tizanidineandgabapentin.blogspot.com technopellet.gr
tizanidineandhydrocodone.blogspot.com telefonsex-ohne0900.net
tizanidinepain.blogspot.com telegramdownload10.com
tksn.ru telesvoboda.ru
top10-way.com thepornsex.org
toproadrunner5.info tizanidine4mgstreetvalue.blogspot.com
toy-shop.top topads.men
toys.erolove.in topsy.com
trafficmania.com treasuretrack-a.akamaihd.net
ttrraacckkrr.com twittrading.com
turist-strani.ru u17795.netangels.ru
tyumen.xrus.org ublaze.ru
ua-company.ru ukr-lit.com
ua.tc unimodemhalfduplefw.pen.io
uasb.ru unlimitdocs.net
unionmarkt.de usadacha.net
valoresito.com vabasa.inwtrade.com
vekzdorov.ru vedomstvo.net
velpanex.ru video-production.com
vertaform.com viel.su
vgoloveboli.net vigrx-original.ru
videochat.bz vipsiterip.org
videooko.weebly.com vk-mus.ru
videositename.com vkgaleria.com
vinylvault.co.uk
vod.com.ua
voloo.ru
voltrknc1.com voltrknc1.com
webalan.ru vzubah.com
webkeyit.com w-journal.ru
webshoppermac.com webnode.me
websitevaluebot.com wingsoffury2.com
wejdz-tu.pl winterclassichockeyjerseys.com
wholesalejerseychinaoutlet.com www.solartek.ru
worldhistory.biz xcombear.ru
wpthemedetector.co.uk xn-----7kcabaipgeakzcss7bjdqdwpfnhv.xn--p1ai
www.event-tracking.com xn----7sbaaabaei0cc8aj5bj0bncejx.xn--p1ai
xblognetwork.com xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
xjrul.com xn--24-glceagatoq7c2a6ioc.xn--p1ai
xn--80aebbcbcdemfkhba4byaehoejh8dza3v.xn--p1ai xn--80aikhbrhr.net
xvideosbay.com xn--l1aengat.xn--p1ai
youbloodyripper.com xnxx-n.com
zeroredirect10.com youradexchange.com
zigzog.ru yourads.website
zzbroya.com.ua zeroredirect8.com
zocaparj.kz
zoogdinsney.com
zootoplist.com
zophim.me

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.40 0; 10.1.0.36 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3588 ### Version: V4.2023.02.3589
### Updated: Sat Feb 25 10:03:26 UTC 2023 ### Updated: Sat Feb 25 22:01:05 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2023.02.3588 ### Version: V4.2023.02.3589
### Updated: Sat Feb 25 10:03:26 UTC 2023 ### Updated: Sat Feb 25 22:01:05 UTC 2023
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##