vikarti.anatra/nginx-ultimate-bad-bot-blocker
1
0
Fork 0
mirror of https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker.git synced 2025-09-03 19:20:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

V4.2022.10.3399 [ci skip]

Browse source
This commit is contained in:
mitchellkrogza 2022-10-09 10:09:13 +00:00
parent 57c4504567
commit 3b771806f6
46 changed files with 90696 additions and 90696 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
AUTO-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3398 #### Version: V4.2022.10.3399
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
MANUAL-CONFIGURATION.md
View file

@ -4,7 +4,7 @@
### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation: ### PLEASE READ CONFIGURATION INSTRUCTIONS BELOW THOROUGHLY :exclamation:
_______________ _______________
#### Version: V4.2022.10.3398 #### Version: V4.2022.10.3399
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

2
README.md
View file

@ -14,7 +14,7 @@
##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots) ##### The Ultimate Nginx Bad Bot, User-Agent, Spam Referrer Blocker, Adware, Malware and Ransomware Blocker, Clickjacking Blocker, Click Re-Directing Blocker, SEO Companies and Bad IP Blocker with Anti DDOS System, Nginx Rate Limiting and Wordpress Theme Detector Blocking. Stop and Block all kinds of bad internet traffic even Fake Googlebots from ever reaching your web sites. [PLEASE SEE: Definition of Bad Bots](#define-bad-bots)
_______________ _______________
#### Version: V4.2022.10.3398 #### Version: V4.2022.10.3399
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11202
_generator_lists/bad-ip-addresses.list
View file

File diff suppressed because it is too large Load diff

2
_google_analytics_ghost_spam/README.md
View file

@ -3,7 +3,7 @@
# EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM # EASY CONFIGURATION INSTRUCTIONS FOR STOPPING GOOGLE ANALYTICS "GHOST" SPAM
_______________ _______________
#### Version: V4.2022.10.3398 #### Version: V4.2022.10.3399
#### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list) #### Bad Referrers Blocked: [7097](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-referrers.list)
#### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list) #### Bad User-Agents (Bots) Blocked: [637](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/bad-user-agents.list)
#### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list) #### Fake Googlebots Blocked: [217](https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/_generator_lists/fake-googlebots.list)

11206
_sample_config_files/Engintron_for_cPanel_WHM_Configuration_Example/etc/nginx/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11206
conf.d/globalblacklist-testing-version.conf
View file

File diff suppressed because it is too large Load diff

11206
conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

11206
dev-tools/beta_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

11206
dev-tools/beta_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

11206
dev-tools/beta_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/beta_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/beta_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

11206
dev-tools/beta_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/buildnumber
View file

@ -1 +1 @@
3398 3399

11206
dev-tools/globalblacklist-testing.template
View file

File diff suppressed because it is too large Load diff

11206
dev-tools/globalblacklist.template
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.6.64 0; 20.7.78.143 0;

11206
dev-tools/test1_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ip_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

11206
dev-tools/test1_conf_files_ip_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_ratelimiting/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_ratelimiting/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

11206
dev-tools/test1_conf_files_ratelimiting/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test1_conf_files_whitelist/bots.d/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

2
dev-tools/test1_conf_files_whitelist/bots.d/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

11206
dev-tools/test1_conf_files_whitelist/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test2_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.6.64 0; 20.7.78.143 0;

11206
dev-tools/test2_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test3_conf_files/bots.d/whitelist-ips.conf
View file

@ -45,4 +45,4 @@
# - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions. # - Whitelisting IP's and RANGES here ONLY affects the IP blocking functions.
# - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks. # - This file will NOT allow your own IP to bypass bad User-Agent or Referrer String checks.
# - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST # - To bypass everything for a certain IP see notes in blockbots.conf on SUPER WHITELIST
40.86.6.64 0; 20.7.78.143 0;

11206
dev-tools/test3_conf_files/conf.d/globalblacklist.conf
View file

File diff suppressed because it is too large Load diff

2
dev-tools/test_units/blacklist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 1; 10.1.0.66 1;
127.0.0.1 1; 127.0.0.1 1;

18
dev-tools/test_units/random-bots-for-test-quick.list
View file

@ -1,10 +1,10 @@
Atomseobot BDCbot
BLEXBot Boardreader
Keybot Translation-Search-Machine GT::WWW
NetZIP Gotit
PHPCrawl GrapeshotCrawler
SEOstats IndeedBot
Sqlworm JikeSpider
Toweyabot LinkScan
Re-re
Typhoeus Typhoeus
Web Sucker

296
dev-tools/test_units/random-bots-for-test.list
View file

@ -1,250 +1,250 @@
404enemy 404enemy
ADmantX AIBOT
ALittle Client Aboundex
ASPSeek Aboundexbot
AhrefsBot AfD-Verbotsverfahren
Anarchy Aipbot
Alexibot
AllSubmitter
Anarchy99
Ankit Ankit
Aspiegel Aspiegel
Asterias AwarioRssBot
BDFetch BLEXBot
BatchFTP BackWeb
Bigfoot Badass
Bandit
BetaBot
Black Hole Black Hole
BlackWidow BlackWidow
Boardreader
Bolt Bolt
BuiltBotTough BotALot
Brandwatch
Buck
BuiltWith
Bullseye Bullseye
BuzzSumo BuzzSumo
CCBot CSHttp
Calculon Cegbfeieh
ChinaClaw CensysInspect
Chlooe Cocolyzebot
Claritybot
Cliqzbot
Collector Collector
Copier
CopyRightCheck CopyRightCheck
Cosmos Craftbot
CrazyWebCrawler Crawling at Home Project
CrunchBot
Custo
CyotekWebCopy CyotekWebCopy
DBLBot DIIbot
DTS Agent DSearch
DataCha0s
DatabaseDriverMysqli DatabaseDriverMysqli
Devil Devil
DigitalPebble
Dirbuster Dirbuster
Disco Dispatch
Discobot DomainAppender
Discoverybot DomainSigmaCrawler
DnBCrawler-Analytics
DomainStatsBot DomainStatsBot
Domains Project
Dotbot
Dragonfly Dragonfly
EasyDL Drip
Ecxi ECCP/1.0
EirGrabber EMail Wolf
EroCrawler EroCrawler
Evil Evil
Express WebPictures Extreme Picture Finder
ExtLinksBot
ExtractorPro
EyeNetIE EyeNetIE
FDM
FHscan
Fimap
Firefox/7.0 Firefox/7.0
Flunky Flunky
FyberSpider Fuzz
G-i-g-a-b-o-t
GetRight GetRight
GetWeb GetWeb
Getintent Getintent
Gigabot Gigabot
GoZilla GoZilla
Gotit
GrabNet
Grabber
Grafula
GrapeFX
GridBot
HEADMasterSEO
HMView
HTTP::Lite HTTP::Lite
HTTrack HTTrack
HaosouSpider Harvest
Havij Havij
Heritrix Heritrix
Hloader
HonoluluBot HonoluluBot
IDBot IDBTE4M
IRLbot IRLbot
Image Fetch IlseBot
Indy Library Indy Library
Intelliseek Intelliseek
InterGET JamesBOT
Internet Ninja
InternetSeer
Jbrofuzz Jbrofuzz
Joomla
JustView
Jyxobot
Kenjin Spider Kenjin Spider
Keyword Density
LeechFTP LeechFTP
LeechGet
LexiBot
Lftp
LibWeb LibWeb
Lipperhey Spider LieBaoFast
Lmspider Likse
Ltx71 LinkpadBot
LinqiaRSSBot
LinqiaScrapeBot
Lipperhey
MIIxpc MIIxpc
MSIECrawler MSIECrawler
Mag-Net MTRobot
MarkWatch Majestic SEO
Majestic-SEO
Masscan Masscan
MauiBot Mb2345Browser
MicroMessenger Meanpathbot
Microsoft Data Access Microsoft Data Access
Mister PiX
Mojolicious Mojolicious
Mr.4x3 Musobot
Name Intelligence NPbot
Nameprotect Nameprotect
Navroad Navroad
NearSite
Needle
NetAnts
NetZIP NetZIP
Netvibes
NextGenSearchBot
Nikto Nikto
NimbleCrawler NimbleCrawler
Ninja Not
Nuclei Octopus
Nutch Offline Explorer
OpenLinkProfiler Offline Navigator
Openfind OpenVAS
OutclicksBot OutfoxBot
PageGrabber PageGrabber
PageThing.com PageThing.com
Panscient
Papa Foto Papa Foto
Pavuk PeoplePal
Pi-Monster
Picscout Picscout
Picsearch Picsearch
Pimonster
PleaseCrawl PleaseCrawl
Pockey Pockey
ProPowerBot
ProWebWalker
Probethenet Probethenet
Pu_iN
Pump Pump
RSSingBot PxBroker
RankFlex PyCurl
RankingBot2 QueryN Metasearch
Re-re RankActiveLinkBot
Reaper RankingBot
RebelMouse RankurBot
Recorder ReGet
RedesScrapy
RepoMonkey
Ripper
Rogerbot Rogerbot
SBIder SBIder
SEOkicks SEOkicks-Robot
SEOlyticsCrawler
SEOstats
SISTRIX
ScanAlert ScanAlert
Scanbot Scrapy
ScoutJet Seekport
Screaming
SearchmetricsBot
SemanticJuice SemanticJuice
SemrushBot
SeobilityBot
Seomoz
Shodan
Site Sucker Site Sucker
SiteExplorer SiteExplorer
SiteSnagger
SiteSucker SiteSucker
Siteimprove Sitebeam
Sitevigil
SmartDownload
Snake Snake
Snapbot
Snoopy Snoopy
Sociscraper Sociscraper
Sottopop Sosospider
SpankBot SpankBot
Spanner
Spbot Spbot
Steeler Sqlworm
Stripper
Sucker Sucker
Suzuran SuperHTTP
T0PHackTeam SurveyBot
T8Abot Szukacz
TeleportPro Teleport
Telesoft
Telesphoreo
Telesphorep Telesphorep
The Intraformant
TheNomad TheNomad
TightTwatBot Thumbor
Titan
Toata Toata
Toweyabot Toweyabot
Tracemyfile
Trendiction Trendiction
True_Robot Turnitin
Turingos Typhoeus
Twice URLy Warning
URLy.Warning URLy.Warning
Upflow V-BOT
Vacuum VB Project
Vagabondo
VelenPublicWebCrawler
VeriCiteCrawler
VoidEYE
Voltron Voltron
WEBDAV
WPScan
WWW-Collector-E
WWW-Mechanize WWW-Mechanize
WallpapersHD WWW::Mechanize
Web Fetch
WebBandit
Web Collage Web Collage
WebCopier Web Enhancer
Web Fuck Web Fuck
Web Sucker
WebAuto
WebEnhancer
WebGo IS
WebImageCollector WebImageCollector
WebLeacher WebLeacher
WebReaper WebPix
WebSauger
WebStripper WebStripper
WebSucker
WebWhacker WebWhacker
Webalta WebZIP
Webshag WebmasterWorldForumBot
Website Quester
WebsiteExtractor WebsiteExtractor
Webster WebsiteQuester
Whack Widow
Whacker
Whatweb
WinHTTrack WinHTTrack
Wonderbot WiseGuys Robot
Wotbox Xaldon WebSpider
Wprecon YoudaoBot
Xaldon_WebSpider Zauba
Xenu Zermelo
Zeus
Zitebot
ZmEu ZmEu
ZoominfoBot
ZumBot ZumBot
ZyBorg ZyBorg
archive.org_bot archive.org_bot
arquivo.pt arquivo-web-crawler
autoemailspider autoemailspider
cah.io.community clark-crawler
check1.exe
coccocbot coccocbot
crawl.sogou.com com.plumanalytics
crawler4j
demandbase-bot
domainsproject.org domainsproject.org
facebookscraper
gopher gopher
heritrix heritrix
internetVista monitor
ips-agent
linkdexbot
magpie-crawler magpie-crawler
netEstate NE Crawler mediawords
plumanalytics
probe-image-size
ripz ripz
s1z.ru satoristudio.net
scalaj-http
scan.lol scan.lol
seobility seobility
seoscanners
seostar seostar
sexsearcher
sitechecker.pro sitechecker.pro
sp_auditbot sp_auditbot
spyfu trendiction.com
sysscan voyagerx.com
trendiction.de webprosbot
webgains-bot x22Mozilla
zauba.io

304
dev-tools/test_units/random-bots-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
01h4x.com
360Spider
404checker
404enemy
80legs 80legs
ADmantX
AIBOT AIBOT
ALittle Client ASPSeek
Abonti
Acunetix Acunetix
AfD-Verbotsverfahren AiHitBot
AhrefsBot
Aipbot Aipbot
Alligator AllSubmitter
Anarchy
Anarchy99
Anthill
Asterias Asterias
Atomseobot Atomseobot
Attach
AwarioSmartBot AwarioSmartBot
BDFetch BDFetch
BackWeb
Backlink-Ceck Backlink-Ceck
Badass Bandit
Bigfoot BatchFTP
Bitacle Blow
Black Hole BotALot
Boardreader
Brandprotect
Brandwatch Brandwatch
Buddy Buck
BuiltBotTough BuiltWith
Bullseye
BunnySlippers BunnySlippers
CATExplorador
CSHttp CSHttp
Calculon CheeseBot
CazoodleBot ChinaClaw
Chlooe Chlooe
Citoid Claritybot
Cocolyzebot
Collector
Copyscape
Cosmos Cosmos
Craftbot
CrazyWebCrawler CrazyWebCrawler
Crescent
CrunchBot CrunchBot
Custo Curious
CyotekWebCopy CyotekWebCopy
DSearch
DTS Agent
DataCha0s DataCha0s
Demon DatabaseDriverMysqli
Devil DigitalPebble
Disco Disco
Discoverybot Discoverybot
DittoSpyder
DnBCrawler-Analytics
DomainSigmaCrawler DomainSigmaCrawler
DomainStatsBot Dotbot
Drip ECCP/1.0
EMail Wolf EasyDL
Ebingbong
Ecxi Ecxi
EirGrabber EirGrabber
EyeNetIE Evil
FDM Exabot
FemtosearchBot Extractor
Fimap Ezooms
Flunky Firefox/7.0
Foobot FlashGet
Freeuploader GermCrawler
Fyrebot Gigabot
GalaxyBot GrabNet
Go!Zilla Grafula
Go-Ahead-Got-It GridBot
Gotit HTMLparser
Grabber HaosouSpider
HTTrack HonoluluBot
Harvest IDBTE4M
Heritrix Id-search
HybridBot
Iblog
Image Fetch
IndeedBot
InfoNaviRobot
InfoTekies InfoTekies
Intelliseek
InterGET InterGET
Internet Ninja InternetSeer
IstellaBot Jbrofuzz
JOC Web Spider JetCar
Joomla JikeSpider
Jorgee
JustView
Keybot Translation-Search-Machine Keybot Translation-Search-Machine
Kozmosbot
LNSpiderguy LNSpiderguy
Lanshanbot
Larbin
Leap Leap
LeechFTP
LexiBot
Libwhisker Libwhisker
LieBaoFast
Lightspeedsystems Lightspeedsystems
LinkScan LinkScan
LinkextractorPro LinkextractorPro
LinksManager LinqiaMetadataDownloaderBot
LinqiaRSSBot LinqiaRSSBot
LinqiaScrapeBot LinqiaScrapeBot
Lipperhey Lipperhey
Ltx71 Litemage_walker
MFC_Tear_Sample MFC_Tear_Sample
MIDown tool
MJ12bot
MSIECrawler MSIECrawler
MTRobot MTRobot
Magnet Mag-Net
Mail.RU_Bot
Majestic SEO Majestic SEO
Majestic-SEO Majestic-SEO
Majestic12 Majestic12
MarkWatch Mass Downloader
Masscan Masscan
Mata Hari
Mb2345Browser Mb2345Browser
Meanpathbot Meanpathbot
Mediatoolkitbot
MegaIndex.ru
Metauri Metauri
Microsoft URL Control
Minefield
Mister PiX
Moblie Safari Moblie Safari
Mojolicious
MolokaiBot MolokaiBot
Morfeus Fucking Scanner Mr.4x3
Mozlila Msrabot
Musobot NICErsPRO
NPbot Net Vampire
Name Intelligence
Nameprotect
Navroad
NetLyzer NetLyzer
NetMechanic NetMechanic
NetZIP
Netvibes Netvibes
Nikto NimbleCrawler
Nimbostratus Ninja
Not Nmap
Nuclei
Nutch
Offline Explorer Offline Explorer
Offline Navigator
OnCrawl OnCrawl
OpenLinkProfiler
OpenVAS OpenVAS
OrangeSpider OrangeBot
OutfoxBot OutfoxBot
PECL::HTTP PECL::HTTP
PHPCrawl
POE-Component-Client-HTTP
PageGrabber
PageScorer
Pandalytics Pandalytics
Petalbot Petalbot
Pi-Monster
Picsearch
Piepmatz Piepmatz
Pimonster Pockey
Pixray Pump
ProPowerBot PxBroker
Psbot
PyCurl PyCurl
QueryN Metasearch Quick-Crawler
RSSingBot RankurBot
RankFlex
RankingBot
RankingBot2
Re-re Re-re
RealDownload Reaper
RebelMouse Recorder
RepoMonkey RedesScrapy
Ripper Ripper
RocketCrawler
SBIder
SEOkicks
SEOkicks-Robot SEOkicks-Robot
SEOprofiler SEOprofiler
SMTBot SEOstats
SISTRIX
ScoutJet
Scrapy
ScrepyBot ScrepyBot
SemanticJuice
Semrush Semrush
SemrushBot
SentiBot SentiBot
SeobilityBot SeoSiteCheckup
Seomoz Seomoz
Siphon Site Sucker
SiteExplorer SiteCheckerBotCrawler
SiteLockSpider SiteLockSpider
Sitebeam Sitebeam
Siteimprove Snapbot
SlySearch Sosospider
Snake Spammen
Snoopy Spinn3r
SocialRankIOBot Sqlworm
Sociscraper
SpaceBison
Sqlmap
Steeler Steeler
Stripper
Sucker Sucker
Surfbot SuperHTTP
TeleportPro Suzuran
T8Abot
Telesoft Telesoft
Telesphorep Telesphoreo
Thumbor TightTwatBot
Titan
Toata Toata
Trendictionbot Trendictionbot
Typhoeus True_Robot
Vacuum Turingos
TwengaBot
Twice
URLy.Warning
Upflow
VCI
VelenPublicWebCrawler VelenPublicWebCrawler
VidibleScraper VeriCiteCrawler
VoidEYE VoidEYE
WASALive-Bot WASALive-Bot
WBSearchBot
WEBDAV WEBDAV
WISENutbot WWW-Collector-E
WPScan
WWW-Mechanize WWW-Mechanize
WWW::Mechanize WWW::Mechanize
Wallpapers Wallpapers/3.0
WallpapersHD WallpapersHD
WeSEE WeSEE
Web Auto
Web Enhancer
Web Pix
Web Sucker
WebAuto
WebCollage
WebEnhancer
Web Fuck Web Fuck
WebBandit
WebCopier
WebFetch
WebGo IS
WebImageCollector
WebPix WebPix
WebSauger WebWhacker
WebZIP
Webshag
WebsiteQuester WebsiteQuester
WebsiteExtractor WinHTTrack
WebsiteQuester
Webster
Whacker
Whatweb
WiseGuys Robot WiseGuys Robot
Wonderbot Wprecon
Xaldon_WebSpider YoudaoBot
Xenu Zermelo
Zade
Zauba
Zeus Zeus
Zitebot
ZmEu ZmEu
arquivo.pt ZoominfoBot
backlink-check arquivo-web-crawler
coccocbot autoemailspider
clark-crawler
cognitiveseo
com.plumanalytics com.plumanalytics
crawler4j
dataforseo.com dataforseo.com
domainsproject.org
evc-batch
facebookscraper facebookscraper
internetVista monitor gopher
heritrix
instabid
ips-agent ips-agent
isitwp.com isitwp.com
linkdexbot iubenda-radar
lwp-trivial magpie-crawler
meanpathbot
mediawords mediawords
netEstate NE Crawler netEstate NE Crawler
oBot
plumanalytics plumanalytics
satoristudio.net
scan.lol scan.lol
seobility seobility
sexsearcher seocompany.store
sitechecker.pro sysscan
sp_auditbot ubermetrics-technologies.com
tAkeOut
trendiction.de
webmeup-crawler
x09Mozilla

20
dev-tools/test_units/random-referrers-for-test-quick.list
View file

@ -1,10 +1,10 @@
babyfactory.fr cityadspix.com
custom-electric-guitar.com darmebel.com.ua
cwetochki.ru googglet.com
goodhumor24.com j33x.com
high-speed1.net kaidalibor.de
threecolumnblogger.com letmacworkfaster.site
tsstcorpcddvdwshbbdriverfb.aircus.com ops.picscout.com
vellings.info takeprofitsystem.com
vip.51.la tizanidinevscyclobenzaprine.blogspot.com
zscalertwo.net welcomeauto.ru

934
dev-tools/test_units/random-referrers-for-test.list
View file

@ -1,500 +1,500 @@
1008.su 1-99seo.com
11235813.webzdarma.cz 1bet.com
123any.com 1flag.co.za
123locker.com 28n2gl3wfyb0.ru
2000k.ru 2ads.co.uk
214jaluwobapef.cf
2itech.ru 2itech.ru
4replicawatch.net 2x2fan.ru
4webmasters.org 4kepics.com
5elementov.ru 5forex.ru
5kstore.com 66cpwgln.space
5ws.dating-app.ru 9icmzvn6.website
7makemoneyonline.com a.pr-cy.ru
8558l.jobs.net aa8780bb28a1de4eb5bff33c28a218a930.com
991mostfm.co.id aa9d046aab36af4ff182f097f840430d51.com
9med.net absolute-s.ru
aarbur.com actionnooz.com
aasoldes.fr admeasures.com
abcdeg.xyz
abtasty.com
acc.eu.org
adinterax.com
adobereader-free.ru
adpremium.org adpremium.org
adventureparkcostarica.com ads-seo.men
adverster.com adultgalls.com
affiliate-fr.com advancedmassagebysara.com
ageofclones.com agysya.ru
aghanyna.com alert-fdm.xyz
albamargroup.com alienwheel.es
albuteroli.com all4invest.ru
alert-fjg.xyz allproblog.com
alive-ua.com allwidewallpapers.com
allbizne.co.ua alpharma.net
alot.com alpinism.ru
ameblo.jp amateurmatch.com
anabolics.shop ameblo.top
animaltoplist.com ampower.me
anime.dougasouko.com anal-acrobats.hol.es
animenime.ru anti-virus-removal.info
apifasterlightin-a.akamaihd.net apifasterlightin-a.akamaihd.net
apiholdingmypage-a.akamaihd.net appiq.mobi
apple.com-cleaner.systems apple.com-webbrowsing-security.review
aproposde.com architecturebest.com
arabseyes.com
arenanews.com.ua
arenda-yeisk.ru
arendatora.ru arendatora.ru
asmxsatadriverin.aircus.com arkartex.ru
asseenontv.ru arpelsreplica.xyz
auto-zapchasti.org artparquet.ru
autobudpostach.club asiengirls.net
autodo.info asrv-a.akamaihd.net
autoloans.com atleticpharm.org
autotop.com.ua atofilms.com
backlinkwatch.com azbukafree.com
badavit.com.ua backlink4u.net
baksman.com balook.com
balkanfarma.org batanga.net
battle.net benchmarkcommunications.co.uk
bcmp.org berdasovivan.ru
beauby.ru best-seo-software.xyz
bedcapdealers.com
belsetirehafi.tk
best-way.men
besthomemadepornsites.com
bestmarriages.com bestmarriages.com
bestwaystogetpaid.us bfz.biz
betune.onlinewebshop.net bharatdefencekavach.com
biketank.ga bigpictures.club
bildsuche.ru
bimatoprost-careprost.com.ua bimatoprost-careprost.com.ua
biographiya.com bioca.org
bird1.ru blpmovies.com
blockworld.ru
bloggedporn.com
bluesalt.co
bmsco.co bmsco.co
bonzybuddy.com bolezniorganov.ru
boostmyppc.com brains2.biz
brewdom.ru bristolhotel.com.ua
brothers-smaller.ru buckspar.gq
buehne-fuer-menschenrechte.de bumble.cheapwebsitehoster.com
bumskontakte.org businesxxl.com
buntube.net buyessay3.blogspot.ru
bvlgaribracelet.xyz buypillsorderonline.com
c-english.ru bvlgariring.xyz
carsdined.org bzero1jewelry.net
casinopinup-wins.com camdolls.com
changde.58.com candycrushshop.com
chatroulette.online canrioloadm.gq
cheapcigarettesc.info canzoni.ru
cheapjerseysa.com cartujano-pre.de
cheapmarmot.online cementaresearch.se
cheapmoncler.xyz centrdebut.ru
cheapsergiorossi.online cerev.info
chinacheapelitejerseys.com cheapestjerseys-wholesale.com
cienum.fr checkm8.com
clash-clans.ru checkpagerank.net
cleanallspyware.com cialisovercounteratwalmartusa.com
clickbank.net ciekawinki.pl
com-secure.download civilwartheater.com
compliance-nelson.top clarithromycin500mg.com
coolyarddecorations.com clean-start.net
copro.pw cleansearch.net
counciltally.com clevernt.com
countbertwistdisp26.soup.io clodo.ru
crackguru.tk cntravelre.com
cranly.net compliance-viktor.top
contentlook.co
continent-e.tv
counter.bloke.com
courtshipgift.com
crd.clan.su crd.clan.su
creditmoney.com.ua creditmoney.com.ua
cubook.supernew.org creditwell.ru
customcollegeessays.net cs-passion.pl
czeshop.info customchocolate.business-for-home.com
dailyfinancefix.com customerguru.in
damasarenai.info cxpromote.com
data.vtc.pw cypernhuset.se
d0t.ru
daooda.com
data-ox.com
dawlenie.com
day-news.info
ddospanel.com
deadlinkchecker.com deadlinkchecker.com
deluxedumps.com decorationspcs.com
detmebel.su dekoration.us
deutschland123.de dengi-pod-zalog-nedvizhimosti.ru
dev-seo.blog detskie-zabavi.ru
digest-project.ru
digital-sale.su
digitalfaq.com digitalfaq.com
divku.ru divatraffic.com
dmmspy.com djekxa.ru
dms-sw.ru dms-sw.ru
dneprsvet.com.ua doctissimo.fr
dnsrsearch.com documentsite.net
docstoc.com doktoronline.no
dolohen.com domini.cat
dominterior.org dominterior.org
driving.kiev.ua dostavka-v-ukrainu.ru
downloader12.ru
downtuptv.gq
droidlook.net
drunkenstepfather.com drunkenstepfather.com
dtnlyss.com drupa.com
easytuningshop.ru dstroy.su
editors.choice6912650.hulfingtonpost.com dumpsmania24.com
ee77ee.com editmedios.com
efkt.jp
ekobata.ru ekobata.ru
elite-sex-finders.com elektrischeziga.livejournal.com
elkacentr.ru enge-fotzen.info
energydiet24.ru enternet.ee
englate.com eralph.tk
erotik-kostenlos.net erotiktreff24.info
ertelecom.ru
es5.com
escortplius.com escortplius.com
eu-cookie-law.blogspot.com etotupo.ru
eurosamodelki.ru europeanwatches.ru
exmasters.com exchanges-bet.com
extrabot.com executehosting.com
ezigarettekaufen2.mpbloggar.se extremez.net
ezigarettenkaufen1.deviantart.com facecup.top
faithe.top fapgon.com
fakehandbags.xyz favorcosmetics.com
falllow.gq fidelityfunding.com
faracontrol.ir fil.ru
fastcrawl.com fillmewithhappiness.com
fatmaelgarny.com finejewelryshop.ru
faz99.com flashlarevista.com
fba-mexico.com forpackningsutveckling.se
fellowshipoftheminds.com fototravel.eu
fifa-coins.online franch.info
filmetricsasia.com freejabs.com
firstsiteguide.com fxtips.ru
fitness-video.net g33.org
fix-website-errors.com gamebackyard.com
flagcounter.me gamerextra.com
flauntyoursite.com games.kolossale.ru
foto-basa.com getpopunder.com
foxinsocks.ru gifspics.com
fres-news.com gilbertbanda.net
fugarif.ga globalscam.ga
fxgallery.com gobongo.info
gagrasector.ru goodly.pro
gallerylisting.com google-liar.ru
gambarkatabaru.com gsasearchenginerankerdiscount.com
gambarxkata.co gta-top.ru
gamerscorps.com gymi.name
gamesprite.me h2monline.com
gay-site.store hatedriveapart.com
gazporno.com heartofpayne.xyz
given2.com help.tpu.ru
gojiberriess.apishops.ru hexpilot.com
gok-kasten.net hit-kino.ru
googlemare.com hobild.net
gotwebsite1.com homegardenlova.com
gratuitxblcodes.com
greenzaim.ru
grupografico-pilar.com.ar
gsmtlf.ru
guitar-master.org
gyffu.com
hackers-crackers.tk
hayate.biz
hdwalls.xyz
heartofbeijing.blogspot.com
hitcpm.com
homik.pw homik.pw
hornymatches.com
horoshieokna.com
hostsshop.ru
hotblognetwork.com hotblognetwork.com
hotxnights.info hotgirlhdwallpaper.com
housekuba.org howmuchdoestizanidinecost.blogspot.com
houston-vikings.com hunterboots.online
hoverboard360.at ideawheel.com
howlongdoestizanidinestayinyoursystem.blogspot.com
huhn.altervista.org
hustoon.over-blog.com
igadgetsworld.com
images.gyffu.com
imagine-ex.co
imitex-plus.ru imitex-plus.ru
inboundlinks.win incolors.club
incomekey.net
indiasourcemart.in
infazavr.ru infazavr.ru
inmate-locator.us instasexyblog.com
intellego.info interfucks.net
intim-uslugi.info investsuccess.org
irkutsk.online-podarki.com investzalog.ru
irkutsk.zrus.org invivo.hu
iscblog.info iopeninghours.co.uk
isitwp.com ipchicken.com
iskalko.ru isoveti.ru
istanbulit.com itservicesthatworkforyou.com
iweblist.info ivanovo.zrus.org
javspace.net jackwolfskinoutlet.online
jav-fetish.site
javxxx18.com
je7.us
jewelryandfiligree.com jewelryandfiligree.com
job.icivil.ir jobmarket.com.ua
joingames.org justkillingti.me
jrpmakati.com jwcialislrt.com
justdating.online jwss.cc
kaac.ru kadashihotel.com
kalb.ru kalb.ru
kanimage.com kam-dom.ru
kemerovo.zrus.org kazrent.com
ketoanhanoi.info kellyonline.xyz
keywordglobal.co.za keywesthideaways.co
kihi.gdn keywords-monitoring-success.com
kimcurlrvsms.com kino-filmi.com
kinomaniatv.pl kinosed.net
kinostorm.net konkursowo-24.pl
kiskinhouse.com kotaku.com
kiwi237au.tk kupit-adenu.ru
knigonosha.net kursy-ege.ru
kongruan.com lafourmiliaire.com
krasnodar-avtolombards.ru landinez.co
kutikomi.net languagecode.com
kvartiry-remont.ucoz.ru lapitec.eu
labelwater.se lazymanyoga.com
lacloop.info letmacworkfaster.site
landscaping.center lexiangwan.com
lankarns.com linkarena.com
lechenie-gemorroya.com linkbuddies.com
lennatin.info live-sexcam.tk
letto.by livejasmin.com
liupis.com liver-chintai.org
liverpool.gsofootball.com lmrauction.com
livesex-amateure.info loadopia.com
louboutinreplica.pw lollypopgaming.com
luckyday.world lomb.co
lyrster.com lombia.co
m292.info lubetube.com
macdamaged.tech lulea-auktionsverk.se
mackeeper-land-672695126.us-east-1.elb.amazonaws.com mabdoola.blogspot.com
magazintiande.ru macdamaged.space
magicalfind-a.akamaihd.net mafa-free.com
maglid.ru manve.info
malwareremovalcenter.com maridan.com.ua
marblestyle.ru markjaybeefractal.com
marcowebonyodziezowe.pl martlinker.com
maximpartnerspr.com maslenka.kz
medispainstitute.com.au master-muznachas.ru
mensandals.xyz matpre.top
mesbuta.info max-eclat.men
mhtr.be mebel-vstroika.ru
mini-modus.ru meble-bogart.info
mirtorrent.net mega-bony-2017.pl
mirzonru.net mega-bony2017.pl
mister-shop.com metalonly.info
mnogabukaff.net mezaruk.info
mockupui.com mil-stak.com
morepoweronmac.trade missclub.info
moscow-region.ru missis.top
mototsikl.org modenamebel.ru
mrmoneymustache.com monclercheap.xyz
mrwhite.biz moncleroutletonline.pw
mundoaberrante.com moneymaster.ru
musicdaddy.net moneyviking-a.akamaihd.net
mwtpludn.review motherboard.vice.com
mydownloadengine.com mountainstream.ms
mydownlodablefiles.com mrinsidesales.com
myfreetutorials.com msk-diplomat.com
mygameplus.com musezone.ru
myhealthcare.com musicktab.com
myindospace.com muz-tracker.net
mymobilemoneypages.com myfreemp3.eu
nacap.ru myftpupload.com
netallergy.ru mypornfree.ru
networkcheck.xyz myseoconsultant.com
newstudio.tv nac-bearings.ru
nomuos.it narkologiya-voronezh.ru
nash-krym.info
nelc.edu.eg
newsperuse.com
newstraveller.ru
nhl09.ru
nicefloor.co.uk
nonameread45.live
novosibirsk.xrus.org
nowtorrents.com
nyfinance.ml nyfinance.ml
o-o-6-o-o.com o.light.d0t.ru
odywpjtw.bloger.index.hr offers.bycontext.com
oklogistic.ru officedocuments.net
offtime.ru
okroshki.ru okroshki.ru
onclickpredictiv.com online-sbank.ru
onlinadverts.com onlineslotmaschine.com
onlinemeetingnow.com
onlythegames.com onlythegames.com
orhonit.com openfrost.net
ostroike.org oren-cats.ru
ozas.net osagonline.ru
paidonlinesites.com pacificair.com
pateaswing.com panchro.xyz
pc4download.co paperwritingservice17.blogspot.ru
pcads.ru parajumpersoutlet.online
pdamods.ru perl.dp.ua
peekyou.com phuketscreen.com
photochki.com pic-re.blogspot.com
photosaga.info piccshare.com
picturesify.com picmoonco.pw
pinkduck.ga piratecams.com
pio.polytopesexempt.com pixshark.com
piter.xrus.org plastjulgranar.se
pix24x7.com play.leadzupc.com
pizda.lol playboyfiles.xblog.in
pizdeishn.com pokemon-go-play.online
platinumdeals.gr pons-presse.com
playlott.com porn-w.org
poiskzakona.ru pornhub-forum.ga
popserve.adscpm.net pornofiljmi.com
porno-home365.com pornoklad.net
pornophoto.xyz pornoslive.net
pornotubs.com porto.abuilder.net
prefersurvey.net privatbank46.ru
pregnant.guru prizefestival.mobi
printingpeach.com pro-poly.ru
professionalwritingservices15.blogspot.ru pro-tec.kz
promodj.com push-ad.com
psa48.ru qwesa.ru
pushdata.sendpulse.com rankia.com
pussyspace.net rapidgator-porn.ga
puteshestvennik.com razamicroelectronics.com
putitin.me realtytimes.com
qcstrtvt.bloger.index.hr rebelmouse.com
quickcashlimited.com rebuildermedical.com
rainbowice.ru recipedays.ru
rankings-analytics.com rednise.com
recordpage-a.akamaihd.net registrysweeper.com
redhotfreebies.co.uk remontvsamare.su
regionshop.biz remote-dba.de
relatodelpresente.com.ar renecaovilla.online
rennlist.com restaurantlescampi.com
ribieiendom.no resultshub-a.akamaihd.net
rixpix.ru rewardit.com
rockprogblog.com riders.ro
rp9.ru ringporno.com
ruclicks.com rockma.se
running-line.ru rossanasaavedra.net
runstocks.com runtnc.net
rus-teh.narod.ru rusoft-zone.ru
sadaholding.com russian-postindex.ru
saitevpatorii.com rutor.group
sammlungfotos.online rvtv.ru
sanidumps.com sack.net
scanner-donald.top salmonfishingsacramentoriver.com
scripted.com salutmontreal.com
sdi-pme.com scanner-fred.top
searchkut.com scanner-ivan.top
seccioncontrabajo.com scanner-mary.top
secretscook.ru scanner-olga.top
securesmrt-dt.com scenarii-1-sentyabrya.uroki.org.ua
semxiu.com search-error.com
seofied.com searchencrypt.com
seokicks.de searchimage.co
serptehnika.ru searchinterneat-a.akamaihd.net
sex-watch.com searchmywindow-a.akamaihd.net
sexad.net seasonvar.ru
shikiso.info segol.tv
shisha-swag.de seo-tools-optimizing.com
shitting.pro seoexperimenty.ru
simpoed.ufop.br seojokes.net
sisi-go.ru seolab.top
sizeplus.work serw.clicksor.com
slavic-magic.ru sex-dating.co
snip.to sexobzor.info
sobecjvuwa.com.ru sexpartygirls.net
social-s-ggg.xyz sextracker.de
soft-program.com sexy-pings.com
softlinesolutions.me share-buttons-for-free.com
songplanet.ru ship-marvel.co.ua
souvenir.cc shop.acim.org
spacash.com site.ru
spammen.de skutecznetabletkinaporostwlosow.pl
special-porn.com sky-mine.ru
spywarenuker.com smokewithrabbits.com
spywarespy.com soaksoak.ru
stard.shop socialbuttons.xyz
streamin.to solution4u.com
stroilka.info sonata-arctica.wz.cz
stroiminsk.com space4updating.win
strongholdsb.ru spb.afora.ru
stuffpride.com specialfinanceoffers.com
sucsesofinspiration.com spy-app.info
sugarlyflex.pw srdrvp.com
starwars.wikia.com
statoutlook.info
stop-zavisimost.com
streetfire.net
studiofmp.com
sundrugstore.com
support.nopeas.sk support.nopeas.sk
surffoundation.nl svetodiodoff.ru
svarbit.com swagbucks.com
svetlotorg.ru sweepstakes.rewardit.com
swsociety.se swsociety.se
syvertsen-da.no
tabletkinaodchudzanie.com.pl tabletkinaodchudzanie.com.pl
telefonsex-ohne0900.net tackletarts.co
testingads.pro tanieaukcje.com.pl
theallgirlarcade.com td-l-market.ru
thebestphotos.eu teastory.co
thepokertimer.com tech4master.com
timer4web.com teencastingporn.com
tizanidinehydrochloride2mg.blogspot.com teesdaleflyballclub.co.uk
tizanidinerestlesslegsyndrome.blogspot.com telegraf.by
texbaza.by
the-torrent-tracker.blogspot.com
thecoolimages.net
theendivechronicles.com
thewomenlife.com
tizanidinehcl2mgtablet.blogspot.com
tizanidinehcl4mgtab.blogspot.com
tizanidinevscyclobenzaprine.blogspot.com tizanidinevscyclobenzaprine.blogspot.com
tjkckpytpnje.com tnctrx.com
tnaionline.org token-lab.org
toloka.hurtom.com top10-online-games.com
tomatis.gospartner.com top10registrycleaners.com
topquality.cf topquality.cf
track112.site torrnada.ru
track2.shop toursmaps.com
trafficreceiver.club trkdf.com
trafficstars.com trucri.me
trahic.ru trustedmaccleaner.com
truck-rece.life turizmus.us
tuberkulezanet.ru u-cheats.ru
tubo360.com ua.tc
twelvevisionspartyofcolorado.com um-razum.ru
twiclub.in uni.me
tx41tclega.ru unionmarkt.de
txxx.com unitygame3d.com
uchebavchehii.ru unmaroll.ya.ru
udsgame.online urzedowski.eu
uginekologa.com usacasino.com
ukrtextbook.com vacuumcleanerguru.com
uptime-gamma.net varikoz24.com
uptimechecker.com
upupa.net
urengoy.pro
url-img.link
urlopener.com
userequip.com
uvozdeckych.info
valsalud.com
vapeface.club
venerologiya.com venerologiya.com
vesnatehno.com via-gra.webstarts.com
vesnatehno.ru video-chat.in
video-production.com videochat.tv.br
videochat.cafe vids18.site
videochat.mx
videochat.ph
vietimgy.pw vietimgy.pw
vipps.com.my vikistars.com
visa-china.ru vip2ch.com
visa-pasport.ru virtualbb.com
vita.com.hr vizitki.net
vkonche.com vkonche.com
vksex.ru vladimir.zrus.org
volgograd.xrus.org voditeltrezviy.ru
vse-pesni.com voloslove.ru
vysigy.su vrotike.ru
watracker.net vzlomsn.org
webshoppermac.com warningzscaler.heraeus.com
website-analyzer.info waycash.net
website-speed-checker.site webmonetizer.net
wejdz-tu.pl webradiology.ru
wheelchairliftsarea.com website-audit.com.ua
wineration.com websitevaluebot.com
wjgony.com westen-z.life
world-mmo.com whatistizanidinehclusedfor.blogspot.com
ww2awards.info whatzmyip.net
www.888.com whiteproduct.com
www.jbetting.com wholesalejerseychinashop.com
xmladserver.com wineitudes.wordpress.com
xmlinde.com withstandingheartwarming.com
xn-----7kceclhb4abre1b4a0ccl2fxch1a.xn--p1ai wladimirpayen.com
xn----8sblgmbj1a1bk8l.xn----161-4vemb6cjl7anbaea3afninj.xn--p1ai worldinternetauthority.com
xn----9sbubg3ambdfl1j.xn--p1ai www.labves.ru
xn--1-8sbcpb0bdm8k6a.xn--p1ai www.wohnkabinen-shop.de
xn--80aanaardaperhcem4a6i.com wzgyyq.com
xn--e1afanlbnfckd7c3d.xn--p1ai x-diesel.biz
xsion.net xn--oogle-wmc.com
yhit.press xtube.com
youtoner.it xtubeporno.net
youtubedownload.org xxart.ru
zaimhelp.ru xz618.com
zakonobosago.ru y8games-free.com
zapiszto.pl yaoguangdj.com
zetmaster.ru yatrk.xyz
zlatnajesen.com yourdesires.ru
znaturaloriginal.com yoursearch.me
zoomovies.org yousense.info
zoosexart.com zambini.ru
zrelaya.pw zed21.net
zskdla.site
zvooq.eu

470
dev-tools/test_units/random-referrers-for-whitelist-test.list
View file

@ -1,250 +1,250 @@
1webmaster.ml 0lovespells0.blogspot.com
adclickthru.net 2drittel.de
addblueoff.com.ua 5elementov.ru
adf.ly aa9d046aab36af4ff182f097f840430d51.com
adloads.net aab94f698f36684c5a852a2ef272e031bb.com
agahinameh.com absolutelyanalog.com
ahrntal.verymes.xyz acheterviagrafr24.com
amazingninja.com adtech.fr
amung.us affrh2015.com
aosexkontakte.net agecheckadult.com
approved.su ahrefs.com
arquivo.pt aktivator-windows10.blogspot.com
atlasvkusov.ru andpolice.com
atout-energie-69.com apartmentbay.ru
autoseo-traffic.com apartmentratings.com
aversis.net apioasisspacenet-a.akamaihd.net
aviav.co apiportalmorecom-a.akamaihd.net
azarttoto.com apiqualitinknet-a.akamaihd.net
appearance-cool.com
arcadepages.com
arcteryxstore.online
atelielembrancaqueencanta.com.br
autoblogger24.info
autodo.info
azinoofficial777.ru
balla-boo.se balla-boo.se
beauty-b0x.pl barbourjackets.online
bensbargains.net basedecor.ru
best-businessman.ru beachfix.co
bio.trade-jp.net beatifullandscape.co
bitcoinremote.com benchmarkcommunications.co.uk
biteg.xyz bestserials.com
bloke.com betterscooter.com
bongacams.com bigcareer.info
braintobe.win bonky.biz
brateg.xyz brillianty.info
bristolhostel.com byme.se
bristolhotel.com.ua canrioloadm.gq
celebsopics.com cartierjusteunclou.xyz
cfjump.com cartierlovestore.xyz
cfsrating.sonicwall.com cartiertrinity.xyz
chlooe.com cenoval.ru
clean-start.net changde.58.com
cobaltpro.ru cialischstgerts.com
cialisovercounteratwalmartusa.com
cicaki.net
cinemaenergy-hd.ru
cleanpcnow.com
click2pawn.com
co.lumb.co
compliance-fred.top
compliance-george.top compliance-george.top
compliance-jane.top compliance-julia.top
compliance-jessica.top compliance-mary.top
costablue.xyz comsysnet.com
countbertwistdisp26.soup.io cornerstone-countertops.com
cowblog.fr craftburg.ru
creditnation.ru creditnation.ru
crest3d.ru
cubook.supernew.org cubook.supernew.org
customcatchcan.com customboxes4less.com
cvety24.by cv.wallhade.co
daretodonate.co cwetochki.ru
dealighted.com cxpromote.com
czat.wp.pl
dating-time-now.com
defenderxtactical.com defenderxtactical.com
deluxedumps.com delta-line.men
dfiles.me digilander.libero.it
disruptingdinnerparties.com digitalassetmanagement.site
djihispano.com dyt.net
domaineaneblanc.com e-commerce-seo.com
drillsaw.ru ednorxmed.com
duawitchrarato.tk enthuse.computernetworksonline.com
elektrozigarette2.wordpressy.pl erotik0049.com
elektrozigarettekaufen1.devhub.com eupornstar.info
emergencyneeds.org ezigarettekaufen2.yolasite.com
en.altezza.travel fandlr.com
escortplius.com filesmonster.porn
evaashop.ru film-one.ru
fickkontaktehobbyhuren.com filmbokep69.com
firma-legion.ru filmi-onlain.info
florida-tourism.net finteks.ru
forexunion.net forex.osobye.ru
forpostlock.ru forum-engineering.ru
freegamesplay.online forum.doctissimo.fr
freza-sverlo.ru free411games.com
fuckmill.com fungamelands.com
fungirlsgames.net gambarxkata.co
g33.org
gambarkataku.co
gamingspark.com
gdcentre.ru gdcentre.ru
go2mike.ru gdzkurokam.ru
golden-catalog.pro getoutofdebtfree.org
goo.ne.jp gojiberriess.apishops.ru
greatgrace.ru googlefeud.com
greenshop.su gotwebsite1.com
growshop.es gpms.org.my
hao123.com guardlink.org
hauleddes.com hatdc.org
hdseriale.pl hausfrauensex18.com
hawaiisurf.com
hccoder.info
headpharmacy.com
heartofbeijing.blogspot.com heartofbeijing.blogspot.com
heygidday.biz heartofpayne.xyz
home.myplaycity.com hebr.myddns-flir.com
hotel-mkad.ru high-speed1.net
hotxnights.info highstairs-a.akamaihd.net
howopen.ru howtowhitenteethfast.xyz
husky-shop.cz humbmosquina.tk
iblogpress.xyz iblogpress.xyz
ifmo.ru iccornacircri.cf
ikritikimou.gr ido3.com
iloveitaly.ru imadedinner.net
infokonkurs.ru informatiecentro.be
istock-mebel.ru innoslicon.com
itservicesthatworkforyou.com insider.pro
javcoast.com insurple.com
jerseyswholesalechinalimited.com iradiology.ru
jetsli.de jofucipiku.tk
jjbabskoe.ru kino-rating.ru
jmat.cn
joingames.org
kadashihotel.com
kakadu-interior.com.ua
kaliningrad.zrus.org
kandidos.com
keywordsdoctor.com
kinosed.net kinosed.net
klejonka.info knigonosha.net
kndxbkdx.bloger.index.hr kolotiloff.ru
konteiner24.com krasnodar-avtolombards.ru
krasnodar.xrus.org kredit-blog.ru
landmania.ru kvartira-sutochno.com
landoftracking.com ladsblue.com
legionalpha.com lakomka.com.ua
lemon-ade.site
lernur.net lernur.net
lezbiyanki.net
liran-locks.com liran-locks.com
livetsomudvekslingstudent.bloggersdelight.dk longgreen.info
loginduepunti.it
longlifelomilomi.info
lostaruban.com
lsp-awak-perikanan.com lsp-awak-perikanan.com
luciddiagnostics.in m-google.xyz
magento-crew.net macdamaged.space
maranbrinfo.com.br magiadinamibia.blogspot.com
masthopehomes.com mainlinehobby.net
medicine-4u.org managerpak204.weebly.com
medmajor.ru marketingtechniques.info
meduza-consult.ru marsgatan.com
mexicosleevegastrectomy.com mebeldekor.com.ua
mirmedinfo.ru meble-bogart.info
morf.snn.gr metalonly.info
moscow-region.ru metaxalonevstizanidine.blogspot.com
moskva.nodup.ru mixed-wrestling.ru
movies-in-theaters.net moetomnenie.com
mvpicton.co.uk monthlywinners.com
mykings.pw mp3films.ru
myliveblog.ru muz-tracker.net
mymobilemoneypages.com mygameplus.com
mysex21.com netfacet.net
naval.jislaaik.com nisuturnetdgu.tk
netvouz.com nudepatch.net
npoet.ru obiavo.kz
nufaq.com obiavo.ru
obiavo.com odesproperty.com
oda.as ok.ru
onthemarch.co onblastblog.online
oyster-green.com onefilms.net
partner-trustworthy.men onlinedomains.ru
percin.biz.ly onlinemegax.com
picscout.com orel-reshka.net
plugingeorgia.com ownshop.cf
popcash.net pacificair.com
poptool.net painting-planet.com
porno-raskazy.ru panouri-solare-acoperis.com
portside.xyz pcgroup.com.uy
positive2b.ru perfectplanned.com
photochki.com
phuketscreen.com
pierrehardysale.online
pintattoos.com
poligon.com
pornohd1080.online
pregnant.guru pregnant.guru
privatamateure.com preparevideosafesystem4unow.site
profitkode.com privacylocationforloc.com
proftests.net pro-poly.ru
progress-upakovka.ru profitsport.club
pskcijdc.bloger.index.hr projefrio.com.br
puteshestvennik.com pulseonclick.com
puzo2arbuza.ru quick-offer.com
ranksonic.com rangoman.date
razleton.com reklama-i-rabota.ru
refads.pro renewablewealth.com
registrydomainservices.com resant.ru
reining.lovasszovetseg.hu rixpix.ru
rvtv.ru rocis.site
sad-torg.com.ua rossanasaavedra.net
sammyweaver.com rusenvironmental.net
saveriopiazza.it sarf3omlat.com
scanner-jessica.top sbwealthsolutions.ca
scanner-julia.top scoopquest.com
schalke04fc.info sdelatmebel.ru
segol.tv seoexperimenty.ru
seo-tools-optimizing.com sexfreepornoxxx.com
seo-traffic-ranking.info sexystrippe.info
seolab.top shop.acim.org
sexkontakteao.info siteheart.net
sharebutton.net social-vestnik.ru
sharebutton.to
shinikiev.com.ua
shodanhq.com
sitopreferito.it
skachat-besplatno-obrazcy.ru
skytraf.xyz
slavic-magic.ru
slayerlife.com
slim.sellany.ru
slowmac.tech
smart-scripts.com
soft-terminal.ru
softtor.com softtor.com
space4updating.win sotechco.co
spasswelt.net sovetogorod.ru
sr-rekneskap.no strana-solnca.ru
stairliftsarea.com strigkaomsk.ru
stroymonolit.su takeprofitsystem.com
strv.se theallgirlarcade.com
svarbit.com thebestphotos.eu
tarad.com theporndude.com
td-l-market.ru tiandeural.ru
thedownloadfreeonlinegames.blogspot.com tiens2010.ru
thejournal.ru titslove.yopoint.in
thepornsex.org tizanidine6mg.blogspot.com
tizanidine4mguses.blogspot.com tmearegion26.com
tizanidinesideeffects.blogspot.com tobeyouday.win
tomatis.gospartner.com tradedeals.biz
topcasinoratings.ru tradgardspartner.se
tri-slona.org trafficnetzwerk.de
trichizobswiv.agddns.net trainoffend.ml
trustaffs.com tramadolandtizanidine.blogspot.com
uchetunet.su typimga.pw
uptime-alpha.net uamtrk.com
urlopener.com ukr-lit.com
valkiria-tk.ru umnovocaminho.com
vapsy.com unionmarkt.de
vavilone.com uptime-delta.net
viagra-soft.ru uzporno.mobi
viagra.pp.ua varikoz24.com
videochat.bz varikozdok.ru
videotuber.ru vbabule.net
villacoloniale.com vikistars.com
vkak.ru voprosotvet24.ru
vod.com.ua we-are-gamers.com
vostoktrade.info weblibrary.win
vsexkontakte.net website-audit.com.ua
vzube.com wieseversa.no
wallpapersdesk.info woman-h.ru
warningwar.ru wufak.com
waysbetter.cn wzgyyq.com
webcamdevochka.com xn----7sbbahaq9bb5afgiqfliv4m.xn--p1ai
webcamtalk.net xn--d1abj0abs9d.in.ua
webkeyit.com xn--e1aggki3c.xn--80adxhks
website-analytics.online yaaknaa.info
whipme.yopoint.in yginekologa.com
wygraj-teraz.com youbrainboost.asia
xmnb.net
yourtemplatefinder.com yourtemplatefinder.com
yugk.net youtubedownload.org
zigarettenonlinekaufen1.blox.pl zeroredirect10.com
zog.link zetmaster.ru
znaturaloriginal.com
zynax.ua

2
dev-tools/test_units/whitelist-ips.conf
View file

@ -1,2 +1,2 @@
10.1.0.56 0; 10.1.0.66 0;
127.0.0.1 0; 127.0.0.1 0;

4
dev-tools/tmp/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3398 ### Version: V4.2022.10.3399
### Updated: Sat Oct 8 22:01:19 UTC 2022 ### Updated: Sun Oct 9 10:03:27 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##

4
robots.txt/robots.txt
View file

@ -6,8 +6,8 @@
### Version Information # ### Version Information #
################################################### ###################################################
### Version: V4.2022.10.3398 ### Version: V4.2022.10.3399
### Updated: Sat Oct 8 22:01:19 UTC 2022 ### Updated: Sun Oct 9 10:03:27 UTC 2022
### Bad Bot Count: 637 ### Bad Bot Count: 637
################################################### ###################################################
### Version Information ## ### Version Information ##