vikarti.anatra/main
1
0
Fork 0
mirror of https://github.com/manualdousuario/marreta.git synced 2025-04-07 14:19:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

readme simplificado

Browse source
This commit is contained in:
Renan Bernordi 2025-02-08 02:07:48 -03:00
parent 9a257efd46
commit d921dcd115
2 changed files with 38 additions and 328 deletions

199
README.en.md
View file

@ -1,7 +1,7 @@
# 🛠️ Marreta # 🛠️ Marreta
[![en](https://img.shields.io/badge/lang-en-red.svg)](https://github.com/manualdousuario/marreta/blob/master/README.en.md)
[![pt-br](https://img.shields.io/badge/lang-pt--br-green.svg)](https://github.com/manualdousuario/marreta/blob/master/README.md) [![pt-br](https://img.shields.io/badge/lang-pt--br-green.svg)](https://github.com/manualdousuario/marreta/blob/master/README.md)
[![en](https://img.shields.io/badge/lang-en-red.svg)](https://github.com/manualdousuario/marreta/blob/master/README.en.md)
[![Forks](https://img.shields.io/github/forks/manualdousuario/marreta)](https://github.com/manualdousuario/marreta/network/members) [![Forks](https://img.shields.io/github/forks/manualdousuario/marreta)](https://github.com/manualdousuario/marreta/network/members)
[![Stars](https://img.shields.io/github/stars/manualdousuario/marreta)](https://github.com/manualdousuario/marreta/stargazers) [![Stars](https://img.shields.io/github/stars/manualdousuario/marreta)](https://github.com/manualdousuario/marreta/stargazers)
@ -9,32 +9,32 @@
Marreta is a tool that breaks access barriers and elements that hinder reading! Marreta is a tool that breaks access barriers and elements that hinder reading!
![Before and after Marreta](https://github.com/manualdousuario/marreta/blob/main/screen.en.png?raw=true) ![Before and after Marreta](https://github.com/manualdousuario/marreta/blob/main/screen.png?raw=true)
Public instance at [marreta.pcdomanual.com](https://marreta.pcdomanual.com)! Public instance at [marreta.pcdomanual.com](https://marreta.pcdomanual.com)!
## ✨ What's cool about it? ## ✨ What's Cool?
- Cleans and corrects URLs automatically - Automatically cleans and corrects URLs
- Removes annoying tracking parameters - Removes annoying tracking parameters
- Forces HTTPS to keep everything secure - Forces HTTPS to keep everything secure
- Changes user agent to avoid blockages - Changes user agent to avoid blocking
- Leaves the HTML clean and optimized - Leaves HTML clean and optimized
- Fixes relative URLs on its own - Fixes relative URLs on its own
- Allows you to put your own styles and scripts - Allows you to add your own styles and scripts
- Removes unwanted elements - Removes unwanted elements
- Cache, cache! - Caching, caching!
- Blocks domains you don't want - Blocks domains you don't want
- Allows you to configure headers and cookies your way - Allows configuring headers and cookies your way
- PHP-FPM and OPcache - PHP-FPM and OPcache
## 🐳 Installing with Docker ## 🐳 Installing with Docker
Install Docker and Docker Compose Install [Docker and Docker Compose](https://docs.docker.com/engine/install/)
`curl -o ./docker-compose.yml https://raw.githubusercontent.com/manualdousuario/marreta/main/docker-compose.yml` `curl -o ./docker-compose.yml https://raw.githubusercontent.com/manualdousuario/marreta/main/docker-compose.yml`
Now modify it with your settings: Now modify with your preferences:
`nano docker-compose.yml` `nano docker-compose.yml`
@ -49,181 +49,36 @@ services:
- SITE_NAME= - SITE_NAME=
- SITE_DESCRIPTION= - SITE_DESCRIPTION=
- SITE_URL= - SITE_URL=
- LANGUAGE=
``` ```
- `SITE_NAME`: Your Marreta's name - `SITE_NAME`: Name of your Marreta
- `SITE_DESCRIPTION`: What it's for - `SITE_DESCRIPTION`: Explain what it's for
- `SITE_URL`: Where it will run, complete address with `https://`. If you change the port in docker-compose (e.g. 8080:80), you must also include the port in SITE_URL (e.g. https://yoursite:8080) - `SITE_URL`: Where it will run, full address with `https://`. If you change the port in docker-compose (e.g., 8080:80), you must also include the port in SITE_URL (e.g., https://yoursite:8080)
- `DNS_SERVERS`: Which DNS servers to use `1.1.1.1, 8.8.8.8` - `SELENIUM_HOST`: Server:PORT of Selenium host (e.g., selenium-hub:4444)
- `SELENIUM_HOST`: Selenium host server:PORT (e.g. selenium-hub:4444) - `LANGUAGE`: pt-br (Brazilian Portuguese), en (English), es (Spanish), de-de (German), ru-ru (Russian)
Now just run `docker compose up -d`
Now you can run `docker compose up -d` ### More configurations:
- Selenium: https://github.com/manualdousuario/marreta/wiki/%F0%9F%92%BB-Selenium-Hub-(Chrome-and-Firefox)
### S3 Cache - S3 Cache: https://github.com/manualdousuario/marreta/wiki/%F0%9F%97%83%EF%B8%8F-Cache-S3
- Maintenance: https://github.com/manualdousuario/marreta/wiki/%F0%9F%9B%A0%EF%B8%8F-Maintenance
Support for cache storage in S3. Configure the following variables in your `.env`:
```env
S3_CACHE_ENABLED=true
S3_ACCESS_KEY=access_key
S3_SECRET_KEY=secret_key
S3_BUCKET=bucket_name
S3_REGION=us-east-1
S3_FOLDER_=cache/
S3_ACL=private
S3_ENDPOINT=
```
Possible configurations:
```
## R2
S3_ACCESS_KEY=access_key
S3_SECRET_KEY=secret_key
S3_BUCKET=bucket_name
S3_ENDPOINT=https://{TOKEN}.r2.cloudflarestorage.com
S3_REGION=auto
S3_FOLDER_=cache/
S3_ACL=private
## DigitalOcean
S3_ACCESS_KEY=access_key
S3_SECRET_KEY=secret_key
S3_BUCKET=bucket_name
S3_ENDPOINT=https://{REGION}.digitaloceanspaces.com
S3_REGION=auto
S3_FOLDER_=cache/
S3_ACL=private
```
### Selenium Integration
Integration with Selenium allows processing sites that require JavaScript or have some more advanced protection barriers. To use this feature, you need to set up a Selenium environment with Firefox. Add the following configuration to your `docker-compose.yml`:
```yaml
services:
selenium-firefox:
container_name: selenium-firefox
image: selenium/node-firefox:4.27.0-20241204
shm_size: 2gb
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_ENABLE_TRACING=false
- SE_NODE_MAX_SESSIONS=10
- SE_NODE_OVERRIDE_MAX_SESSIONS=true
entrypoint: bash -c 'SE_OPTS="--host $$HOSTNAME" /opt/bin/entry_point.sh'
depends_on:
- selenium-hub
selenium-hub:
image: selenium/hub:4.27.0-20241204
container_name: selenium-hub
environment:
- SE_ENABLE_TRACING=false
- GRID_MAX_SESSION=10
- GRID_BROWSER_TIMEOUT=10
- GRID_TIMEOUT=10
ports:
- 4442:4442
- 4443:4443
- 4444:4444
```
Important settings:
- `shm_size`: Sets the shared memory size for Firefox (2GB recommended)
- `SE_NODE_MAX_SESSIONS`: Maximum number of concurrent sessions per node
- `GRID_MAX_SESSION`: Maximum number of concurrent sessions on the hub
- `GRID_BROWSER_TIMEOUT` and `GRID_TIMEOUT`: Timeouts in seconds
After configuring Selenium, make sure to set the `SELENIUM_HOST` variable in your environment to point to the Selenium hub (usually `selenium-hub:4444`).
## Development
1. First, clone the project:
```bash
git clone https://github.com/manualdousuario/marreta/
cd marreta/app
```
2. Install the project dependencies:
```bash
composer install
npm install
```
3. Create the configuration file:
```bash
cp .env.sample .env
```
4. Configure the environment variables in `.env`
5. Use the `default.conf` as a base for NGINX or point your webservice to `app/`
Gulp is used to compile Sass to CSS, minify JavaScript, use: `gulp`
### ⚙️ Customizing
The settings are organized in `data/`:
- `domain_rules.php`: Specific rules for each site
- `global_rules.php`: Rules that apply to all sites
- `blocked_domains.php`: List of blocked sites
### Translations
- `/languages/`: Each language is in its ISO id (`pt-br, en, es or de-de`) and can be defined in the environment `LANGUAGE`
## 🛠️ Maintenance
### Logging System
Logs are stored in `app/logs/*.log` with automatic rotation every 7 days.
Log settings available in `.env` or docker:
```env
LOG_LEVEL=WARNING
```
Available log levels:
- DEBUG: Detailed information for debugging
- INFO: General information about operations
- WARNING: Warnings that deserve attention (default)
- ERROR: Errors that do not interrupt operation
- CRITICAL: Critical errors that need immediate attention
View the application logs:
```bash
docker-compose logs app
# or directly from the log file
cat app/logs/*.log
```
### Clearing the cache
When you need to clear:
```bash
docker-compose exec app rm -rf /app/cache/*
```
## 🚀 Integrations ## 🚀 Integrations
- 🤖 **Telegram**: [Official Bot](https://t.me/leissoai_bot) - 🤖 **Telegram**: [Official Bot](https://t.me/leissoai_bot)
- 🦊 **Firefox**: Extension by [Clarissa Mendes](https://claromes.com/pages/whoami) - [Download](https://addons.mozilla.org/pt-BR/firefox/addon/marreta/) | [Source Code](https://github.com/manualdousuario/marreta-extensao) - 🦊 **Firefox**: Extension by [Clarissa Mendes](https://claromes.com/pages/whoami) - [Download](https://addons.mozilla.org/en-US/firefox/addon/marreta/) | [Source Code](https://github.com/manualdousuario/marreta-extensao)
- 🌀 **Chrome**: Extension by [Clarissa Mendes](https://claromes.com/pages/whoami) - [Download](https://chromewebstore.google.com/detail/marreta/ipelapagohjgjcgpncpbmaaacemafppe) | [Source Code](https://github.com/manualdousuario/marreta-extensao) - 🌀 **Chrome**: Extension by [Clarissa Mendes](https://claromes.com/pages/whoami) - [Download](https://chromewebstore.google.com/detail/marreta/ipelapagohjgjcgpncpbmaaacemafppe) | [Source Code](https://github.com/manualdousuario/marreta-extensao)
- 🦋 **Bluesky**: Bot by [Joselito](https://bsky.app/profile/joseli.to) - [Profile](https://bsky.app/profile/marreta.pcdomanual.com) | [Source Code](https://github.com/manualdousuario/marreta-bot) - 🦋 **Bluesky**: Bot by [Joselito](https://bsky.app/profile/joseli.to) - [Profile](https://bsky.app/profile/marreta.pcdomanual.com) | [Source Code](https://github.com/manualdousuario/marreta-bot)
- 🍎 **Apple**: Integration with [Shortcuts](https://www.icloud.com/shortcuts/3594074b69ee4707af52ed78922d624f) - 🍎 **Apple**: Integration with [Shortcuts](https://www.icloud.com/shortcuts/3594074b69ee4707af52ed78922d624f)
--- ---
Made with ❤️! If you have any questions or suggestions, open an issue and we'll help! 😉 Made with ❤️! If you have questions or suggestions, open an issue and we'll help! 😉
Thanks to the [https://github.com/burlesco/burlesco](Burlesco) and [https://github.com/nang-dev/hover-paywalls-browser-extension/](Hover) projects that served as the basis for several rules! Special thanks to the projects [Burlesco](https://github.com/burlesco/burlesco) and [Hover](https://github.com/nang-dev/hover-paywalls-browser-extension/) which served as the basis for many rules!
## Star History ## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=manualdousuario/marreta&type=Date)](https://star-history.com/#manualdousuario/marreta&Date) [![Star History Chart](https://api.star-history.com/svg?repos=manualdousuario/marreta&type=Date)](https://star-history.com/#manualdousuario/marreta&Date)

167
README.md
View file

@ -1,7 +1,7 @@
# 🛠️ Marreta # 🛠️ Marreta
[![pt-br](https://img.shields.io/badge/lang-pt--br-green.svg)](https://github.com/manualdousuario/marreta/blob/master/README.md)
[![en](https://img.shields.io/badge/lang-en-red.svg)](https://github.com/manualdousuario/marreta/blob/master/README.en.md) [![en](https://img.shields.io/badge/lang-en-red.svg)](https://github.com/manualdousuario/marreta/blob/master/README.en.md)
[![pt-br](https://img.shields.io/badge/lang-pt--br-green.svg)](https://github.com/manualdousuario/marreta/blob/master/README.md)
[![Forks](https://img.shields.io/github/forks/manualdousuario/marreta)](https://github.com/manualdousuario/marreta/network/members) [![Forks](https://img.shields.io/github/forks/manualdousuario/marreta)](https://github.com/manualdousuario/marreta/network/members)
[![Stars](https://img.shields.io/github/stars/manualdousuario/marreta)](https://github.com/manualdousuario/marreta/stargazers) [![Stars](https://img.shields.io/github/stars/manualdousuario/marreta)](https://github.com/manualdousuario/marreta/stargazers)
@ -30,11 +30,11 @@ Instancia publica em [marreta.pcdomanual.com](https://marreta.pcdomanual.com)!
## 🐳 Instalando em Docker ## 🐳 Instalando em Docker
Instale Docker e Docker Compose Instale [Docker e Docker Compose](https://docs.docker.com/engine/install/)
`curl -o ./docker-compose.yml https://raw.githubusercontent.com/manualdousuario/marreta/main/docker-compose.yml` `curl -o ./docker-compose.yml https://raw.githubusercontent.com/manualdousuario/marreta/main/docker-compose.yml`
Agora modifique com suas configurações: Agora modifique com suas preferencias:
`nano docker-compose.yml` `nano docker-compose.yml`
@ -49,166 +49,21 @@ services:
- SITE_NAME= - SITE_NAME=
- SITE_DESCRIPTION= - SITE_DESCRIPTION=
- SITE_URL= - SITE_URL=
- LANGUAGE=
``` ```
- `SITE_NAME`: Nome do seu Marreta - `SITE_NAME`: Nome do seu Marreta
- `SITE_DESCRIPTION`: Conta pra que serve - `SITE_DESCRIPTION`: Conta pra que serve
- `SITE_URL`: Onde vai rodar, endereço completo com `https://`. Se você alterar a porta no docker-compose (ex: 8080:80), você também deve incluir a porta no SITE_URL (ex: https://seusite:8080) - `SITE_URL`: Onde vai rodar, endereço completo com `https://`. Se você alterar a porta no docker-compose (ex: 8080:80), você também deve incluir a porta no SITE_URL (ex: https://seusite:8080)
- `DNS_SERVERS`: Quais servidores DNS usar `1.1.1.1, 8.8.8.8`
- `SELENIUM_HOST`: Servidor:PORTA do host do Selenium (ex: selenium-hub:4444) - `SELENIUM_HOST`: Servidor:PORTA do host do Selenium (ex: selenium-hub:4444)
- - `LANGUAGE`: pt-br (Português Brasil), en (Inglês), es (Espanhol) ou de-de (Alemão), ru-ru (Russo)
Agora pode rodar `docker compose up -d`
Agora só rodar `docker compose up -d`
### Cache S3 ### Mais configurações:
- Selenium: https://github.com/manualdousuario/marreta/wiki/%F0%9F%92%BB-Selenium-Hub-(Chrome-and-Firefox)
Suporte de armazenamento do cache em S3. Configure as seguintes variáveis no seu `.env`: - Cache S3: https://github.com/manualdousuario/marreta/wiki/%F0%9F%97%83%EF%B8%8F-Cache-S3
- Manutenção: https://github.com/manualdousuario/marreta/wiki/%F0%9F%9B%A0%EF%B8%8F-Maintenance
```env
S3_CACHE_ENABLED=true
S3_ACCESS_KEY=access_key
S3_SECRET_KEY=secret_key
S3_BUCKET=nome_do_bucket
S3_REGION=us-east-1
S3_FOLDER_=cache/
S3_ACL=private
S3_ENDPOINT=
```
Configurações possiveis:
```
## R2
S3_ACCESS_KEY=access_key
S3_SECRET_KEY=secret_key
S3_BUCKET=nome_do_bucket
S3_ENDPOINT=https://{TOKEN}.r2.cloudflarestorage.com
S3_REGION=auto
S3_FOLDER_=cache/
S3_ACL=private
## DigitalOcean
S3_ACCESS_KEY=access_key
S3_SECRET_KEY=secret_key
S3_BUCKET=nome_do_bucket
S3_ENDPOINT=https://{REGIAO}.digitaloceanspaces.com
S3_REGION=auto
S3_FOLDER_=cache/
S3_ACL=private
```
### Integração com Selenium
Integração com Selenium permite processar sites que requerem javascript ou têm algumas barreiras de proteção mais avançadas. Para usar esta funcionalidade, você precisa configurar um ambiente Selenium com Firefox. Adicione a seguinte configuração ao seu `docker-compose.yml`:
```yaml
services:
selenium-firefox:
container_name: selenium-firefox
image: selenium/node-firefox:4.27.0-20241204
shm_size: 2gb
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_ENABLE_TRACING=false
- SE_NODE_MAX_SESSIONS=10
- SE_NODE_OVERRIDE_MAX_SESSIONS=true
entrypoint: bash -c 'SE_OPTS="--host $$HOSTNAME" /opt/bin/entry_point.sh'
depends_on:
- selenium-hub
selenium-hub:
image: selenium/hub:4.27.0-20241204
container_name: selenium-hub
environment:
- SE_ENABLE_TRACING=false
- GRID_MAX_SESSION=10
- GRID_BROWSER_TIMEOUT=10
- GRID_TIMEOUT=10
ports:
- 4442:4442
- 4443:4443
- 4444:4444
```
Configurações importantes:
- `shm_size`: Define o tamanho da memória compartilhada para o Firefox (2GB recomendado)
- `SE_NODE_MAX_SESSIONS`: Número máximo de sessões simultâneas por nó
- `GRID_MAX_SESSION`: Número máximo de sessões simultâneas no hub
- `GRID_BROWSER_TIMEOUT` e `GRID_TIMEOUT`: Timeouts em segundos
Após configurar o Selenium, certifique-se de definir a variável `SELENIUM_HOST` no seu ambiente para apontar para o hub do Selenium (geralmente `selenium-hub:4444`).
## Desenvolvimento
1. Primeiro, clone o projeto:
```bash
git clone https://github.com/manualdousuario/marreta/
cd marreta/app
```
2. Instale as dependências do projeto:
```bash
composer install
npm install
```
3. Cria o arquivo de configuração:
```bash
cp .env.sample .env
```
4. Configure as variáveis de ambiente no `.env`
5. Utilize o `default.conf` como base do NGINX ou aponte seu webservice para `app/`
O Gulp é usado para compilar Sass para CSS, minificar JavaScript, utilize: `gulp`
### ⚙️ Personalizando
As configurações estão organizadas em `data/`:
- `domain_rules.php`: Regras específicas para cada site
- `global_rules.php`: Regras que se aplicam a todos os sites
- `blocked_domains.php`: Lista de sites bloqueados
### Traduções
- `/languages/`: Cada lingua está em seu ISO id (`pt-br, en, es ou de-de`) e pode ser definida no environment `LANGUAGE`
## 🛠️ Manutenção
### Sistema de Logs
Os logs são armazenados em `app/logs/*.log` com rotação automática a cada 7 dias.
Configurações de log disponíveis no `.env` ou docker:
```env
LOG_LEVEL=WARNING
```
Níveis de log disponíveis:
- DEBUG: Informações detalhadas para debug
- INFO: Informações gerais sobre operações
- WARNING: Avisos que merecem atenção (padrão)
- ERROR: Erros que não interrompem a operação
- CRITICAL: Erros críticos que precisam de atenção imediata
Ver os logs da aplicação:
```bash
docker-compose logs app
# ou diretamente do arquivo de log
cat app/logs/*.log
```
### Limpando o cache
Quando precisar limpar:
```bash
docker-compose exec app rm -rf /app/cache/*
```
## 🚀 Integrações ## 🚀 Integrações