diff --git a/README.en.md b/README.en.md index 55e49e6..3fff96c 100644 --- a/README.en.md +++ b/README.en.md @@ -1,7 +1,7 @@ # 🛠️ Marreta -[](https://github.com/manualdousuario/marreta/blob/master/README.en.md) [](https://github.com/manualdousuario/marreta/blob/master/README.md) +[](https://github.com/manualdousuario/marreta/blob/master/README.en.md) [](https://github.com/manualdousuario/marreta/network/members) [](https://github.com/manualdousuario/marreta/stargazers) @@ -9,32 +9,32 @@ Marreta is a tool that breaks access barriers and elements that hinder reading! - + Public instance at [marreta.pcdomanual.com](https://marreta.pcdomanual.com)! -## ✨ What's cool about it? +## ✨ What's Cool? -- Cleans and corrects URLs automatically +- Automatically cleans and corrects URLs - Removes annoying tracking parameters - Forces HTTPS to keep everything secure -- Changes user agent to avoid blockages -- Leaves the HTML clean and optimized +- Changes user agent to avoid blocking +- Leaves HTML clean and optimized - Fixes relative URLs on its own -- Allows you to put your own styles and scripts +- Allows you to add your own styles and scripts - Removes unwanted elements -- Cache, cache! +- Caching, caching! - Blocks domains you don't want -- Allows you to configure headers and cookies your way +- Allows configuring headers and cookies your way - PHP-FPM and OPcache ## 🐳 Installing with Docker -Install Docker and Docker Compose +Install [Docker and Docker Compose](https://docs.docker.com/engine/install/) `curl -o ./docker-compose.yml https://raw.githubusercontent.com/manualdousuario/marreta/main/docker-compose.yml` -Now modify it with your settings: +Now modify with your preferences: `nano docker-compose.yml` @@ -49,181 +49,36 @@ services: - SITE_NAME= - SITE_DESCRIPTION= - SITE_URL= + - LANGUAGE= ``` -- `SITE_NAME`: Your Marreta's name -- `SITE_DESCRIPTION`: What it's for -- `SITE_URL`: Where it will run, complete address with `https://`. If you change the port in docker-compose (e.g. 8080:80), you must also include the port in SITE_URL (e.g. https://yoursite:8080) -- `DNS_SERVERS`: Which DNS servers to use `1.1.1.1, 8.8.8.8` -- `SELENIUM_HOST`: Selenium host server:PORT (e.g. selenium-hub:4444) +- `SITE_NAME`: Name of your Marreta +- `SITE_DESCRIPTION`: Explain what it's for +- `SITE_URL`: Where it will run, full address with `https://`. If you change the port in docker-compose (e.g., 8080:80), you must also include the port in SITE_URL (e.g., https://yoursite:8080) +- `SELENIUM_HOST`: Server:PORT of Selenium host (e.g., selenium-hub:4444) +- `LANGUAGE`: pt-br (Brazilian Portuguese), en (English), es (Spanish), de-de (German), ru-ru (Russian) + +Now just run `docker compose up -d` -Now you can run `docker compose up -d` - -### S3 Cache - -Support for cache storage in S3. Configure the following variables in your `.env`: - -```env -S3_CACHE_ENABLED=true - -S3_ACCESS_KEY=access_key -S3_SECRET_KEY=secret_key -S3_BUCKET=bucket_name -S3_REGION=us-east-1 -S3_FOLDER_=cache/ -S3_ACL=private -S3_ENDPOINT= -``` - -Possible configurations: - -``` -## R2 -S3_ACCESS_KEY=access_key -S3_SECRET_KEY=secret_key -S3_BUCKET=bucket_name -S3_ENDPOINT=https://{TOKEN}.r2.cloudflarestorage.com -S3_REGION=auto -S3_FOLDER_=cache/ -S3_ACL=private - -## DigitalOcean -S3_ACCESS_KEY=access_key -S3_SECRET_KEY=secret_key -S3_BUCKET=bucket_name -S3_ENDPOINT=https://{REGION}.digitaloceanspaces.com -S3_REGION=auto -S3_FOLDER_=cache/ -S3_ACL=private -``` - -### Selenium Integration - -Integration with Selenium allows processing sites that require JavaScript or have some more advanced protection barriers. To use this feature, you need to set up a Selenium environment with Firefox. Add the following configuration to your `docker-compose.yml`: - -```yaml -services: - selenium-firefox: - container_name: selenium-firefox - image: selenium/node-firefox:4.27.0-20241204 - shm_size: 2gb - environment: - - SE_EVENT_BUS_HOST=selenium-hub - - SE_EVENT_BUS_PUBLISH_PORT=4442 - - SE_EVENT_BUS_SUBSCRIBE_PORT=4443 - - SE_ENABLE_TRACING=false - - SE_NODE_MAX_SESSIONS=10 - - SE_NODE_OVERRIDE_MAX_SESSIONS=true - entrypoint: bash -c 'SE_OPTS="--host $$HOSTNAME" /opt/bin/entry_point.sh' - depends_on: - - selenium-hub - - selenium-hub: - image: selenium/hub:4.27.0-20241204 - container_name: selenium-hub - environment: - - SE_ENABLE_TRACING=false - - GRID_MAX_SESSION=10 - - GRID_BROWSER_TIMEOUT=10 - - GRID_TIMEOUT=10 - ports: - - 4442:4442 - - 4443:4443 - - 4444:4444 -``` - -Important settings: -- `shm_size`: Sets the shared memory size for Firefox (2GB recommended) -- `SE_NODE_MAX_SESSIONS`: Maximum number of concurrent sessions per node -- `GRID_MAX_SESSION`: Maximum number of concurrent sessions on the hub -- `GRID_BROWSER_TIMEOUT` and `GRID_TIMEOUT`: Timeouts in seconds - -After configuring Selenium, make sure to set the `SELENIUM_HOST` variable in your environment to point to the Selenium hub (usually `selenium-hub:4444`). - -## Development - -1. First, clone the project: -```bash -git clone https://github.com/manualdousuario/marreta/ -cd marreta/app -``` - -2. Install the project dependencies: -```bash -composer install -npm install -``` - -3. Create the configuration file: -```bash -cp .env.sample .env -``` - -4. Configure the environment variables in `.env` - -5. Use the `default.conf` as a base for NGINX or point your webservice to `app/` - -Gulp is used to compile Sass to CSS, minify JavaScript, use: `gulp` - -### ⚙️ Customizing - -The settings are organized in `data/`: - -- `domain_rules.php`: Specific rules for each site -- `global_rules.php`: Rules that apply to all sites -- `blocked_domains.php`: List of blocked sites - -### Translations - -- `/languages/`: Each language is in its ISO id (`pt-br, en, es or de-de`) and can be defined in the environment `LANGUAGE` - -## 🛠️ Maintenance - -### Logging System - -Logs are stored in `app/logs/*.log` with automatic rotation every 7 days. - -Log settings available in `.env` or docker: - -```env -LOG_LEVEL=WARNING -``` - -Available log levels: -- DEBUG: Detailed information for debugging -- INFO: General information about operations -- WARNING: Warnings that deserve attention (default) -- ERROR: Errors that do not interrupt operation -- CRITICAL: Critical errors that need immediate attention - -View the application logs: -```bash -docker-compose logs app -# or directly from the log file -cat app/logs/*.log -``` - -### Clearing the cache - -When you need to clear: -```bash -docker-compose exec app rm -rf /app/cache/* -``` +### More configurations: +- Selenium: https://github.com/manualdousuario/marreta/wiki/%F0%9F%92%BB-Selenium-Hub-(Chrome-and-Firefox) +- S3 Cache: https://github.com/manualdousuario/marreta/wiki/%F0%9F%97%83%EF%B8%8F-Cache-S3 +- Maintenance: https://github.com/manualdousuario/marreta/wiki/%F0%9F%9B%A0%EF%B8%8F-Maintenance ## 🚀 Integrations - 🤖 **Telegram**: [Official Bot](https://t.me/leissoai_bot) -- 🦊 **Firefox**: Extension by [Clarissa Mendes](https://claromes.com/pages/whoami) - [Download](https://addons.mozilla.org/pt-BR/firefox/addon/marreta/) | [Source Code](https://github.com/manualdousuario/marreta-extensao) +- 🦊 **Firefox**: Extension by [Clarissa Mendes](https://claromes.com/pages/whoami) - [Download](https://addons.mozilla.org/en-US/firefox/addon/marreta/) | [Source Code](https://github.com/manualdousuario/marreta-extensao) - 🌀 **Chrome**: Extension by [Clarissa Mendes](https://claromes.com/pages/whoami) - [Download](https://chromewebstore.google.com/detail/marreta/ipelapagohjgjcgpncpbmaaacemafppe) | [Source Code](https://github.com/manualdousuario/marreta-extensao) - 🦋 **Bluesky**: Bot by [Joselito](https://bsky.app/profile/joseli.to) - [Profile](https://bsky.app/profile/marreta.pcdomanual.com) | [Source Code](https://github.com/manualdousuario/marreta-bot) - 🍎 **Apple**: Integration with [Shortcuts](https://www.icloud.com/shortcuts/3594074b69ee4707af52ed78922d624f) --- -Made with ❤️! If you have any questions or suggestions, open an issue and we'll help! 😉 +Made with ❤️! If you have questions or suggestions, open an issue and we'll help! 😉 -Thanks to the [https://github.com/burlesco/burlesco](Burlesco) and [https://github.com/nang-dev/hover-paywalls-browser-extension/](Hover) projects that served as the basis for several rules! +Special thanks to the projects [Burlesco](https://github.com/burlesco/burlesco) and [Hover](https://github.com/nang-dev/hover-paywalls-browser-extension/) which served as the basis for many rules! ## Star History -[](https://star-history.com/#manualdousuario/marreta&Date) +[](https://star-history.com/#manualdousuario/marreta&Date) \ No newline at end of file diff --git a/README.md b/README.md index 6c03428..0873e3a 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # 🛠️ Marreta -[](https://github.com/manualdousuario/marreta/blob/master/README.md) [](https://github.com/manualdousuario/marreta/blob/master/README.en.md) +[](https://github.com/manualdousuario/marreta/blob/master/README.md) [](https://github.com/manualdousuario/marreta/network/members) [](https://github.com/manualdousuario/marreta/stargazers) @@ -30,11 +30,11 @@ Instancia publica em [marreta.pcdomanual.com](https://marreta.pcdomanual.com)! ## 🐳 Instalando em Docker -Instale Docker e Docker Compose +Instale [Docker e Docker Compose](https://docs.docker.com/engine/install/) `curl -o ./docker-compose.yml https://raw.githubusercontent.com/manualdousuario/marreta/main/docker-compose.yml` -Agora modifique com suas configurações: +Agora modifique com suas preferencias: `nano docker-compose.yml` @@ -49,166 +49,21 @@ services: - SITE_NAME= - SITE_DESCRIPTION= - SITE_URL= + - LANGUAGE= ``` - `SITE_NAME`: Nome do seu Marreta - `SITE_DESCRIPTION`: Conta pra que serve - `SITE_URL`: Onde vai rodar, endereço completo com `https://`. Se você alterar a porta no docker-compose (ex: 8080:80), você também deve incluir a porta no SITE_URL (ex: https://seusite:8080) -- `DNS_SERVERS`: Quais servidores DNS usar `1.1.1.1, 8.8.8.8` - `SELENIUM_HOST`: Servidor:PORTA do host do Selenium (ex: selenium-hub:4444) -- -Agora pode rodar `docker compose up -d` +- `LANGUAGE`: pt-br (Português Brasil), en (Inglês), es (Espanhol) ou de-de (Alemão), ru-ru (Russo) + +Agora só rodar `docker compose up -d` -### Cache S3 - -Suporte de armazenamento do cache em S3. Configure as seguintes variáveis no seu `.env`: - -```env -S3_CACHE_ENABLED=true - -S3_ACCESS_KEY=access_key -S3_SECRET_KEY=secret_key -S3_BUCKET=nome_do_bucket -S3_REGION=us-east-1 -S3_FOLDER_=cache/ -S3_ACL=private -S3_ENDPOINT= -``` - -Configurações possiveis: - -``` -## R2 -S3_ACCESS_KEY=access_key -S3_SECRET_KEY=secret_key -S3_BUCKET=nome_do_bucket -S3_ENDPOINT=https://{TOKEN}.r2.cloudflarestorage.com -S3_REGION=auto -S3_FOLDER_=cache/ -S3_ACL=private - -## DigitalOcean -S3_ACCESS_KEY=access_key -S3_SECRET_KEY=secret_key -S3_BUCKET=nome_do_bucket -S3_ENDPOINT=https://{REGIAO}.digitaloceanspaces.com -S3_REGION=auto -S3_FOLDER_=cache/ -S3_ACL=private -``` - -### Integração com Selenium - -Integração com Selenium permite processar sites que requerem javascript ou têm algumas barreiras de proteção mais avançadas. Para usar esta funcionalidade, você precisa configurar um ambiente Selenium com Firefox. Adicione a seguinte configuração ao seu `docker-compose.yml`: - -```yaml -services: - selenium-firefox: - container_name: selenium-firefox - image: selenium/node-firefox:4.27.0-20241204 - shm_size: 2gb - environment: - - SE_EVENT_BUS_HOST=selenium-hub - - SE_EVENT_BUS_PUBLISH_PORT=4442 - - SE_EVENT_BUS_SUBSCRIBE_PORT=4443 - - SE_ENABLE_TRACING=false - - SE_NODE_MAX_SESSIONS=10 - - SE_NODE_OVERRIDE_MAX_SESSIONS=true - entrypoint: bash -c 'SE_OPTS="--host $$HOSTNAME" /opt/bin/entry_point.sh' - depends_on: - - selenium-hub - - selenium-hub: - image: selenium/hub:4.27.0-20241204 - container_name: selenium-hub - environment: - - SE_ENABLE_TRACING=false - - GRID_MAX_SESSION=10 - - GRID_BROWSER_TIMEOUT=10 - - GRID_TIMEOUT=10 - ports: - - 4442:4442 - - 4443:4443 - - 4444:4444 -``` - -Configurações importantes: -- `shm_size`: Define o tamanho da memória compartilhada para o Firefox (2GB recomendado) -- `SE_NODE_MAX_SESSIONS`: Número máximo de sessões simultâneas por nó -- `GRID_MAX_SESSION`: Número máximo de sessões simultâneas no hub -- `GRID_BROWSER_TIMEOUT` e `GRID_TIMEOUT`: Timeouts em segundos - -Após configurar o Selenium, certifique-se de definir a variável `SELENIUM_HOST` no seu ambiente para apontar para o hub do Selenium (geralmente `selenium-hub:4444`). - -## Desenvolvimento - -1. Primeiro, clone o projeto: -```bash -git clone https://github.com/manualdousuario/marreta/ -cd marreta/app -``` - -2. Instale as dependências do projeto: -```bash -composer install -npm install -``` - -3. Cria o arquivo de configuração: -```bash -cp .env.sample .env -``` - -4. Configure as variáveis de ambiente no `.env` - -5. Utilize o `default.conf` como base do NGINX ou aponte seu webservice para `app/` - -O Gulp é usado para compilar Sass para CSS, minificar JavaScript, utilize: `gulp` - -### ⚙️ Personalizando - -As configurações estão organizadas em `data/`: - -- `domain_rules.php`: Regras específicas para cada site -- `global_rules.php`: Regras que se aplicam a todos os sites -- `blocked_domains.php`: Lista de sites bloqueados - -### Traduções - -- `/languages/`: Cada lingua está em seu ISO id (`pt-br, en, es ou de-de`) e pode ser definida no environment `LANGUAGE` - -## 🛠️ Manutenção - -### Sistema de Logs - -Os logs são armazenados em `app/logs/*.log` com rotação automática a cada 7 dias. - -Configurações de log disponíveis no `.env` ou docker: - -```env -LOG_LEVEL=WARNING -``` - -Níveis de log disponíveis: -- DEBUG: Informações detalhadas para debug -- INFO: Informações gerais sobre operações -- WARNING: Avisos que merecem atenção (padrão) -- ERROR: Erros que não interrompem a operação -- CRITICAL: Erros críticos que precisam de atenção imediata - -Ver os logs da aplicação: -```bash -docker-compose logs app -# ou diretamente do arquivo de log -cat app/logs/*.log -``` - -### Limpando o cache - -Quando precisar limpar: -```bash -docker-compose exec app rm -rf /app/cache/* -``` +### Mais configurações: +- Selenium: https://github.com/manualdousuario/marreta/wiki/%F0%9F%92%BB-Selenium-Hub-(Chrome-and-Firefox) +- Cache S3: https://github.com/manualdousuario/marreta/wiki/%F0%9F%97%83%EF%B8%8F-Cache-S3 +- Manutenção: https://github.com/manualdousuario/marreta/wiki/%F0%9F%9B%A0%EF%B8%8F-Maintenance ## 🚀 Integrações