Commit graph

16 commits

Author SHA1 Message Date
Nicolas Iragne
c057f6e062
feat: use gitpython for git stuff (#504)
Some checks failed
CI / test (macos-latest, 3.8) (push) Has been cancelled
CI / test (true, ubuntu-latest, 3.13) (push) Has been cancelled
CI / test (ubuntu-latest, 3.8) (push) Has been cancelled
CI / test (macos-latest, 3.13) (push) Has been cancelled
CI / test (windows-latest, 3.13) (push) Has been cancelled
CI / test (windows-latest, 3.8) (push) Has been cancelled
CodeQL / Analyze (push) Has been cancelled
Build & Push Container / ECR (push) Has been cancelled
Build & Push Container / GHCR (push) Has been cancelled
release-please / release (push) Has been cancelled
OSSF Scorecard / Scorecard analysis (push) Has been cancelled
Co-authored-by: Iwan Burel <iwan.burel@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-11 17:27:42 +02:00
Mickael
414e85189f
feat: implement S3 integration for storing and retrieving digest files (#427)
Co-authored-by: Filip Christiansen <22807962+filipchristiansen@users.noreply.github.com>
Co-authored-by: Nicolas Iragne <nicoragne@hotmail.fr>
2025-07-26 16:28:17 +02:00
Filip Christiansen
a99089ab11
refactor: consistent cloning & pattern-handling (#388)
Some checks are pending
CI / test (macos-latest, 3.10) (push) Waiting to run
CI / test (macos-latest, 3.11) (push) Waiting to run
CI / test (macos-latest, 3.12) (push) Waiting to run
CI / test (macos-latest, 3.13) (push) Waiting to run
CI / test (macos-latest, 3.8) (push) Waiting to run
CI / test (macos-latest, 3.9) (push) Waiting to run
CI / test (true, ubuntu-latest, 3.13) (push) Waiting to run
CI / test (ubuntu-latest, 3.10) (push) Waiting to run
CI / test (ubuntu-latest, 3.11) (push) Waiting to run
CI / test (ubuntu-latest, 3.12) (push) Waiting to run
CI / test (ubuntu-latest, 3.8) (push) Waiting to run
CI / test (ubuntu-latest, 3.9) (push) Waiting to run
CI / test (windows-latest, 3.10) (push) Waiting to run
CI / test (windows-latest, 3.11) (push) Waiting to run
CI / test (windows-latest, 3.12) (push) Waiting to run
CI / test (windows-latest, 3.13) (push) Waiting to run
CI / test (windows-latest, 3.8) (push) Waiting to run
CI / test (windows-latest, 3.9) (push) Waiting to run
CodeQL / Analyze (push) Waiting to run
Build & Push Container / docker-build (push) Waiting to run
release-please / release (push) Waiting to run
OSSF Scorecard / Scorecard analysis (push) Waiting to run
Co-authored-by: ix-56h <n.guintini@protonmail.com>
2025-07-23 16:35:32 +02:00
Filip Christiansen
016817d559
feat: add Tailwind CSS pipeline, tag-aware cloning & overhaul CI/CD (#352)
Frontend

* introduce Tailwind CSS (package.json, tailwind.config.js, input CSS)
* build site.css on-the-fly (removed tracked artefact; added .gitignore)
* new favicon/icon assets & template cleanup
* split JS into modular files

Docker

* replace single-stage image with 3-stage build
  • css-builder (Node 20 alpine) → compiles Tailwind
  • python-builder installs project with PEP 621 metadata
  • runtime image copies site-packages + compiled CSS, runs as uid 1000

CI/CD

* ci.yml: cache by pyproject.toml, install with `pip -e .[dev]`
* new frontend job builds/archives CSS after tests
* publish.yml: build CSS first, then wheel/sdist; trusted OIDC upload
* tidy scorecard workflow

Core library

* clone.py, parser & utils now resolve tags in addition to branches/commits
* fallback branch/tag discovery when `git ls-remote` fails
* compat\_func.py back-ports Path.readlink / str.removesuffix for Py 3.8

Tooling & docs

* add `[dev]` extra, drop requirements-dev.txt & its pre-commit fixer
* refreshed CONTRIBUTING.md with Node/Tailwind instructions
* updated tests for new tag logic
2025-07-02 21:31:14 +02:00
Filip Christiansen
2f447ae632
chore: switch to ruff + pydoclint, deprecate .gitingest, and perform a repo-wide quality sweep (#329)
Some checks failed
CI / test (macos-latest, 3.10) (push) Has been cancelled
CI / test (macos-latest, 3.11) (push) Has been cancelled
CI / test (macos-latest, 3.12) (push) Has been cancelled
CI / test (macos-latest, 3.13) (push) Has been cancelled
CI / test (macos-latest, 3.8) (push) Has been cancelled
CI / test (macos-latest, 3.9) (push) Has been cancelled
CI / test (ubuntu-latest, 3.10) (push) Has been cancelled
CI / test (ubuntu-latest, 3.11) (push) Has been cancelled
CI / test (ubuntu-latest, 3.12) (push) Has been cancelled
CI / test (ubuntu-latest, 3.13) (push) Has been cancelled
CI / test (ubuntu-latest, 3.8) (push) Has been cancelled
CI / test (ubuntu-latest, 3.9) (push) Has been cancelled
CI / test (windows-latest, 3.10) (push) Has been cancelled
CI / test (windows-latest, 3.11) (push) Has been cancelled
CI / test (windows-latest, 3.12) (push) Has been cancelled
CI / test (windows-latest, 3.13) (push) Has been cancelled
CI / test (windows-latest, 3.8) (push) Has been cancelled
CI / test (windows-latest, 3.9) (push) Has been cancelled
OSSF Scorecard / Scorecard analysis (push) Has been cancelled
* **Pre-commit**: replace `black` & `darglint` with `ruff-check` / `ruff-format`;
  add `pydoclint` for docstring quality
* **Deps**: drop `tomli`; tighten `typing_extensions`; add `eval-type-backport`;
  remove `black`, `djlint`, `pylint` from `requirements-dev`
* **Ignore files**: deprecate TOML-based `.gitingest`; introduce
  `.gitingestignore` (git-wildmatch, parsed via `_parse_ignore_file`)
* **Config**: new unified `[tool.ruff]` (lint + format + isort); delete
  `[tool.black]`, keep minimal `[tool.isort]` for now
* **Refactor/style**: adopt `from __future__ import annotations`, kw-only args,
  richer types; reorder params & `__all__`; move type-only imports under
  `if TYPE_CHECKING`; extract `_CLIArgs` `TypedDict`, migrate form data to
  `pydantic.QueryForm`; deduplicate `cli.main` / `_async_main`; use `pathlib`,
  avoid file-IO in async; replace magic numbers with constants; delete
  `is_text_file` (logic now lives in `FileSystemNode.content`)
* **Bug fix**: remove silent error in `notebook_utils._process_cell`
* **Docs**: refresh README badges
* **Tests**: update fixtures & assertions

**BREAKING**: new `.gitingestignore` file replaces (now-deprecated) `.gitingest`.

No functional API or CLI changes.
2025-06-28 18:49:37 +02:00
Filip Christiansen
95009bdf15
test: add pytest-mock, introduce fixtures & type hints (#290)
* Added pytest-mock to dev dependencies and pre-commit hooks
* Introduced InvalidGitHubTokenError for clearer token-validation failures
* Refactored tests:
  * Replaced ad-hoc mocks with reusable fixtures
  * Parametrised URL/branch matrices to cut duplication
  * Added type hints throughout
* New coverage:
  * validate_github_token (happy & error paths)
  * create_git_command / create_git_auth_header
2025-06-21 21:26:29 +02:00
Filip Christiansen
3cee6725d3
Remove unused pattern_type parameter from IngestionQuery fixture (#228) 2025-03-13 02:35:18 +01:00
Romain Courtois
b098bb4534
Refactor/pydantic(#226) 2025-03-11 00:56:58 +01:00
Romain Courtois
d6cb920660
Refactor/ingestion (#209)
Co-authored-by: Filip Christiansen <22807962+filipchristiansen@users.noreply.github.com>
2025-03-04 01:11:54 +01:00
Filip Christiansen
4397a45281
feat: Add Python 3.7 Support and Restore Compatibility with Older Syntax (#181)
* Add Python 3.9 support by using ParamSpec from typing_extensions and removing match statements

* Add Python 3.7 support by reverting inline generics and removing walrus usage

* Update pyproject.toml
2025-02-17 11:36:57 +01:00
Filip Christiansen
58dbe2cb7e
refactor: code cleanup, utils extraction, and test improvements (#141)
* Refactor code for lifespan, template usage, and improved tests

- Move background tasks and rate-limit handler into utils.py
- Reference TEMPLATES from config instead of inline Jinja2Templates
- Adopt Given/When/Then docstrings for test clarity
- Parametrize some tests and consolidate code across query_parser tests
- Add pytest.warns context handler to test_parse_repo_source_with_failed_git_command
2025-01-23 02:46:17 +01:00
Filip Christiansen
d721b00d03
Replace dict-based query with ParsedQuery dataclass (#133)
- Introduce ParsedQuery dataclass to store query parameters and metadata
- Update ingestion and parser modules to use ParsedQuery instead of dict[str, Any]
- Convert ignore_patterns and include_patterns to sets
- Clean references to max size and pattern handling
- Update tests to reflect new dataclass usage
2025-01-17 16:53:32 +01:00
Filip Christiansen
d2825eac20
feat: add support for improved handling of jupyter notebooks (#105) 2025-01-08 23:46:24 +01:00
Joydeep Tripathy
9024c33376
docs: added docs for test files (#104) 2025-01-07 10:18:20 +01:00
Filip Christiansen
123f0ef0d7
Refactor: Replace os.path usage with pathlib.Path for improved maintainability (#106) 2025-01-07 10:13:22 +01:00
Filip Christiansen
d1b21d97ac
Refactor project structure, enhance logic, update configurations, and improve code quality (#85)
* Refactor project structure, enhance logic, update configurations, and improve code quality

Refactoring and Logic Improvements

- Refactored the `_scan_directory` function in `src/gitingest/ingest_from_query.py` by extracting loop logic into the new `_process_item` function, and further separating functionality into `_process_symlink` and `_process_file`
- Replaced multiple return statements with error raising and catching, introducing custom exceptions (`MaxFilesReachedError`, `MaxFileSizeReachedError`, `AlreadyVisitedError`) in the `_process_item` and `_scan_directory` functions
- Enhanced the logic in the `process_query` function in `src/process_query.py` for better flow and maintainability
- Improved the logic in `_generate_token_string` in `src/gitingest/ingest_from_query.py`
- Refined the `download_ingest` function in `src/routers/download.py` for better clarity and functionality

Exception Handling Enhancements

- Replaced broad `Exception` handling with specific `OSError` in the `_read_file_content` function in `src/gitingest/ingest_from_query.py`
- Refined exception handling throughout the codebase, including removing redundant try-except-raise blocks, e.g., in `clone_repo` function in `src/gitingest/clone.py`
- Added custom exceptions to `src/gitingest/exceptions.py`: `MaxFilesReachedError`, `MaxFileSizeReachedError`, and `AlreadyVisitedError`
- Included explicit re-raising of exceptions in various functions for improved error propagation

Test Suite Refactoring

- Cleaned up and reorganized test files:
  - Moved tests from `src/gitingest/tests/` to `tests/`
  - Consolidated fixtures from `tests/test_ingest.py` into `tests/conftest.py`
  - Removed redundant content from `tests/conftest.py`
- Migrated configuration from `pytest.ini` to `pyproject.toml`, deleted `pytest.ini`, and updated `.dockerignore`

Documentation Improvements

- Added `darglint` for enforcing `numpy` docstring style in `.pre-commit-config.yaml` for `src/` files
- Updated docstrings throughout the codebase, including adding module docstrings where needed
- Updated `README.md`:
  - Added "GitHub stars" badge
  - Moved the "Discord" badge to its own line
  - Replaced occurrences of "Gitingest" with "GitIngest" for consistency and clarity

Linting and Code Quality

- Integrated `pylint` into `.pre-commit-config.yaml` for both `src/` and `tests/` directories
- Created `tests/.pylintrc` for linting configuration specific to test files

Code Clean-up

- Removed the redundant `src/__init__.py` file

Naming Conventions and Code Style

- Renamed `logSliderToSize` to `log_slider_to_size` in `src/server_utils.py` for consistency with Python's naming conventions
- Added explicit encoding specification in multiple instances of `open` throughout the code
2025-01-03 08:33:28 +01:00