* feat: add optional --include-submodules flag to CLI and ingestion
- Adds --include-submodules CLI flag to control submodule analysis
- Propagates include_submodules through ingestion, schemas, and clone logic
- Updates tests to cover submodule inclusion
- Adds a helper function (_checkout_partial_clone) to avoid repetition
- Adds include_submodules example in README.md
- Web UI for this option is not implemented for now (https://github.com/cyclotruc/gitingest/pull/313#issuecomment-3019912523)
---------
Co-authored-by: Filip Christiansen <22807962+filipchristiansen@users.noreply.github.com>
* refactor: centralize PAT validation, streamline repo checks & housekeeping
* `.venv*` to `.gitignore`
* `# type: ignore[attr-defined]` hints in `compat_typing.py` for IDE-agnostic imports
* Helpful PAT string in `InvalidGitHubTokenError` for easier debugging
* Bump **ruff-pre-commit** hook → `v0.12.1`
* CONTRIBUTING:
* Require **Python 3.9+**
* Recommend signed (`-S`) commits
* PAT validation now happens **only** in entry points
(`utils.auth.resolve_token` for CLI/lib, `server.process_query` for Web UI)
* Unified `_check_github_repo_exists` into `check_repo_exists`, replacing
`curl -I` with `curl --silent --location --write-out %{http_code} -o /dev/null`
* Broaden `_GITHUB_PAT_PATTERN`
* `create_git_auth_header` raises `ValueError` when hostname is missing
* Tests updated to expect raw HTTP-code output
* Superfluous “token can be set via `GITHUB_TOKEN`” notes in docstrings
* `.gitingestignore` & `.terraform` from `DEFAULT_IGNORE_PATTERNS`
* Token validation inside `create_git_command`
* Obsolete `test_create_git_command_invalid_token`
* Adjust `test_clone.py` and `test_git_utils.py` for new status-code handling
* Consolidate mocks after token-validation relocation
BREAKING CHANGE:
`create_git_command` no longer validates GitHub tokens; callers must ensure
tokens are valid (via `validate_github_token`) before invoking lower-level
git helpers.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor project structure, enhance logic, update configurations, and improve code quality
Refactoring and Logic Improvements
- Refactored the `_scan_directory` function in `src/gitingest/ingest_from_query.py` by extracting loop logic into the new `_process_item` function, and further separating functionality into `_process_symlink` and `_process_file`
- Replaced multiple return statements with error raising and catching, introducing custom exceptions (`MaxFilesReachedError`, `MaxFileSizeReachedError`, `AlreadyVisitedError`) in the `_process_item` and `_scan_directory` functions
- Enhanced the logic in the `process_query` function in `src/process_query.py` for better flow and maintainability
- Improved the logic in `_generate_token_string` in `src/gitingest/ingest_from_query.py`
- Refined the `download_ingest` function in `src/routers/download.py` for better clarity and functionality
Exception Handling Enhancements
- Replaced broad `Exception` handling with specific `OSError` in the `_read_file_content` function in `src/gitingest/ingest_from_query.py`
- Refined exception handling throughout the codebase, including removing redundant try-except-raise blocks, e.g., in `clone_repo` function in `src/gitingest/clone.py`
- Added custom exceptions to `src/gitingest/exceptions.py`: `MaxFilesReachedError`, `MaxFileSizeReachedError`, and `AlreadyVisitedError`
- Included explicit re-raising of exceptions in various functions for improved error propagation
Test Suite Refactoring
- Cleaned up and reorganized test files:
- Moved tests from `src/gitingest/tests/` to `tests/`
- Consolidated fixtures from `tests/test_ingest.py` into `tests/conftest.py`
- Removed redundant content from `tests/conftest.py`
- Migrated configuration from `pytest.ini` to `pyproject.toml`, deleted `pytest.ini`, and updated `.dockerignore`
Documentation Improvements
- Added `darglint` for enforcing `numpy` docstring style in `.pre-commit-config.yaml` for `src/` files
- Updated docstrings throughout the codebase, including adding module docstrings where needed
- Updated `README.md`:
- Added "GitHub stars" badge
- Moved the "Discord" badge to its own line
- Replaced occurrences of "Gitingest" with "GitIngest" for consistency and clarity
Linting and Code Quality
- Integrated `pylint` into `.pre-commit-config.yaml` for both `src/` and `tests/` directories
- Created `tests/.pylintrc` for linting configuration specific to test files
Code Clean-up
- Removed the redundant `src/__init__.py` file
Naming Conventions and Code Style
- Renamed `logSliderToSize` to `log_slider_to_size` in `src/server_utils.py` for consistency with Python's naming conventions
- Added explicit encoding specification in multiple instances of `open` throughout the code
2025-01-03 08:33:28 +01:00
Renamed from src/gitingest/tests/test_clone.py (Browse further)
