Commit graph

20 commits

Author SHA1 Message Date
Arman
ba701a80c9
feat: ignore .gitignore files by default (use --include-gitignored to stay
Some checks are pending
CI / test (macos-latest, 3.10) (push) Waiting to run
CI / test (macos-latest, 3.11) (push) Waiting to run
CI / test (macos-latest, 3.12) (push) Waiting to run
CI / test (macos-latest, 3.13) (push) Waiting to run
CI / test (macos-latest, 3.8) (push) Waiting to run
CI / test (macos-latest, 3.9) (push) Waiting to run
CI / test (ubuntu-latest, 3.10) (push) Waiting to run
CI / test (ubuntu-latest, 3.11) (push) Waiting to run
CI / test (ubuntu-latest, 3.12) (push) Waiting to run
CI / test (ubuntu-latest, 3.13) (push) Waiting to run
CI / test (ubuntu-latest, 3.8) (push) Waiting to run
CI / test (ubuntu-latest, 3.9) (push) Waiting to run
CI / test (windows-latest, 3.10) (push) Waiting to run
CI / test (windows-latest, 3.11) (push) Waiting to run
CI / test (windows-latest, 3.12) (push) Waiting to run
CI / test (windows-latest, 3.13) (push) Waiting to run
CI / test (windows-latest, 3.8) (push) Waiting to run
CI / test (windows-latest, 3.9) (push) Waiting to run
OSSF Scorecard / Scorecard analysis (push) Waiting to run
* use_gitignore flag to exclude gitignore
---------

Co-authored-by: Filip Christiansen <22807962+filipchristiansen@users.noreply.github.com>
2025-06-25 05:04:50 +02:00
Amgad Hasan
2dea7c8865
Use gpt-4o's tokenizer (#258)
feat: switch to o200k_base, require tiktoken ≥ 0.7.0, drop Python 3.7

Context
-------
Token counting now uses **o200k_base** (native to GPT-4o / 4o-mini).  
That encoding ships only with **tiktoken ≥ 0.7.0**, whose wheels need Python 3.8+.  
CI already tests 3.8-3.13, so we align our documented minimums.

Changes
-------
* src/gitingest/output_formatters.py – `cl100k_base` → `o200k_base`
* README.md – “Python 3.7+” → “Python 3.8+”
* pyproject.toml  
  * `tiktoken` → `tiktoken>=0.7.0` (o200k support)  
  * remove classifier *Programming Language :: Python :: 3.7*
* requirements.txt – same `tiktoken` bump

Impact
------
* **Breaking** for users pinned to Python 3.7 → upgrade to 3.8+.  
* Environments on `tiktoken==0.6.*` must `pip install -U tiktoken>=0.7.0`.  
* No other runtime deps added.

Co-authored-by: Filip Christiansen <22807962+filipchristiansen@users.noreply.github.com>
2025-06-18 11:57:34 +02:00
Filip Christiansen
31484298b5
chore: remove unused dependencies and pin versions to address vulnerabilities
- Remove chardet and fastapi-analytics references from .pre-commit-config.yaml and requirements
- Pin fastapi, starlette, and uvicorn to versions fixing known vulnerabilities
- Add pydantic to requirements
- Update ingestion_schema to use pydantic's new ConfigDict
2025-03-13 13:04:21 +01:00
Filip Christiansen
89d2dc6d24
chore: bump dependencies to address security vulnerabilities (#227) 2025-03-12 17:32:05 +01:00
Romain Courtois
d6cb920660
Refactor/ingestion (#209)
Co-authored-by: Filip Christiansen <22807962+filipchristiansen@users.noreply.github.com>
2025-03-04 01:11:54 +01:00
Yanampally Abhiram Reddy
f90595de7c
feat(cli):Add support for .gitingest file processing in query ingestion (#191)
Co-authored-by: Romain Courtois <romain@coderamp.io>
Co-authored-by: Filip Christiansen <22807962+filipchristiansen@users.noreply.github.com>
2025-02-19 10:31:02 +01:00
cyclotruc
e86032e38e replace fastapi-analytics with posthog 2025-02-10 07:44:33 +00:00
Filip Christiansen
848b9dc1ed
Modernize Codebase and Enhance CI Workflow (#67)
Updated Type Hints:

Used pyupgrade with --py39-plus and --py310-plus flags to upgrade type hints to Python 3.9+ and 3.10+ syntax.
Enforced Double Quotes:

Removed skip-string-normalization = true from the black configuration in pyproject.toml.
Reran black via pre-commit hooks to enforce double quotes in the codebase.
Refactored Dependency Management:

Split requirements.txt into requirements.txt (runtime dependencies) and requirements-dev.txt (development dependencies).
Enhanced CI Workflow:

Integrated pre-commit hooks into the CI pipeline to enforce code quality checks automatically.
Added pip caching to the CI workflow to speed up dependency installation.
Automated Package Publishing:

Added a publish.yml GitHub Actions workflow to automate publishing to PyPI.
The workflow triggers on release creation or manual dispatch, builds the package, and publishes it to PyPI using twine.
2024-12-29 06:52:24 +01:00
Filip Christiansen
eb73a0cc1f
chore: add pre-commit config, type hints, badges, and lint codebase (#57)
* chore: add pre-commit config, type hints, badges, and lint codebase

- Add .pre-commit-config.yaml and pyproject.toml for Black and isort
- Add missing type hints throughout the code (Dict[...] for Python 3.8 compatibility)
- Added badges and convert existing badges to use <a><img></a> format
- Lint Markdown files
- Lint Jinja templates with djlint

* Resolve error and fix remaining type hint violations

* Fix absolute imports and mock paths in test_clone.py to resolve test failures.

* Replace deprecated 'dotenv' with 'python-dotenv' in requirements.txt to resolve installation errors.
2024-12-28 05:59:11 +01:00
cyclotruc
04687c11ee Fix token estimation 2024-12-22 05:25:51 +00:00
cyclotruc
fcb87a576d Package preparation work 2024-12-21 23:49:51 +00:00
Justin Levi Winter
728c4af862
feat(cli): Add Global CLI Tool for Local Directory Analysis (#38)
* feat(cli): add comprehensive CLI documentation to README

- Add installation instructions for global CLI usage
- Document all CLI options and flags with examples
- Include detailed usage examples for common scenarios
- Describe output format and automatic file ignoring
- Maintain consistent formatting with existing README sections
2024-12-20 05:18:22 +01:00
ebawen
823f01b3bc
test: add unit tests and coverage configuration (#28)
Co-authored-by: Mohamed Azerkane <mohamed.azerkane@cure51.com>
2024-12-14 23:59:10 +01:00
nol
9996a06a94
Fix file management and permissions issues (#23) 2024-12-12 19:39:29 +01:00
Romain Courtois
45651cc6a4
Revert "added click support for local repo ingestion (#8)" (#13)
This reverts commit bb23af2039.
2024-12-11 02:09:13 +01:00
James
bb23af2039
added click support for local repo ingestion (#8)
* added click support for local repo ingestion

Co-authored-by: James Richards <heiho1@MacBookPro.lan>
Co-authored-by: Romain Courtois <romain@coderamp.io>
2024-12-11 01:29:33 +01:00
cyclotruc
27653a8f24 Add rate limiting 2024-12-10 16:59:09 +00:00
Romain Courtois
ce12df5c3e
Removed tokencount for now 2024-12-05 16:27:24 +01:00
cyclotruc
5663f745e6 Remove codebase-digest dependency 2024-11-30 08:19:44 +00:00
cyclotruc
d0c289cf93 Initial commit 2024-11-29 08:33:50 +00:00