Commit graph

4 commits

Author SHA1 Message Date
Pedro Rodriguez
4ae7a62594
Run black/isort to fix linting (#117)
Some checks failed
Lint with Black / lint (push) Has been cancelled
Lint with isort / lint (push) Has been cancelled
Summary:

Test Plan:
2025-05-22 12:17:10 -07:00
Pedro Rodriguez
a582e5c703
Upgrade stool (#107)
Some checks failed
Lint with Black / lint (push) Has been cancelled
Lint with isort / lint (push) Has been cancelled
Summary:

Modify stool so that:
- It uses jinja2 for better templating
- Add args to allow force non-interactive overrides
- Add config_parser argument to allow instantiation of pydantic class when some defaults don't exist and use it in stool, to allow the rest of the configuration parser to work.

Test Plan:

```
uv run python -m bytelatent.stool config=/storage/home/par/code/internal-blt/configs/par/stool/run_blt_1b_aws.yaml
uv run python -m bytelatent.train config=../internal-blt/configs/par/train/aws/blt_1b_dclm_aws.yaml dump_dir=/checkpoint/comem/par/scratch/debug data.batch_size=2
```
2025-05-12 10:00:19 -07:00
Srinivasan Iyer
a5ceaaa226
When merging configs, do not merge data sources (#79)
Some checks are pending
Lint with Black / lint (push) Waiting to run
Lint with isort / lint (push) Waiting to run
* When merging configs, do not merge data sources

* Add todo

---------

Co-authored-by: Srini Iyer <sviyer@meta.com>
2025-03-11 11:03:24 -07:00
Pedro Rodriguez
82ab5930ec
Make it possible to specify multiple config files (#54)
Summary:

Make it possible to specify multiple config files.
Parsing CLI is not a special case anymore, just uses the same config inheritance method.

Test Plan:

Test that this iterpolates in the right order via unit tests

Sample usage, loads the internal config, which references bytelatent/configs/entropy_model.yaml. The precendence order is:

- Default pydantic args
- Included configs, eg `config`
- CLI args

```
python -m bytelatent.print_config config=internal/configs/entropy_model.yaml eval=null

```


Summary:

Test Plan:
2025-02-18 10:42:44 -08:00