Pedro Rodriguez
48df9ce785
Merge c6ef4285e2 into sapling-pr-archive-EntilZha
2025-02-04 10:03:26 -08:00
Pedro Rodriguez
c6ef4285e2
Several changes to enable entropy model training/eval
...
Summary:
- Make arrow iterator able to read from jsonl files, the entropies are omitted in this case
- Make the data/checkpoint code fsspec compatible
- Fix issues with all reduce with non-bf16 in dist_sum and norm computation.
- Minimal fixes to get eval to run, it is slow currently
- Add bpb numbers during training
Test Plan:
2025-02-04 18:03:19 +00:00
Pedro Rodriguez
7044771a12
This includes fixes that make checkpointing and reloading work correctly. ( #35 )
...
Lint with Black / lint (push) Has been cancelled
Lint with isort / lint (push) Has been cancelled
It also batches in a first set of changes for fixing eval code
Summary:
Test Plan:
2025-01-27 16:56:42 -08:00
Pedro Rodriguez
c2f1e4845e
Merge e02ba763b0 into sapling-pr-archive-EntilZha
2025-01-27 16:38:54 -08:00
Pedro Rodriguez
e02ba763b0
This includes fixes that make checkpointing and reloading work correctly.
...
It also batches in a first set of changes for fixing eval code
Summary:
Test Plan:
2025-01-28 00:38:46 +00:00
Pedro Rodriguez
7622d28b74
Initial codes and scripts for training entropy model ( #34 )
...
Lint with Black / lint (push) Waiting to run
Lint with isort / lint (push) Waiting to run
Summary:
Test Plan:
2025-01-27 09:46:44 -08:00
Pedro Rodriguez
b1c12dd275
Merge 34ca1f7d4b into sapling-pr-archive-EntilZha
Lint with Black / lint (push) Has been cancelled
Lint with isort / lint (push) Has been cancelled
2025-01-24 13:59:47 -08:00
Pedro Rodriguez
34ca1f7d4b
Initial codes and scripts for training entropy model
...
Summary:
Test Plan:
2025-01-24 21:59:42 +00:00
Pedro Rodriguez
fb09022e5e
Initial codes and scripts for training entropy model
...
Summary:
Test Plan:
2025-01-24 21:55:41 +00:00
Pedro Rodriguez
6ffeb66b53
Changes for training entropy model and correcting attention in local models ( #25 )
...
Lint with Black / lint (push) Has been cancelled
Lint with isort / lint (push) Has been cancelled
Summary:
- Refactor local model configs to be separate and clearer
- Add attention arguments and correct which attention is used in local models
- Preparation for being able to have an entropy train script
- Fix failing unit tests
Test Plan:
2025-01-17 14:23:01 -08:00
Pedro Rodriguez
bcc039bb75
Initial commit
2024-12-12 15:32:30 -08:00
