vikarti.anatra/blt
1
0
Fork 0
mirror of https://github.com/facebookresearch/blt.git synced 2025-04-03 16:29:09 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
blt/bytelatent
History
Pedro Rodriguez b79eb3ef11
Get generation working for BLT () 
Summary:

Create a script for simple generation from BLT

Test Plan:

```
python -m bytelatent.generate_blt config=../internal-blt/configs/eval_blt.yaml
```
2025-04-01 16:07:55 -07:00
..
configs Remove byte tokenizer and add config args to switch between byte/patch packing () 2025-02-25 11:10:59 -08:00
data Get generation working for BLT () 2025-04-01 16:07:55 -07:00
model Fix in-place addition of patch_embds () 2025-03-20 16:46:32 -07:00
plotting Add plotting code from paper () 2025-01-09 12:11:50 -08:00
preprocess Allow ArrowIterator to read from json () 2025-02-06 09:57:22 -08:00
tokenizers Remove byte tokenizer and add config args to switch between byte/patch packing () 2025-02-25 11:10:59 -08:00
.DS_Store Initial commit 2024-12-12 15:32:30 -08:00
__init__.py Initial commit 2024-12-12 15:32:30 -08:00
args.py Get generation working for BLT () 2025-04-01 16:07:55 -07:00
base_transformer.py Initialize rope embeddings properly for the entropy model () 2025-02-25 15:35:25 -08:00
checkpoint.py Add way to call consolidate () 2025-03-11 16:53:33 -07:00
config_parser.py When merging configs, do not merge data sources () 2025-03-11 11:03:24 -07:00
constants.py Initial commit 2024-12-12 15:32:30 -08:00
distributed.py Get generation working for BLT () 2025-04-01 16:07:55 -07:00
entropy_model.py Some fixes for entropy model predictions () 2025-03-13 10:28:42 -07:00
eval.py Get generation working for BLT () 2025-04-01 16:07:55 -07:00
float8.py Initial commit 2024-12-12 15:32:30 -08:00
generate.py Get generation working for BLT () 2025-04-01 16:07:55 -07:00
generate_blt.py Get generation working for BLT () 2025-04-01 16:07:55 -07:00
iterate_data.py Update iterate_data () 2025-03-13 10:14:41 -07:00
logger.py Update checkpointing to use fsspec () 2025-02-06 09:41:58 -08:00
metrics.py Get evals working again. () 2025-03-11 09:57:19 -07:00
norms.py Fix distributed all reduce grad norm () 2025-02-04 16:53:50 -08:00
optim.py Initial commit 2024-12-12 15:32:30 -08:00
print_config.py Make it possible to specify multiple config files () 2025-02-18 10:42:44 -08:00
probe.py Initial commit 2024-12-12 15:32:30 -08:00
profiling.py Initial commit 2024-12-12 15:32:30 -08:00
stool.py Fix rsync to not preserve original permissions, instead use destination () 2025-03-05 11:49:41 -08:00
test_blt.py Initial codes and scripts for training entropy model () 2025-01-27 09:46:44 -08:00
test_config_parser.py Make it possible to specify multiple config files () 2025-02-18 10:42:44 -08:00
test_entropy_model.py Test first batch matches () 2025-02-13 10:05:08 -08:00
train.py Get evals working again. () 2025-03-11 09:57:19 -07:00
transformer.py Initialize rope embeddings properly for the entropy model () 2025-02-25 15:35:25 -08:00