vikarti.anatra/blt
1
0
Fork 0
mirror of https://github.com/facebookresearch/blt.git synced 2025-01-18 16:37:46 +00:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity

docs: update README.md (#1)
Some checks failed
Lint with Black / lint (push) Has been cancelled
Details
Lint with isort / lint (push) Has been cancelled
Details

Browse source 
folowing -> following
This commit is contained in:
Ikko Eltociear Ashimine 2025-01-04 05:08:00 +09:00 committed by GitHub
parent 898671b66b
commit 9065bb1cce
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -53,7 +53,7 @@ This command will download the `fineweb_edu` and prepare it for training in the
python setup/download_prepare_hf_data.py fineweb_edu <MEMORY> --data_dir ./data --seed 42 --nchunks <NCHUNKS> python setup/download_prepare_hf_data.py fineweb_edu <MEMORY> --data_dir ./data --seed 42 --nchunks <NCHUNKS>
``` ```
to download tokenizer (here llama3), use the folowing script: to download tokenizer (here llama3), use the following script:
```bash ```bash
python setup/download_tokenizer.py llama3 <SAVE_PATH> --api_key <HUGGINGFACE_TOKEN> python setup/download_tokenizer.py llama3 <SAVE_PATH> --api_key <HUGGINGFACE_TOKEN>