* feature: use uv to setup python environment
* TrainProcessService add singleten method: get_instance
* feat: fix code
* Added CUDA support (#228)
* Add CUDA support
- CUDA detection
- Memory handling
- Ollama model release after training
* Fix logging issue
added cuda support flag so log accurately reflected cuda toggle
* Update llama.cpp rebuild
Changed llama.cpp to only check if cuda support is enabled and if so rebuild during the first build rather than each run
* Improved vram management
Enabled memory pinning and optimizer state offload
* Fix CUDA check
rewrote llama.cpp rebuild logic, added manual y/n toggle if user wants to enable cuda support
* Added fast restart and fixed CUDA check command
Added make docker-restart-backend-fast to restart the backend and reflect code changes without causing a full llama.cpp rebuild
Fixed make docker-check-cuda command to correctly reflect cuda support
* Added docker-compose.gpu.yml
Added docker-compose.gpu.yml to fix error on machines without nvidia gpu and made sure "\n" is added before .env modification
* Fixed cuda toggle
Last push accidentally broke cuda toggle
* Code review fixes
Fixed errors resulting from removed code:
- Added return save_path to end of save_hf_model function
- Rolled back download_file_with_progress function
* Update Makefile
Use cuda by default when using docker-restart-backend-fast
* Minor cleanup
Removed unnecessary makefile command and fixed gpu logging
* Delete .gpu_selected
* Simplified cuda training code
- Removed dtype setting to let torch automatically handle it
- Removed vram logging
- Removed Unnecessary/old comments
* Fixed gpu/cpu selection
Made "make docker-use-gpu/cpu" command work with .gpu_selected flag and changed "make docker-restart-backend-fast" command to respect flag instead of always using gpu
* Fix Ollama embedding error
Added custom exception class for Ollama embeddings, which seemed to be returning keyword arguments while the Python exception class only accepts positional ones
* Fixed model selection & memory error
Fixed training defaulting to 0.5B model regardless of selection and fixed "free(): double free detected in tcache 2" error caused by cuda flag being passed incorrectly
* fix: train service singlten
---------
Co-authored-by: Zachary Pitroda <30330004+zpitroda@users.noreply.github.com>
* Add CUDA support
- CUDA detection
- Memory handling
- Ollama model release after training
* Fix logging issue
added cuda support flag so log accurately reflected cuda toggle
* Update llama.cpp rebuild
Changed llama.cpp to only check if cuda support is enabled and if so rebuild during the first build rather than each run
* Improved vram management
Enabled memory pinning and optimizer state offload
* Fix CUDA check
rewrote llama.cpp rebuild logic, added manual y/n toggle if user wants to enable cuda support
* Added fast restart and fixed CUDA check command
Added make docker-restart-backend-fast to restart the backend and reflect code changes without causing a full llama.cpp rebuild
Fixed make docker-check-cuda command to correctly reflect cuda support
* Added docker-compose.gpu.yml
Added docker-compose.gpu.yml to fix error on machines without nvidia gpu and made sure "\n" is added before .env modification
* Fixed cuda toggle
Last push accidentally broke cuda toggle
* Code review fixes
Fixed errors resulting from removed code:
- Added return save_path to end of save_hf_model function
- Rolled back download_file_with_progress function
* Update Makefile
Use cuda by default when using docker-restart-backend-fast
* Minor cleanup
Removed unnecessary makefile command and fixed gpu logging
* Delete .gpu_selected
* Simplified cuda training code
- Removed dtype setting to let torch automatically handle it
- Removed vram logging
- Removed Unnecessary/old comments
* Fixed gpu/cpu selection
Made "make docker-use-gpu/cpu" command work with .gpu_selected flag and changed "make docker-restart-backend-fast" command to respect flag instead of always using gpu
* Fix Ollama embedding error
Added custom exception class for Ollama embeddings, which seemed to be returning keyword arguments while the Python exception class only accepts positional ones
* Fixed model selection & memory error
Fixed training defaulting to 0.5B model regardless of selection and fixed "free(): double free detected in tcache 2" error caused by cuda flag being passed incorrectly
* split train process service
* split train process service
* split train param
* save train param to file
---------
Co-authored-by: kevin-mindverse <kevin@mindverse.ai>
* change progress order
* feat: change progress frontend type
* feat: update _load_progress for progress's new format
* fix: fix formatUnderscoreToName
* fix:simplify TrainProgress with direct JSON structure and mapping
* fix：add necessary accessors (getters and setters) to maintain compatibility with existing code while adopting the new data structure.
* fix: train log jump
* fix: fix current step update failed
* fix:fix stop problem
* fix trainprogress has no attribute status problem
* fix: merge master
---------
Co-authored-by: Ye Xiangle <yexiangle@mail.mindverse.ai>
* feat(trainprocess):add receive & get training params
* feat: support low/standard option for L2 data generate
* feat:improve training script invocation with direct bash command and internationalize comments
* fix: restore code
* feat: add error handling for GraphRAG index and modify model save frequency(epoch -> steps)
* feat: support low/medium/high mode for L2 data generation
* feat: set defaut value
* feat: add train params
* feat: use ScriptExecutor for training process and check return code
* fix: fix params round
* feat:use subprocess instead of script_executor to keep logs
* feat:hot fix name 'monitor_result' is not defined
* fix: delete useless message
* fix: fix params status
* feat: add status suspended
* feat:fix stop status
* fix: fix resume status
* fix: disable error
---------
Co-authored-by: Crabboss Mr <1123357821@qq.com>
Co-authored-by: ryangyuan <ryangyuan@mail.mindverse.ai>
* Potential fix for code scanning alert no. 110: Uncontrolled data used in path expression
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* Potential fix for code scanning alert no. 109: Uncontrolled data used in path expression
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
