vrr/Second-Me
2
0
Fork 0
mirror of https://github.com/mindverse/Second-Me.git synced 2026-07-15 20:28:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
Second-Me/lpm_kernel/L2
History
Exact
JimmyZQX f5bb0dad59
Data Filtering with Gemma (#396) 
* Add code for data filtering llm judge

* Ignore log file created on root (mainly for synthetic_data_generation.log)

* Fix metadata API compatibility issues by commenting out metadata tags in LLM API calls

- Commented out metadata.tags parameters in all LLM API calls across the codebase
- This fixes compatibility issues with custom LLM providers that don't support metadata
- Affects shades generation, topics generation, wiki generation, bio QA, and question generation
- Preserves the original code structure for future re-enabling if needed

* feat: add data filtering pipeline with Ollama integration

- Add MergedDataJudge class for intelligent data filtering using Ollama Gemma
- Integrate automatic Ollama CLI installation into project setup process
- Add DATA_FILTERING step to training pipeline with concurrent processing
- Include testing for MergedDataJudge in its local main() function
- Add Ollama dependency to pyproject.toml

* feat: add automatic Ollama model cleanup after data filtering

* Add logging for outputting data filtering parameters

* fix: adjust error handling for MergedDataJudge:
- Keep original merged.json unchanged when any error occurs
- Exit filtering process immediately on errors instead of continuing with defaults
- Ensure training pipeline continues safely even if data filtering fails

* Add frontend for data filtering pipeline

* resolve data filtering quality_level error by commenting out problematic fields, change TrainProcessService back to original class definition

* fix: quote unquoted shade icons to prevent JSON parsing errors

* Fixed wiki_res.json missing due to no database connection at wiki/base.py module import

* Added scoring reasoning as part of the merged data

* fix: filter ANSI escape sequences from Ollama logs in data filtering step

* fix: Add data filtering steps to cloud training to resolve KeyError

- Added 'Data Filtering' step to cloud training progress holder
- Added data filtering step execution in cloud training service
- Added data filtering parameters to cloud training routes
- Updated frontend to send data filtering parameters
- Fixed missing except clause in cloud training service

This resolves the KeyError: 'data_filtering' when switching from cloud to local training.
2025-08-15 11:19:12 +08:00
..
data_pipeline Feature/new data (#393) 2025-07-01 15:39:04 +08:00
dpo remove additional data (#398) 2025-07-28 16:11:25 +08:00
gguf-py Feature/cloud upgrade (#388) 2025-06-23 14:21:23 +08:00
mlx_training Feature/mlx training support new (#131) 2025-04-01 10:31:09 +08:00
__init__.py Initial commit 2025-03-20 00:37:54 +08:00
convert_hf_to_gguf.py Feature/cloud upgrade (#388) 2025-06-23 14:21:23 +08:00
convert_model_to_gguf.sh Initial commit 2025-03-20 00:37:54 +08:00
convert_to_single_line.py fix/DockerWindowsCommandLineError (#138) 2025-04-02 11:47:46 +08:00
data.py Feature/new data (#393) 2025-07-01 15:39:04 +08:00
download_model.sh Added CUDA support (#228) 2025-04-25 10:20:36 +08:00
l2_generator.py Feature/new data (#393) 2025-07-01 15:39:04 +08:00
memory_manager.py Added CUDA support (#228) 2025-04-25 10:20:36 +08:00
merge_lora_weights.py Added CUDA support (#228) 2025-04-25 10:20:36 +08:00
merge_weights_for_user.sh fix/DockerWindowsCommandLineError (#138) 2025-04-02 11:47:46 +08:00
merged_data_judge.py Data Filtering with Gemma (#396) 2025-08-15 11:19:12 +08:00
note_templates.py Initial commit 2025-03-20 00:37:54 +08:00
README.md feat: support LongCoT mode for DeepSeek-R1 data synthesis and model training (#126) 2025-04-07 10:11:23 +08:00
train.py Fix/new data fix (#394) 2025-07-03 18:56:37 +08:00
train_for_user.sh Fix/new data fix (#394) 2025-07-03 18:56:37 +08:00
training_prompt.py Initial commit 2025-03-20 00:37:54 +08:00
utils.py Data Filtering with Gemma (#396) 2025-08-15 11:19:12 +08:00

README.md

Long Chain-of-Thought (CoT) Feature Implementation

Overview

This implementation adds Long Chain-of-Thought (CoT) capability to the data synthesis pipeline when using DeepSeek R1 as the base model. The feature enables multi-step reasoning for enhanced context-aware responses.

Feature Description

  • Long CoT Mode: When enabled, the system generates synthetic data with extended reasoning chains

  • DeepSeek R1 Integration: Exclusive use of DeepSeek-R1 model for CoT data generation

  • Enhanced Training: Produces models with improved long-context reasoning capabilities

Implementation Details

Configuration Options

  1. Backend Configuration:

    • Set is_cot=True in trainprocess_service.py initialization

    • Configure via train_for_user.sh with --is_cot True/False

    • Environment variables in lpm_kernel/L2/.env:

        DEEPSEEK_MODEL_NAME=deepseek-*

    DEEPSEEK_API_KEY=your_api_key

    DEEPSEEK_BASE_URL=your_base_url

Data Synthesis Pipeline

  1. Supported Data Types:

    • SelfQA data

    • Preference data

    • Diversity data

  2. Prompt Structure:

	<think>reasoning_content</think>
    <answer>final_content</answer>

  1. Model Whitelisting:

    • Only DeepSeek-R1 is allowed for CoT data generation

Code Changes

  1. Modified Files:

    • selfqa.py:

      • Added is_cot initialization option

      • Updated prompt templates

      • Modified response handling

    • preference_QA_generate.py:

      • Added CoT support

      • Enhanced question extraction

    • diversity_data_generator.py:

      • Added CoT templates

      • Updated generation logic

  2. New Functions:

    • Unified get_remote_response() function

    • Enhanced logging with tqdm integration