mirror of
https://github.com/mindverse/Second-Me.git
synced 2026-07-15 12:18:25 +00:00
f3e4d289e6
* Enhance GGUF model handling with timestamps, metadata and memory training status * Check if is_trained exists * fix * cloud service * Change the data type of the is_trained field to boolean and update the related logic to reflect this change * Change the data type of the is_trained field to boolean and update the related logic to reflect this change * Add gguf path to json file * Added model selection function, updated model list acquisition logic, and enhanced model information display * Update the model service startup logic, add integrity check for the model path, and support obtaining the model path from different fields * Service Change * full cloud service * feat: implement async cloud training process with job tracking and API key management * Progress bar modification * feat: Add Local and Cloud Training Configuration Components - Introduced LocalTrainingConfig component for configuring local training parameters. - Updated TrainingConfiguration component to include tabs for Local and Cloud training configurations. - Added API functions for setting and getting cloud service API keys. - Created useCloudProviderStore for managing cloud provider configurations. - Enhanced event utility to include a new event for showing cloud provider modal. * Refactor cloud provider and training configuration components - Updated CloudProviderModal to handle cloud service API key management. - Replaced API key handling with model configuration updates in CloudProviderModal. - Enhanced CloudTrainingConfig to manage cloud models based on API key availability. - Introduced new cloud service functions for listing available models and managing training jobs. - Modified LocalTrainingConfig to ensure default model selection and synchronization. - Updated TrainingConfiguration to manage model switching between local and cloud environments. - Refactored useCloudProviderStore to integrate cloud service API key handling. - Adjusted useTrainingStore to prioritize model name selection based on the active environment. * Stream Output * feat: Enhance training configuration and progress components - Updated LocalTrainingConfig to improve default model handling and avoid unnecessary updates. - Introduced LocalTrainingProgress component to manage local training progress display. - Refactored TrainingConfiguration to support both local and cloud training types, including updated button text and actions. - Modified TrainingProgress to conditionally render local or cloud training progress based on the selected training type. - Added cloud service functions for starting training and managing job information. - Adjusted training parameter interfaces to ensure consistency across local and cloud models. * Stream response change * feat: Enhance cloud training and inference capabilities - Updated TrainingProgress component to handle cloud training progress data and job ID. - Modified trainExposureModel to allow nullable path and added optional stageName. - Enhanced useSSE hook to support cloud model inference with new parameters. - Introduced CloudProgressData type to align cloud training progress with local training structure. - Implemented cloud inference request handling with local knowledge retrieval in cloudService. - Added utility functions for managing active cloud model state in cloudModelUtils. - Updated cloud inference endpoint to support local knowledge retrieval before cloud inference. - Refactored advanced chat service to utilize new message structure for cloud inference. - Enhanced prompt strategies to incorporate knowledge retrieval based on user messages. * feat: Delete the training parameter debugging information component * Resume training at breakpoint * Repair data redundancy * Stop system modification * fix error: reset training * fix stop and reset * Change chat reply format * Enhance cloud and local service management with status tracking and improved progress reporting - Implemented service status file management in cloud and local services to track active status and model information. - Added endpoints to start and stop cloud services, including validation for existing services. - Enhanced local service management with status checks and progress updates during document processing and chunk embedding. - Introduced real-time progress tracking for document embedding and chunk processing, allowing for incremental updates. - Improved error handling and logging throughout the service management processes. - Refactored chat request handling to intelligently route between local and cloud services based on current status. * feat:Cleaned up code comment * translate Chinese comments to English in cloud service modules * translate into chinese * feat: Enhance cloud provider configuration and training management with API key handling and tab switching logic * bug fix * Add is_trained field modification in the cloud * feat: Refactor training parameters management to separate local and cloud configurations * feat: Update training parameter types to improve type safety and consistency * feat: Add data synthesis mode to cloud training parameters and update related components * feat: The document embedding part is restored to its original state * refactor: optimize cloud training process with improved stop handling and file path updates * feat: Update the default values and merging logic of cloud training parameters to ensure parameter consistency * feat: Add API key preloading function to optimize the loading experience when the modal box is opened * feat: Optimize CloudProviderModal component, add API key preloading and state management * fix: Simplify cloud provider display by removing conditional rendering for Alibaba Cloud * feat: Update .gitignore to include job_id.json and add .gitkeep for gguf directory --------- Co-authored-by: wyx-hhhh <1360479992@qq.com>
55 lines
2 KiB
Python
55 lines
2 KiB
Python
from datetime import datetime
|
|
from sqlalchemy import Column, String, BigInteger, JSON, DateTime, Enum, Boolean
|
|
from sqlalchemy.sql import func
|
|
from lpm_kernel.common.repository.database_session import Base
|
|
import os
|
|
|
|
|
|
class Memory(Base):
|
|
"""Memory model class"""
|
|
|
|
__tablename__ = "memories"
|
|
|
|
id = Column(String(36), primary_key=True)
|
|
name = Column(String(255), nullable=False)
|
|
size = Column(BigInteger, nullable=False)
|
|
type = Column(String(50), nullable=False)
|
|
path = Column(String(1024), nullable=False)
|
|
meta_data = Column(JSON)
|
|
document_id = Column(String(36), nullable=True) # associated document ID
|
|
created_at = Column(DateTime, server_default=func.now())
|
|
updated_at = Column(DateTime, server_default=func.now(), onupdate=func.now())
|
|
status = Column(Enum("active", "deleted"), nullable=False, default="active")
|
|
is_trained = Column(Boolean, nullable=False, default=False)
|
|
|
|
def __init__(self, name, size, path, metadata=None):
|
|
import uuid
|
|
|
|
self.id = str(uuid.uuid4())
|
|
self.name = name
|
|
self.size = size
|
|
self.path = path
|
|
self.meta_data = metadata or {}
|
|
# get type from file extension, if no extension, set to 'unknown'
|
|
_, ext = os.path.splitext(path)
|
|
self.type = ext[1:].lower() if ext else "unknown"
|
|
# set default time
|
|
self.created_at = datetime.now()
|
|
self.updated_at = datetime.now()
|
|
# set default training status
|
|
self.is_trained = False
|
|
|
|
def to_dict(self):
|
|
"""Convert to dictionary, including document_id"""
|
|
result = {
|
|
"id": self.id,
|
|
"name": self.name,
|
|
"type": self.type,
|
|
"path": self.path,
|
|
"created_at": self.created_at.isoformat() if self.created_at else None,
|
|
"meta_data": self.meta_data,
|
|
"is_trained": self.is_trained,
|
|
}
|
|
if self.document_id:
|
|
result["document_id"] = self.document_id
|
|
return result
|