mirror of
https://github.com/ruvnet/RuVector.git
synced 2026-05-23 12:55:26 +00:00
23684ed1b9
* feat(musica): structure-first audio separation via dynamic mincut Complete audio source separation system using graph partitioning instead of traditional frequency-first DSP. 34 tests pass, all benchmarks validated. Modules: - stft: Zero-dep radix-2 FFT with Hann window and overlap-add ISTFT - lanczos: SIMD-optimized sparse Lanczos eigensolver for graph Laplacians - audio_graph: Weighted graph construction (spectral, temporal, harmonic, phase edges) - separator: Spectral clustering via Fiedler vector + mincut refinement - hearing_aid: Binaural streaming enhancer (<0.13ms latency, <8ms budget PASS) - multitrack: 6-stem separator (vocals/bass/drums/guitar/piano/other) - crowd: Distributed speaker identity tracker (hierarchical sensor fusion) - wav: 16/24-bit PCM WAV I/O with binaural test generation - benchmark: SDR/SIR/SAR evaluation with comparison baselines Key results: - Hearing aid: 0.09ms avg latency (87x margin under 8ms budget) - Lanczos: Clean Fiedler cluster split in 4 iterations (16us) - Multitrack: Perfect mask normalization (0.0000 sum error) - WAV roundtrip: 0.000046 max quantization error https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * refactor(musica/crowd): use DynamicGraph for local + global graphs Agent-improved crowd tracker using Gaussian-kernel similarity edges, dense Laplacian spectral bipartition, and exponential moving average embedding merging. All 34 tests pass. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * enhance(musica/lanczos): add batch_lanczos with cross-frame alignment Adds batch processing mode for computing eigenpairs across multiple STFT windows with automatic Procrustes sign alignment between frames. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * enhance(musica/hearing_aid): improve binaural pipeline with mincut refinement Agent-enhanced hearing aid module adds dynamic mincut boundary refinement via MinCutBuilder, temporal coherence bias, and improved speech scoring. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * docs(musica): comprehensive README with benchmarks and competitive analysis Detailed documentation covering all 9 modules, usage examples, benchmark results, competitive positioning vs SOTA, and improvement roadmap. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): add 6 enhancement modules — 55 tests passing New modules: - multi_res: Multi-resolution STFT (short/medium/long windows per band) - phase: Griffin-Lim iterative phase estimation - neural_refine: Tiny 2-layer MLP mask refinement (<100K params) - adaptive: Grid/random/Bayesian graph parameter optimization - streaming_multi: Frame-by-frame streaming 6-stem separation - wasm_bridge: C-FFI WASM interface for browser deployment https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica/wasm): add browser demo with drag-and-drop separation UI Self-contained HTML+CSS+JS demo for WASM-based audio separation. Dark theme, waveform visualization, Web Audio playback. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): HEARmusica — Rust hearing aid DSP framework (Tympan port) Complete hearing aid processing pipeline with 10 DSP blocks: - BiquadFilter: 8 filter types (LP/HP/BP/notch/allpass/peaking/shelves) - WDRCompressor: Multi-band WDRC with soft knee + attack/release - FeedbackCanceller: NLMS adaptive filter - GainProcessor: Audiogram fitting + NAL-R prescription - GraphSeparatorBlock: Fiedler vector + dynamic mincut (novel) - DelayLine: Sample-accurate circular buffer - Limiter: Brick-wall output protection - Mixer: Weighted signal combination - Pipeline: Sequential block runner with latency tracking - 4 preset configs: standard, speech-in-noise, music, max-clarity ADR-143 documents architecture decisions. 87 tests passing. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): 8-part benchmark suite + HEARmusica pipeline benchmarks Part 7: HEARmusica pipeline — 4 presets benchmarked (0.01-0.75ms per block) Part 8: Streaming 6-stem separation (0.35ms avg, 0.68ms max) Updated README with benchmark results and 87-test / 11K-line stats. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): add enhanced separator, evaluation module, and adaptive tuning Complete the remaining optimization modules: - enhanced_separator.rs: multi-res STFT + neural mask refinement pipeline with comparison report - evaluation.rs: realistic audio signal generation (speech, drums, bass, noise) and full BSS metrics (SDR/SIR/SAR) - Adaptive parameter tuning benchmark (Part 9) with random search - Enhanced separator comparison (Part 10) across 4 modes - Real audio evaluation (Part 11) across 4 scenarios - WASM build verification script 100 tests passing, 11-part benchmark suite validated. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): add candle-whisper transcription integration (ADR-144) Pure-Rust speech transcription pipeline using candle-whisper: - ADR-144: documents candle-whisper choice over whisper-rs (pure Rust, no C++ deps) - transcriber.rs: Whisper pipeline with feature-gated candle deps, simulated transcriber for offline benchmarking, SNR-based WER estimation, resampling - Part 12 benchmark: before/after separation quality for transcription across 3 scenarios (two speakers, speech+noise, cocktail party) - 109 tests passing, 12-part benchmark suite validated Enable with: cargo build --features transcribe https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): add real audio evaluation with public domain WAV files - real_audio.rs: loads ESC-50, Signalogic speech, SampleLib music WAVs - 6 real-world separation scenarios: speech+rain, male+female, music+crowd, birds+bells, speech+dog, speech+music - Automatic resampling, mono mixing, SNR-controlled signal mixing - Part 13 benchmark with per-scenario SDR measurement - Download script (scripts/download_test_audio.sh) for test audio - .gitignore for test_audio/ binary files - 115 tests passing, 13-part benchmark suite https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * perf(musica): optimize critical hot loops across 5 modules Profiler-guided optimizations targeting 2-3x cumulative speedup: - stft.rs: reuse FFT buffers across frames (eliminates per-frame allocation) - audio_graph.rs: cache frame base indices, precompute harmonic bounds - separator.rs: K-means early stopping on convergence (saves ~15 iterations) - lanczos.rs: selective reorthogonalization (full every 5 iters, partial otherwise) - neural_refine.rs: manual loop for auto-vectorizable matrix multiply 115 tests passing. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): add advanced SOTA separator with Wiener filtering, cascaded refinement, and multi-resolution fusion Implements three techniques to push separation quality toward SOTA: - Wiener filter mask refinement (M_s = |S_s|^p / sum_k |S_k|^p) - Cascaded separation with iterative residual re-separation and decaying alpha blend - Multi-resolution graph fusion across 256/512/1024 STFT windows Part 14 benchmark compares basic vs advanced on 3 scenarios. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * fix(musica): adaptive quality selection in advanced separator Add permutation-invariant SDR evaluation, source alignment via cross-correlation for multi-resolution fusion, and composite quality metric (independence + reconstruction accuracy) for adaptive pipeline selection. Advanced now consistently matches or beats basic: +3.0 dB on well-separated, +1.5 dB on harmonic+noise. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): add instantaneous frequency graph edges for close-tone separation Add IF-based temporal edge weighting and cross-frequency IF edges. Instantaneous frequency = phase advance rate across STFT frames. Bins tracking the same sinusoidal component get stronger edges, improving separation of close tones (400Hz+600Hz: +0.3 → +2.3 dB). https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * refactor(musica): best-of-resolutions strategy replaces lossy mask interpolation Instead of interpolating masks between STFT resolutions (which introduces artifacts), try each window size independently with Wiener refinement, then pick the best by composite quality score. Well-separated tones: +4.7 → +18.1 dB (+13.4 dB improvement). https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): multi-exponent Wiener search and energy-balanced quality metric Try Wiener exponents 1.5/2.0/3.0 per resolution for broader search. Add energy balance to quality score (penalizes degenerate partitions). Close tones: consistently +1.4-1.8 dB over basic. 121 tests pass. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): SOTA push — 8 major improvements across all modules Quick wins: - 8-bit and 32-bit WAV support in wav.rs (ESC-50 noise files now load) - SDR variance reduction: seeded Fiedler init with 100 iterations Core separation improvements: - Multi-eigenvector spectral embedding: Lanczos k>2 eigenvectors with spectral k-means for multi-source separation - Onset/transient detection edges: spectral flux onset detector groups co-onset bins for better drum/percussion separation - Spatial covariance model: IPD/ILD-based stereo separation with far-field spatial model for binaural hearing aids Research & benchmarking: - Learned graph weights via Nelder-Mead simplex optimization - MUSDB18 SOTA comparison framework with published results (Open-Unmix, Demucs, HTDemucs, BSRNN) - Longer signal benchmarks (2-5s realistic duration) Parts 15-17 added to benchmark suite. 131 tests pass. https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): terminal visualizer, weight optimization, multi-source separation Add Part 18-20 to benchmark suite: - Terminal audio visualizer (waveform, spectrum, masks, Lissajous, separation comparison) using ANSI escape codes and Unicode block characters, zero dependencies - Nelder-Mead weight optimization benchmark with 3 training scenarios - Multi-source (3+4 source) separation benchmark with permutation-invariant SDR - Public evaluate_params wrapper for learned_weights module 276 tests passing (139 lib + 137 bin). https://claude.ai/code/session_015KxNFsV5GQjQn6u9HbS9MK * feat(musica): STFT padding, Lanczos batch improvements, WASM bridge cleanup Improve STFT module with proper zero-padding and power-of-two FFT sizing. Refactor Lanczos resampler batch processing and WASM bridge for clarity. Clean up react_memo_cache_sentinel research files. Co-Authored-By: claude-flow <ruv@ruv.net> --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Reuven <cohen@ruv-mac-mini.local> |
||
|---|---|---|
| .. | ||
| musica | ||
| btsp_usage.rs | ||
| monitoring_example.md | ||
| sparsevec_examples.sql | ||