backend-infra-engineer: Post v0.3.9-hotfix7 snapshot (build cleanup)
This commit is contained in:
scawful
172
docs/internal/hand-off/HANDOFF_MUSIC_AUDIO_SPEED.md
Normal file
172
docs/internal/hand-off/HANDOFF_MUSIC_AUDIO_SPEED.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# MusicEditor 1.5x Audio Speed Bug - Handoff Document
|
||||
|
||||
**Date:** 2025-12-05
|
||||
**Status:** Unresolved
|
||||
**Priority:** High
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The MusicEditor plays audio at approximately 1.5x speed. The exact ratio (48000/32040 = 1.498) indicates that **samples generated at 32040 Hz are being played at 48000 Hz without proper resampling**.
|
||||
|
||||
Additionally, there's a "first play" issue where clicking Play produces no audio the first time, but stopping and playing again works (at 1.5x speed).
|
||||
|
||||
## Audio Pipeline Overview
|
||||
|
||||
```
|
||||
MusicPlayer::Update() [called at ~60 Hz]
|
||||
│
|
||||
▼
|
||||
Emulator::RunAudioFrame()
|
||||
│
|
||||
├─► Snes::RunAudioFrame()
|
||||
│ │
|
||||
│ ├─► cpu_.RunOpcode() loop until vblank
|
||||
│ │ └─► RunCycle() → CatchUpApu() → apu_.RunCycles()
|
||||
│ │ └─► DSP generates ~533 samples at 32040 Hz
|
||||
│ │
|
||||
│ └─► [At vblank] dsp.NewFrame() sets lastFrameBoundary
|
||||
│
|
||||
└─► snes_.SetSamples() → dsp.GetSamples()
|
||||
│
|
||||
└─► Reads ~533 samples from DSP ring buffer
|
||||
│
|
||||
▼
|
||||
audio_backend->QueueSamplesNative(samples, 533, 2, 32040)
|
||||
│
|
||||
├─► SDL_AudioStreamPut(samples) at 32040 Hz
|
||||
│
|
||||
└─► SDL_AudioStreamGet(resampled) → SDL_QueueAudio()
|
||||
└─► Output at 48000 Hz (resampled by SDL)
|
||||
```
|
||||
|
||||
## What Has Been Verified Working
|
||||
|
||||
### 1. APU Timing (VERIFIED CORRECT)
|
||||
- APU runs at ~1,024,000 Hz (tests pass)
|
||||
- DSP generates samples at ~32040 Hz (tests pass)
|
||||
- ~533 samples generated per NTSC frame
|
||||
|
||||
### 2. SDL_AudioStream Resampling (VERIFIED CORRECT)
|
||||
Diagnostic logs confirm correct resampling ratio:
|
||||
```
|
||||
QueueSamplesNative: In=2132 bytes (32040Hz) → Out=3192 bytes (48000Hz)
|
||||
Resampling ratio: 1.497 (expected: 1.498)
|
||||
```
|
||||
|
||||
### 3. Audio Backend Configuration (VERIFIED CORRECT)
|
||||
- SDL audio device opens at 48000 Hz
|
||||
- SDL_AudioStream created: 32040 Hz stereo → 48000 Hz stereo
|
||||
- `audio_stream_enabled_ = true` confirmed in logs
|
||||
|
||||
### 4. Shared Audio Backend (IMPLEMENTED)
|
||||
- MusicPlayer's `audio_emulator_` now uses external backend from main emulator
|
||||
- `Emulator::RunAudioFrame()` uses `audio_backend()` accessor (not direct member)
|
||||
- Single SDL device shared between main emulator and MusicPlayer
|
||||
|
||||
## What Has Been Tried and Ruled Out
|
||||
|
||||
### 1. Duplicate NewFrame() Calls - REMOVED
|
||||
Preview methods had explicit `dsp.NewFrame()` calls that conflicted with the internal call in `RunAudioFrame()`. These were removed but didn't fix the issue.
|
||||
|
||||
### 2. Audio Backend Member vs Accessor - FIXED
|
||||
`Emulator::RunAudioFrame()` was using `audio_backend_` directly instead of `audio_backend()` accessor. When external backend was set, `audio_backend_` was null, so no audio was queued. Fixed to use accessor.
|
||||
|
||||
### 3. Two SDL Audio Devices - FIXED
|
||||
Main emulator and MusicPlayer were creating separate SDL audio devices. Implemented `SetExternalAudioBackend()` to share a single device. Verified in logs that same device ID is used.
|
||||
|
||||
### 4. Initialization Order - VERIFIED CORRECT
|
||||
- `SetSharedAudioBackend()` called in `MusicEditor::Initialize()`
|
||||
- `EnsureAudioReady()` sets external backend before `EnsureInitialized()`
|
||||
- Resampling configured before playback starts
|
||||
|
||||
### 5. First Play Silence - PARTIALLY UNDERSTOOD
|
||||
Logs show the device is already "playing" with stale audio from main emulator when MusicPlayer starts. The exclusivity callback sets `running=false` on main emulator, but this may not immediately stop audio generation.
|
||||
|
||||
## Current Code State
|
||||
|
||||
### Key Files Modified
|
||||
- `src/app/emu/emulator.h` - Added `SetExternalAudioBackend()`, `audio_backend()` accessor
|
||||
- `src/app/emu/emulator.cc` - `RunAudioFrame()` and `ResetFrameTiming()` use accessor
|
||||
- `src/app/editor/music/music_player.h` - Added `SetSharedAudioBackend()`
|
||||
- `src/app/editor/music/music_player.cc` - Uses shared backend, removed duplicate NewFrame() calls
|
||||
- `src/app/editor/music/music_editor.cc` - Shares main emulator's backend with MusicPlayer
|
||||
- `src/app/emu/audio/audio_backend.cc` - Added diagnostic logging
|
||||
|
||||
### Diagnostic Logging Added
|
||||
- `QueueSamplesNative()` logs input/output byte counts and resampling ratio
|
||||
- `GetStatus()` logs device ID and queue state
|
||||
- `Clear()` logs device ID and queue before/after
|
||||
- `Play()` logs device status transitions
|
||||
- `RunAudioFrame()` logs which backend is being used (external vs owned)
|
||||
|
||||
## Remaining Hypotheses
|
||||
|
||||
### 1. SDL_AudioStream Not Actually Being Used
|
||||
**Theory:** Despite logs showing resampling, audio might be taking a different path.
|
||||
**Investigation:** Add logging at every audio queue call site to trace actual execution path.
|
||||
|
||||
### 2. Frame Timing Issue
|
||||
**Theory:** `MusicPlayer::Update()` might not be called at the expected rate, or `RunAudioFrame()` might be called multiple times per frame.
|
||||
**Investigation:** Add frame timing logs to verify Update() is called at ~60 Hz and RunAudioFrame() once per call.
|
||||
|
||||
### 3. DSP Sample Extraction Bug
|
||||
**Theory:** `dsp.GetSamples()` might return wrong number of samples or from wrong position.
|
||||
**Investigation:** Log actual sample counts returned by GetSamples() vs expected (~533).
|
||||
|
||||
### 4. Main Emulator Still Generating Audio
|
||||
**Theory:** Even with `running=false`, main emulator's Update() might still be called and generating audio.
|
||||
**Investigation:** Add logging to main emulator's audio generation path to verify it stops when MusicPlayer is active.
|
||||
|
||||
### 5. Audio Stream Bypass Path
|
||||
**Theory:** There might be a code path that calls `QueueSamples()` (direct, non-resampled) instead of `QueueSamplesNative()`.
|
||||
**Investigation:** Search for all `QueueSamples` calls and verify none are being hit during music playback.
|
||||
|
||||
### 6. Resampling Disabled Mid-Playback
|
||||
**Theory:** `audio_stream_config_dirty_` or another flag might disable resampling during playback.
|
||||
**Investigation:** Add logging to `SetAudioStreamResampling()` to catch any disable calls.
|
||||
|
||||
## Suggested Next Steps
|
||||
|
||||
1. **Add comprehensive tracing** to follow a single frame of audio from DSP generation through to SDL queue
|
||||
2. **Verify frame timing** - confirm Update() runs at expected rate
|
||||
3. **Check for bypass paths** - ensure all audio goes through QueueSamplesNative()
|
||||
4. **Monitor resampling state** - ensure it stays enabled throughout playback
|
||||
5. **Test with simpler case** - generate known test tone and verify output rate
|
||||
|
||||
## Test Commands
|
||||
|
||||
```bash
|
||||
# Build with debug
|
||||
cmake --preset mac-dbg && cmake --build build --target yaze -j8
|
||||
|
||||
# Run with logging visible
|
||||
./build/bin/Debug/yaze.app/Contents/MacOS/yaze 2>&1 | grep -E "(AudioBackend|MusicPlayer|Emulator)"
|
||||
|
||||
# Run audio timing tests
|
||||
ctest --test-dir build -R "apu_timing|dsp_sample" -V
|
||||
```
|
||||
|
||||
## Key Constants
|
||||
|
||||
| Value | Meaning |
|
||||
|-------|---------|
|
||||
| 32040 Hz | Native SNES DSP sample rate |
|
||||
| 48000 Hz | SDL audio device sample rate |
|
||||
| 1.498 | Correct resampling ratio (48000/32040) |
|
||||
| 533 | Samples per NTSC frame at 32040 Hz |
|
||||
| ~60.0988 Hz | NTSC frame rate |
|
||||
| 1,024,000 Hz | APU clock rate |
|
||||
|
||||
## Files to Investigate
|
||||
|
||||
| File | Relevance |
|
||||
|------|-----------|
|
||||
| `src/app/editor/music/music_player.cc` | Main playback logic, Update() loop |
|
||||
| `src/app/emu/emulator.cc` | RunAudioFrame(), audio queuing |
|
||||
| `src/app/emu/audio/audio_backend.cc` | SDL audio, resampling |
|
||||
| `src/app/emu/audio/dsp.cc` | Sample generation, GetSamples() |
|
||||
| `src/app/emu/snes.cc` | RunAudioFrame(), SetSamples() |
|
||||
|
||||
## Contact
|
||||
|
||||
Previous investigation done by Claude Code agents. See git history for detailed changes.
|
||||
Reference in New Issue
Block a user