3 ms·

Shipping a Tauri desktop shell wrapping a Python/PyTorch inference backend is tricky to get right across platforms. A few thoughts and questions on the packagin

Shipping a Tauri desktop shell wrapping a Python/PyTorch inference backend is tricky to get right across platforms. A few thoughts and questions on the packaging and runtime side: - Sidecar packaging vs portable runtime: Are you bundling a frozen Python environment with PyTorch/CUDA embedded in the native installer, or bootstrapping wheels on first launch? Bundling torch + CUDA binaries easily pushes installers past 2-3GB, whereas bootstrapping requires reliable network access on end-user machines. - Peak memory on 6-stem Demucs: Running htdemucs_6s with default segment sizes and shifts can spike VRAM/RAM pretty heavily on longer tracks. Are you dynamically clamping the segment window or chunk overlap when falling back to CPU or 4GB/6GB consumer GPUs? - Localhost port collisions: Using FastAPI over a fixed localhost port often runs into conflicts with other local dev tools or corporate firewall/VPN endpoint blockers. If you ever hit that, switching the Tauri-to-Python IPC to named pipes (Windows) / domain sockets (Unix) or dynamically negotiating an ephemeral loopback port saves a ton of support headaches. Great work putting this together. Having zero-cloud, zero-telemetry local audio processing in a clean native UI is a huge win.

thclpr 28d ago
cool questions right there : For packaging, StemDeck uses a private portable Python runtime rather than depending on the user’s system Python. The CPU builds include the CPU version of PyTorch. The NVIDIA builds intentionally do not bundle the full CUDA stack because that would add roughly 2.5 GB and can exceed GitHub’s release-asset limit. Instead, first launch detects the GPU and driver, then installs the matching CUDA-enabled PyTorch build into the private runtime. If CUDA setup or verification fails, StemDeck restores the CPU build and remains usable. Model weights are downloaded separately and cached. For Demucs memory use, I am not dynamically adjusting the segment size or overlap based on available VRAM yet. The current worker uses Demucs splitting with its default segment selection, 0.25 overlap, and either one or two shifts depending on the selected quality mode. A GPU failure automatically retries the separation on CPU, but that is recovery rather than proactive memory management. Detecting available VRAM and selecting safer parameters before starting would be a worthwhile improvement. The desktop port issue is already handled similarly to your suggestion. StemDeck first attempts to reserve the configured port, which defaults to 8000. If that port is unavailable, it asks the operating system for a free ephemeral port and launches the backend there. The shell also gives each backend instance a unique token and verifies it through the health endpoint, so it cannot accidentally connect to another StemDeck instance or an unrelated service. Named pipes or domain sockets could still reduce the loopback surface, but they would complicate the browser and mobile interfaces, which intentionally connect to the same FastAPI backend. Thank you for the thoughtful review. These are exactly the kinds of implementation details I hoped people would challenge. :)
- yeasin-arafat 25d ago
  [flagged]

thclpr 28d ago

cool questions right there : For packaging, StemDeck uses a private portable Python runtime rather than depending on the user’s system Python. The CPU builds include the CPU version of PyTorch. The NVIDIA builds intentionally do not bundle the full CUDA stack because that would add roughly 2.5 GB and can exceed GitHub’s release-asset limit. Instead, first launch detects the GPU and driver, then installs the matching CUDA-enabled PyTorch build into the private runtime. If CUDA setup or verification fails, StemDeck restores the CPU build and remains usable. Model weights are downloaded separately and cached. For Demucs memory use, I am not dynamically adjusting the segment size or overlap based on available VRAM yet. The current worker uses Demucs splitting with its default segment selection, 0.25 overlap, and either one or two shifts depending on the selected quality mode. A GPU failure automatically retries the separation on CPU, but that is recovery rather than proactive memory management. Detecting available VRAM and selecting safer parameters before starting would be a worthwhile improvement. The desktop port issue is already handled similarly to your suggestion. StemDeck first attempts to reserve the configured port, which defaults to 8000. If that port is unavailable, it asks the operating system for a free ephemeral port and launches the backend there. The shell also gives each backend instance a unique token and verifies it through the health endpoint, so it cannot accidentally connect to another StemDeck instance or an unrelated service. Named pipes or domain sockets could still reduce the loopback surface, but they would complicate the browser and mobile interfaces, which intentionally connect to the same FastAPI backend. Thank you for the thoughtful review. These are exactly the kinds of implementation details I hoped people would challenge. :)

yeasin-arafat 25d ago
[flagged]