lunixbochs

9 ms·

31.
by lunixbochs 4y ago
Ok, my test harness is ready. My A40 box will be busy until later tonight, but on an NVIDIA A2 [1], this is the batchsize=1 throughput I'm seeing. Common Voice, default Whisper settings, card is staying at 97-100% utilization: tiny
32.
by lunixbochs 4y ago
See https://news.ycombinator.com/item?id=32929029 re accuracy, I'm working on a wider comparison. My models are generally more robust than open-source models such as Vosk and Silero, but I'm definitely interested
33.
by lunixbochs 4y ago
17x realtime on a 3090 I did some basic tests on CPU, the "small" Whisper model is in the ballpark of 0.5x realtime, which is probably not great for interactive use. My models in Talon run closer to 100x realtime on CPU.
34.
by lunixbochs 4y ago
Looks like it defaults to the model called "small". I just ran some benchmarks - M1 Max, pytorch, with a 1.29 second flac (looks like the matrix math was running on a single thread): tiny 146.522ms detect_lang 549.13
35.
by lunixbochs 4y ago
I suspect Whisper is more robust than other "SOTA" models, but this release is likely leaving a fair bit of accuracy on the table considering the amount of resources OpenAI is capable of throwing at training it. Comparing the read
36.
by lunixbochs 4y ago
My own experience agrees: the generally available "SOTA" models are not especially robust, and can be _extremely_ bad (>50% absolute error rate) at some tasks. I'll post some preliminary numbers in a sibling comment and lo
37.
by lunixbochs 4y ago
I'm able to trivially sustain 83% (250W / 300W) of the TDP of an Ampere A40 training a model the size of resnet50 with a basic pytorch training loop. Remember the lower-TDP PCIe H100 has 20% slower compute and 33% slower memory th
38.
by lunixbochs 4y ago
I've had luck using Instruments whole system profiling to debug this kind of system process
39.
by lunixbochs 4y ago
Sure, I think having dedicated acceleration blocks for transformers explains that pretty well, and is definitely exciting for people who can take advantage of it. I was more contextualizing this, where a potential 75% increase in power (on
40.
by lunixbochs 4y ago
Remember they also went from 400W on A100 to 700W on H100
41.
by lunixbochs 4y ago
> unofficially not much more than a few paragraphs There's a lot more than a few paragraphs on the wiki (which is linked right after the setup instructions in the official docs), including ~30 pages about the scripting system alone:
42.
by lunixbochs 4y ago
There's an open issue for me to implement that: https://github.com/talonvoice/talon/issues/487
43.
by lunixbochs 4y ago
I am responding in this way because your top post heavily implies details that are not the case: - "very much not open source": Talon has a lot of open-source code, including the users' entire custom voice user interface nece
44.
by lunixbochs 4y ago
Talon's alphabet is extremely carefully designed. There's de bruijn sequence style redundant phonetic information in each letter allowing the single syllable words to be mashed together at high speed without pauses (even allows fo
45.
by lunixbochs 4y ago
> very much not open source I contribute quite a bit to open source as part of my work on Talon, including the speech engine, the tools I use for model training and dataset creation, and I've upstreamed security fixes and other impr
46.
by lunixbochs 4y ago
I'll repost that in sibling. I just realized that I'd rather have people saying "ah, typical of proprietary software to not support Linux" than "your work has no value because I can't compete with you using you
47.
by lunixbochs 4y ago
Are you on Linux?
48.
by lunixbochs 4y ago
> for Talon it's more of an eye tracking+head pointing+voice clicking mouse > If you have tmj/vocal cords/neck problems you won't be able to use eye tracker as a mouse replacement. I disagree with these assertions.
49.
by lunixbochs 4y ago
I agree the non programmer use of Talon is still unpolished. That was an intentional prioritization as I work to solidify the underlying tech. Note that your best option for documentation right now is the Slack, where I am very active. I ha
50.
by lunixbochs 4y ago
Talon has an eye tracking mouse, the necessary hardware runs around $250.
51.
by lunixbochs 4y ago
You talking about Talon? It comes with all of the dependencies. Use ./run.sh to set the library path and ask on the Slack if there are any further issues. (fwiw it's not skiasharp itself, it's just the mono/skia repo, wh
52.
by lunixbochs 4y ago
My middle paragraph up-thread proposes that you can emulate it much faster than we're doing now, at full precision with integer SIMD and a specialized JIT. I'll reiterate the 80-bit softfloat stuff I've seen in use now is not
53.
by lunixbochs 4y ago
I'm excited Rosetta2 can be used in Linux VMs as of macOS Ventura. QEMU tends to be the most accurate emulation option for me on Linux and is nowhere near the speed of Rosetta. (Rosetta is quite accurate as well, it just wasn't a
54.
by lunixbochs 4y ago
TCG has historically had more of a focus on accuracy than performance. It lifts a lot of guest architectures to a lot of host architectures, and isn't particularly specialized to any given host cpu type. It lifts many instructions to C
55.
by lunixbochs 4y ago
I don't know if I agree with "impossible". There's a lot of performance left on the table with SoftFloat. A non x86 architecture can add an 80-bit FPU if they want. There are architectures with 128-bit float, and CPUs wi
56.
by lunixbochs 5y ago
Article is dated 2009. Previous discussions: - [2009] https://news.ycombinator.com/item?id=902958 - [2015] https://news.ycombinator.com/item?id=9629667
57.
by lunixbochs 5y ago
Author here. Strace seems to add more overhead than I realized at the time, but not quite as much overhead as you suggest. For instance, you're measuring the time it takes to run strace itself, while my blog post uses the times from st
58.
by lunixbochs 5y ago
[2012]
59.
by lunixbochs 5y ago
I think you could do it in such a way that you only restrict the ports available for alternate mode display output, and everything else still works. I don't think it's rude. It's about choice, just make sure it's clear t
60.
by lunixbochs 5y ago
Seems to me like you could power some of the ports with a hub/switch if you're willing to restrict which ports accept advanced peripherals. You could even use a PCIe switch to get more Thunderbolt.

More ›