Const-me

48 ms·

661.
by Const-me 4y ago
I wish Windows was different in that regard. On Windows, large pages are borderline unusable. They need special security privilege SeLockMemoryPrivilege, since vista introduced UAC most processes don't have that privilege. Windows is u
662.
by Const-me 4y ago
> I have no idea how GPU stats influence neural network benchmarks I don’t have any idea either, I don’t do ML stuff professionally. On my day job I’m using the same tech (C++, SSE and AVX SIMD, DirectCompute) for a CAM/CAE applicat
663.
by Const-me 4y ago
My implementation of Whisper uses slightly over 4GB VRAM running their large multilingual model: https://github.com/Const-me/Whisper
664.
by Const-me 4y ago
That’s what I thought, and I think we have our answer. Apparently, these compute shaders are memory bound on our two GPUs, and 1080Ti has faster VRAM.
665.
by Const-me 4y ago
GP asked about the difference between whisper.cpp and my version, not OpenAI’s implementation and my version. By “the original version” in that paragraph I meant whisper.cpp. On a general note, I believe using ffmpeg or gstreamer on Windows
666.
by Const-me 4y ago
Here's another founders edition on that web site, with GDDR6 memory: https://www.techpowerup.com/gpu-specs/nvidia-geforce-rtx-306... They have a tool to find out for sure: https://www.techpowerup.com&#x
667.
by Const-me 4y ago
Wikipedia says there're two versions of 3060Ti, one has GDDR6 memory with 448 GB/second bandwidth, another one has GDDR6X memory with 608 GB/second bandwidth: https://en.wikipedia.org/wiki/GeForce_30_seri
668.
by Const-me 4y ago
> What does this mean? It probably means you flipped the combobox on the first screen. In the build on github, the only included model implementation is GPU. The other two implementations are disabled with macros, there: https:/&#x
669.
by Const-me 4y ago
I have uploaded two sample clips: https://github.com/Const-me/Whisper/tree/master/SampleClips The text files in that folder contain performance data from two computers: a desktop with nVidia 1080Ti, and
670.
by Const-me 4y ago
That latency is in my code, not in some Windows component. I’m accumulating several seconds of the audio before running the model to transcribe the buffer with these audio samples. The logic is in that method: https://github.com&
671.
by Const-me 4y ago
whisper.cpp runs on CPU. My version runs on GPU, because Windows includes a good vendor-agnostic GPU API, Direct3D. On my desktop computer, the performance difference between them is about an order of magnitude. My version is even twice as
672.
Show HN: Windows port of OpenAI's Whisper automatic speech recognition model(github.com)
43 points by Const-me 4y ago|20 comments
673.
by Const-me 4y ago
All of these APIs are very low-level, that’s why the quirks. There’s one system-specific video acceleration API which is not on the list, supports all GPUs, and just works without quirks, that’s IMFMediaEngine interface https://l
674.
by Const-me 4y ago
I thought game developers are using libraries like DirectXMath, or GLM, or the equivalents provided by game engines?
675.
by Const-me 4y ago
I did pretty much the same in HLSL just a few days ago. Here’s one idea to try. You don’t actually need 3 tiles in groupshared memory. With some care, it’s possible to go with 2 tiles there. One tile is loaded from the first source matrices
676.
by Const-me 4y ago
On computers, it's usually an even number. In fact, all numbers larger than 2^24 (FP32) or 2^54 (FP64) are even numbers.
677.
by Const-me 4y ago
This was awesome. Also, the idea of using text/assembly as an intermediate format of binary files is pure genius. I will probably use that occasionally for similar use cases.
678.
by Const-me 4y ago
They have a nice table on that page: https://github.com/facebook/zstd#benchmarks Looking at that table, I think LZ4 is a winner. The compression ratio is not too far, compression speed is slightly faster, decompression
679.
by Const-me 4y ago
I recently tried to do that as well, but failed. Specifically, I have implemented AA sort [1] but for my use case the performance was about the same as std::sort in C++, for the substantial code complexity cost. I reverted to std::sort. The
680.
by Const-me 4y ago
JPEG format normally compresses pictures in 8x8 blocks, and these blocks are aligned so the first one starts at top-left corner of the image. I think these leaks you’re talking about are limited to the inside of the 8x8 blocks. So, if you w
681.
by Const-me 4y ago
I think that might be accurate. Steam survey currently lists following top 5 Windows versions: Win10 x64 65.60% Win11 x64 27.98% Win7 x64 1.88% Win8.1 x64 0.41% Win7 x86 0.11 As you see, the most popular
682.
by Const-me 4y ago
How do you know they are AMD bugs? Another possibility is emulator bugs, caused by the fact emulator developers were using nVidia GPUs.
683.
by Const-me 4y ago
Feel free to merge my fork, about 20% faster on my computer (Ryzen 7 5700G CPU, medium.en model): https://github.com/Const-me/whisper.cpp It also contains VS2022 projects to build on Windows, your cmake project results
684.
by Const-me 4y ago
If you want SSE/AVX, I wrote that article couple years ago: http://const.me/articles/simd/simd.pdf
685.
by Const-me 4y ago
I prefer intrinsics over assembly. There're too many different assemblies: inline, MASM, NASM, FASM, YASM. They come with their unique quirks, and they complicate build. Intrinsics are more portable. It's trivial to re-compile leg
686.
by Const-me 4y ago
> The blue LED, with its narrow wavelength, enabled all sorts of modern tech, from the Blu-ray disc to flat-screen monitors. I don't think the part about monitors is true. For a first few years on the market, flat-screen monitors us
687.
by Const-me 4y ago
UTF-16 is cheaper to decode. All CPUs have branch predictors, 99.99% of all UTF-16 strings don't contain any surrogate pairs. For East Asian languages UTF-16 uses 50% less memory, these characters take 3 bytes in UTF-8, but only 2 byte
688.
by Const-me 4y ago
> Floating point operations are not deterministic in other situations That article was written in 2013. A decade has passed, and in modern world these situations became rather rare. x87 FPU is deprecated and not used by 64-bit programs.
689.
by Const-me 4y ago
> transforming a float based algorithm to integers in order to make it bit-exact Modern CPUs, and most programming languages, guarantee bit-exactness for floats most of the times, because they implement IEEE 754. Relatively easy to break
690.
by Const-me 4y ago
> For how things actually work in practice Another version of the demuxer, way simpler: https://github.com/Const-me/Vrmac/blob/master/VrmacVideo/Con... A complete mpeg4 media file, with enough fe

More ›