Const-me

56 ms·

631.
by Const-me 4y ago
Micro-ops aren’t simpler than the AMD64 instructions, the complexity is about the same. For instance, the following instruction vfmadd231ps ymm3, ymm1, YMMWORD PTR [rax-256] does quite a few things (memory load, and 8-wide fuse
632.
by Const-me 4y ago
I don’t know what the chances are, but over time it might happen. Economy of scale should not be underestimated. 30 years ago, there was a consensus x86 architecture was only good for small computers, while servers need faster specially-des
633.
by Const-me 4y ago
I don’t think OpenAI needs a new ecosystem. I think they just need a cost-efficient way to run inference of their ML models, for large userbase. That smaller task is way more manageable. It shouldn’t take a decade, maybe a month or two. Vul
634.
by Const-me 4y ago
> You could say that about some of nvidia's other GPUs too. Technically, yes. Practically, nVidia forbids to deploy their other GPUs in data centers, in the EULA of their drivers: https://www.datacenterdynamics.com/e
635.
by Const-me 4y ago
> a lot of new models require the higher-end 80GB A100 variants For training, yeah. But at least in some cases, inference can be optimized to use less memory. For their large Whisper model, OpenAI says they need 10GB of VRAM: https:&#x2
636.
by Const-me 4y ago
A100 has 1555 GB/sec of memory bandwidth. 50% more than Radeon 7900 XTX, but the AMD GPU costs 10x less. nVidia’s top management did amazing job with their strategy. They have spent decades, and millions of dollars, developing and prom
637.
by Const-me 4y ago
> I cannot imagine using it in any other way The good thing about that library is the permissive license. Here’s a project where I managed to reuse substantial parts of that library by copy-pasting and reworking the code: https:/&#
638.
by Const-me 4y ago
PSM includes the dependencies, instead of getting rid of them. Not flexible enough. An example is logging. Geogram includes a logger implementation which uses console streams. GUI apps don't have these streams, they sometimes even dead
639.
by Const-me 4y ago
The algorithms are awesome. For instance, their Delaunay tetrahedralization algorithms are state of the art, in terms of both accuracy and performance. However, software engineering and API design could have been much better. That’s not qui
640.
by Const-me 4y ago
You could also port that code from nVidia CUDA somewhere else. If you wanna run on servers, Vulkan Compute will probably do the trick. I have ported ML inference to Windows desktops with DirectCompute https://github.com/Cons
641.
by Const-me 4y ago
> All they will ever offer is a paid prompt. They sometimes open their older models: https://github.com/Const-me/Whisper
642.
by Const-me 4y ago
> what is the most articulate and sensible piece on why the most likely culprit is Russia? I think this video is articulate and sensible: https://www.youtube.com/watch?v=hk-0qJXyido
643.
by Const-me 4y ago
No, you don’t have to. For development, Microsoft ships free VM images, there: https://developer.microsoft.com/en-us/windows/downloads/virt... VirtualBox is free and open-source software, GPLv3 license.
644.
by Const-me 4y ago
> To develop windows desktop app you have to pay MS tax. No, making windows desktop apps is free. Microsoft even has a free IDE, with a caveat that it's only usable by independent developers, or small businesses.
645.
by Const-me 4y ago
Tensor cores are simply multiplying matrices. Relatively easy to do with a compute shader. Here's an example of such shader in D3D11: https://github.com/Const-me/Whisper/blob/master/ComputeShade...
646.
by Const-me 4y ago
I wonder how this compares to the OS built-in NTFS compression? Windows shell has "Compress contents to save disk space" checkbox in folder properties. Usually, that compressed flag is inherited by new files created in a folder wi
647.
by Const-me 4y ago
> it's more reliable Hardware codecs are complicated. See V4L2 Linux kernel API, it exposes hardware codecs without doing anything on top. GPU vendors have higher chances of correctly using their hardware, compared to Discord develo
648.
by Const-me 4y ago
> the process of probing the GPU for what codecs it supports I think you can ask that question to Windows, instead of probing. Call MFTEnumEx API function with MFT_CATEGORY_VIDEO_ENCODER category, MFT_ENUM_FLAG_HARDWARE | MFT_ENUM_FLAG_T
649.
by Const-me 4y ago
I’ve asked ChatGPT about hardware. Here’s the response: > As for your specific computer, with 64 GB of RAM and a high-performance GPU like the GeForce 1080 Ti, it should have sufficient resources to run a language model like me for many
650.
by Const-me 4y ago
> do you actually have any concrete example of any difficulty setting up a C++ project? Take a look: https://stackoverflow.com/questions/75270938/thread-pool-wit...
651.
by Const-me 4y ago
Never heard of that software. Indeed, it says “VueScan is compatible on Windows, macOS and Linux.” Thanks for the tip! Will give it a try next time I need to scan something.
652.
by Const-me 4y ago
I also have an old Mustek scanner with outdated drivers, with no hardware issues. Luckily, my scanner is USB, and VMware Workstation allows to forward complete USB devices to a guest OS. When I need to scan something, I fire up a virtual ma
653.
by Const-me 4y ago
Doing that for decades. An app for Windows phone, downloaded 140k times: https://github.com/Const-me/SkyFM Cross-platform graphics library for .NET: https://github.com/Const-me/Vrmac Recently, off
654.
by Const-me 4y ago
Sometimes. Very rarely. The main issue with recursion is not even speed, it's the fact the stack is not too large. As far as I remember, by default 2MB on Windows. You can easily keep 2GB of data in std::vector or an equivalent. And if
655.
by Const-me 4y ago
> direct access to the command buffers of the GPU Impossible given the platform limitations at that time, on Linux that code works on top of GLES 3.1. I didn't want too many Windows-only features there. > No matrix transform rout
656.
by Const-me 4y ago
I think this needs much more complexity to be useful. For the rendering, ideally it needs GPU support. Input needs much more work, here's an overview for Windows: http://blog.ngedit.com/2005/06/13/whats-b
657.
by Const-me 4y ago
> `_mm_blend_ps` for sse4 That instruction uses a mask provided at compile time; it encodes the mask as a part of the instruction. SSE 4 set has a similar one, `_mm_blendv_ps`, which does take the mask from another vector register.
658.
by Const-me 4y ago
> Unlike CUDA, there's no capable API that works on the whole range Microsoft's D3D is very capable, and vendor-agnostic, same code works on all GPUs because the user-mode half of the GPU driver includes a JIT compiler for thes
659.
by Const-me 4y ago
Here's my D3D11 implementation of speech-to-text https://github.com/Const-me/Whisper With medium model it needs 1.43 GB of assets, 2 GB of VRAM, and on gaming GPUs works at 10x realtime speed. These performance fi
660.
by Const-me 4y ago
> not only are more interesting libraries available, there seem to be generally high quality versions I wonder what's an equivalent of Eigen? https://eigen.tuxfamily.org/index.php?title=Main_Page > if you’re on w

More ›