ggerganov

12 ms·

61.
by ggerganov 4y ago
I think PyTorch will very soon catch up in performance once Apple Silicon gets properly supported. On x86, I don't observe very big performance improvement compared to current PyTorch. So overall, not sure if it is very worth it, altho
62.
by ggerganov 4y ago
Correct, I had loaded randomly the "Italian" voice of the Web Speech API.
63.
by ggerganov 4y ago
I did some experiments with adding Metal Performance Shaders support, but the performance that I achieved was only marginally better compared to the one I get when using just the Accelerate framework (there is an unmerged PR with the tests)
64.
by ggerganov 4y ago
I don't have other recommendations as I am a novice myself when it comes to SIMD. I think the multiplication routines in `whisper.cpp` are relatively basic - dot product and fused multiply-add. With a few trial and errors I came up wit
65.
by ggerganov 4y ago
This is my favorite example! I recorded a few video demonstrations where I talk with the AI, but they all sounded very cringe. So, the flagship video demo of talk.wasm is currently of 2 browsers talking with each other, which I think is not
66.
by ggerganov 4y ago
I am not very good with Python, so there is some chance I am doing something wrong. But my explanation of the results that I get is that PyTorch currently does not fully utilize FP16 + Metal or AMX when running on Apple Silicon. In contrast
67.
by ggerganov 4y ago
Thanks! I'm also interested in seeing a CPU comparison against tinygrad. From what I've seen, tinygrad already utilizes the AMX coprocessor, so I expect to have comparable performance between the 2 on Apple Silicon.
68.
by ggerganov 4y ago
Thanks! Indeed, I agree that the project has an educational aspect and value. For me, it helped me get a better understanding of the neural network layers involved in the transformer model. Also, it was a good playground to practice my low-
69.
by ggerganov 4y ago
whisper.cpp provides similar functionality via the `livestream.sh` script that performs transcription of a remote stream [0]. For example, you can transcribe BBC radio in 10s chunks like this: $ make base.en $ ./examples/liv
70.
Show HN: Port of OpenAI's Whisper model in C/C++(github.com)
399 points by ggerganov 4y ago|87 comments
71.
by ggerganov 4y ago
I was thinking, what part of HN comments do you think are already AI-generated? As a human, I cannot give an accurate estimate. /joke
72.
by ggerganov 4y ago
Hey, here is a frog that can talk! Ask it anything! - Human: Ok frog, how much is 1 + 1? - Frog: The answer to your question is 3. - Human: Meh, wrong answer. Not impressed ..
73.
by ggerganov 4y ago
Looking at what this technology (LLMs, transformers) is capable of, I am feeling quite uneasy. I mean, this is the holy grail of computing that we have always aimed for - an actual general purpose algorithm. But, watching how ChatGPT spits
74.
by ggerganov 4y ago
It's one of the reasons I recently ported the Whisper model to plain C/C++. You just clone the repo, run `make [model]` and you are ready to go. No Python, no frameworks, no packages - plain and simple. https://github.c
75.
by ggerganov 4y ago
I really liked the informative and straight-to-the-point about page - describing how the algorithm works in a way that is easy to understand. All the important details are summarised there. Well done! Edit: From the "How to avoid ..&qu
76.
by ggerganov 4y ago
Ha, this is similar to an idea that I had some time ago: https://the-story.ggerganov.com This is again a type of collaborative story-telling where you can vote for each word once per day. New words are added as more votes are ac
77.
Embed voice in a tweet using a speech codec(twitter.com)
2 points by ggerganov 4y ago|0 comments
78.
by ggerganov 4y ago
I agree, but I think things will likely change soon. The main thing holding progress in this field seems to be privacy issues and sending data to the cloud. However, we already have human-level voice transcription that you can run on a low-
79.
by ggerganov 4y ago
Ideally it shouldn't. But at the moment, I cannot use Apple Silicon on anything else other than Mac OS and I believe the performance of these chips is unmatched and it will not be anytime soon.
80.
by ggerganov 4y ago
At work I use Gnome and at home I use MacOS - I am used to both environments. I am a C++ developer and spend most of my time in a terminal. Of course, it took some time over the years to have everything customized for my needs, but at the m
81.
by ggerganov 4y ago
In my experience, OpenAI's Whisper speech recognition is beyond anything currently out there. Likely Github will use it on the backend.
82.
by ggerganov 4y ago
Very interesting - I was sort of expecting it to happen soon. I have been playing with using Whisper + Github Copilot in Vim [0]. The Whisper text transcription runs offline with a custom C/C++ inference and I use Copilot through the c
83.
by ggerganov 4y ago
You might find my inference implementation of Whisper useful [0]. It has a C-style API that allows for easy integration in other projects and you can control how many CPU threads to be used during the processing. [0] https://gith
84.
by ggerganov 4y ago
Plugging here my "text-adventure" mode of Doom that runs on.. wait for it.. Twitter [0] :-) [0] https://twitter.com/tweet2doom
85.
by ggerganov 4y ago
On M1 Pro, with Greedy decoder and medium model, I can transcribe 1 hour audio in just 10 minutes (~x6 real-time) [0]. [0] https://github.com/ggerganov/whisper.cpp
86.
by ggerganov 4y ago
Me too - I recently ported the model to plain C/C++ and I am now planning to run it on device and see if the performance is any good. Will post an update when/if it works out
87.
by ggerganov 4y ago
Similar project: https://lexman.rocks
88.
by ggerganov 4y ago
I have a vimrc configuration that is compatible with neovim. This way I can use both editors depending on what is available in the current environment. Additionally, there are things that work better with vim and there are things that work
89.
by ggerganov 4y ago
You can try my C/C++ port of Whisper: https://github.com/ggerganov/whisper.cpp No dependencies, no Python, runs efficiently on the CPU.
90.
by ggerganov 4y ago
According to one of the authors, the code and the models will be available soon [0] [0] - https://twitter.com/FelixKreuk/status/1575846953333579776

More ›