imjonse

44 ms·

181.
by imjonse 2y ago
Unsloth have a bunch of notebooks that actually tune LLMs on the free Colab tier. https://github.com/unslothai/unsloth
182.
by imjonse 2y ago
It's not wrong, but "wrong" as both comments above had :)
183.
by imjonse 2y ago
This is calling out to a proprietary API, it is not happening inside Colab.
184.
by imjonse 2y ago
It has been for 20 years :) https://lists.ubuntu.com/archives/ubuntu-announce/2004-Septe...
185.
by imjonse 2y ago
On gitlab at least it is merge request, but the platform using the "wrong" term is more popular.
186.
by imjonse 2y ago
No GPT in the name, so maybe not transformer based?
187.
by imjonse 2y ago
The article itself has this towards the end: " There are big problems that remain. None of the above should give us reason to become complacent. On the contrary, it shows us that a lot of work still needs to be done — accomplishing the
188.
by imjonse 2y ago
Wonder why no Llama-3.1-8B based variant if the new training method has such good results. UPDATE: didn't work well https://x.com/mattshumer_/status/1831775436420083753?t=flm41...
189.
by imjonse 2y ago
My bad, I did not read it carefully.
190.
by imjonse 2y ago
Unclear how much of their coding knowledge is in the space of syntax/semantics of a given language and how much in the latent space that generalizes across languages and logic in general. If I were to guess I'd say 80% is in the l
191.
by imjonse 2y ago
I agree site authors should be able to tell what content they would like to be used for LLM training (even though that opinion will likely be ignored by LLM training scrapers), but the format of it is really up to those gathering and cleani
192.
by imjonse 2y ago
My comment is in the context of 'their own Scheme implementation'. Since they already handle some .scm files internally it's not that wild to pick Scheme over other languages as the base for plugins. It's a power user ed
193.
by imjonse 2y ago
My first multicursor experience is with Helix (it goes to show what it means to have it built in instead of trying to find plugins for it) so it may be unsatisfactory for people who have used similar functionality in other editors. I use it
194.
by imjonse 2y ago
I should have specified internal config files for the runtime queries that you mention. Those are .scm files.
195.
by imjonse 2y ago
They already have Scheme based config files and I'd say it is simpler that Lua.
196.
by imjonse 2y ago
A plugin system is much less of an issue when Helix has built-in functionality for most of what comes as plugins in Vim. That said integration with git or with code assistants/chat is something that is probably better done via plugins
197.
by imjonse 2y ago
My totally unscientific model is that memories are run length encoded, so the monotonous periods' length is compressed, whereas if they are more diverse, the more info you have, so it needs an expanded storage which is proportional to
198.
by imjonse 2y ago
Is this for Ampere and newer only as FA2?
199.
by imjonse 2y ago
It is still an excellent article if you care about how the GPU works. Triton doesn't magically hide all the hardware details.
200.
by imjonse 2y ago
MS and Google keep releasing small models like Phi and Gemma, I don't think they want to hide the fact these can be very capable.
201.
by imjonse 2y ago
The MBPP column should bold DeepSeek as it has a better score than Codestral.
202.
by imjonse 2y ago
TIL that 40% of cancers are preventable.
203.
by imjonse 2y ago
It is one of the few professions where at least in theory you can create something that gets used by millions, with relatively few resources required. Few other professions are like this (being a celebrity, politician, power broker) and tho
204.
by imjonse 2y ago
YOLO 9 and YOLO 10 were both published after this paper and according to them they are better than RTDETR.
205.
by imjonse 2y ago
This convolutional view and fast GPU kernels for FFT were used in some pre-Mamba state space models for long sequence modeling where the polynomial is the input sequence. The Hazy Research blog posts from 2020-2023 have a lot of information
206.
by imjonse 2y ago
True. But an LLM product is closely associated with a single company and unlike a search engine which can claim it only shows you what is already available, the LLM will seem like it personally tells you something harmful. When they want to
207.
by imjonse 2y ago
It will refuse to describe the process of making napalm using only double entendres.
208.
by imjonse 2y ago
In the Llama 3 case I think the increase in parameters is mostly due to the input embeddings and output logits layers, reflecting the context size increase.
209.
by imjonse 2y ago
Would Pixar's existing software run on GPUs without much work?
210.
by imjonse 2y ago
Section 1.3 of the paper has lots of examples. https://dl.acm.org/doi/pdf/10.1145/3630106.3659005#page=6

More ›