samus

34 ms·

691.
by samus 3y ago
If you abstract far enough then yes, everything what we are doing is somehow akin to what we have done before. But that then also applies to what Einstein has done.
692.
by samus 3y ago
In not sure I understood your question. This model should have much lower computational cost since only one out of eight layers is a traditional transformer layer with masked self-attention. Additionally, half of the Mamba layers are MoEs.
693.
by samus 3y ago
This one should have you covered :-) one out of every eight layers is a traditional Transformer layer, which should ensure precision, at least over short distances.
694.
by samus 3y ago
A lot of the problems of Stable Diffusion arise because of using a text encoder that isn't up to the task to encode the meaning sufficiently precise. For example, if you mention the word "green" in the prompt, it often carrie
695.
by samus 3y ago
I guess the version with donor cells is fine if one is already taking immunosuppressants because of some other reason.
696.
by samus 3y ago
Even if they are of the same variety (California Cabernet Sauvignon), there is a vast difference in flavor profiles and it would be surprising if the ranking would not come down to personal preference. It would be more fair to compare win
697.
by samus 3y ago
Pyenv just worked for me. I am actually using Fedora Silverblue and have GCC and the CUDA SDK available only inside a toolbox container. Therefore, I have to enter that toolbox to install things like FlashAttention.
698.
by samus 3y ago
If it didn't fulfil a function, then random mutations and deletions during Meiosis would quite quickly get rid of it. Even if it only serves to support the structure of chromosomes or some similarly arcane purpose that we will only dis
699.
by samus 3y ago
Admittedly pumped hydro can't be scaled up that much. However, the few places that are suitable are very well appreciated and likely won't ever be dismantled like these dams. For grid-level batteries, absolute space efficiency is
700.
by samus 3y ago
The problem is that they have to be tested for 7B models at least to show promise for larger models. And that requires significant compute resources.
701.
by samus 3y ago
Grid-level batteries and pumped hydro (which only requires a height differential, not a riverbed) have been a thing for quite a while now.
702.
by samus 3y ago
It's a give and take. Historically, the danger from flooding actually increased because the floodplains next to the riverside were mostly eliminated. They would provide flexibility for the river to meander and to expand in times of hea
703.
by samus 3y ago
While you are not wrong, crypto is not what this is being compared with.
704.
by samus 3y ago
Backprop is just applied calculus. People simply didn't think about using it for neuronal networks yet.
705.
by samus 3y ago
It's still a monstrosity compared to a traditional parser. You can even be fancy and use complex parsers that backtrack and can deal with mildly context-sensitive languages (as required for HTML, XML, and many programmin languages), an
706.
by samus 3y ago
The example here has HTML with a somewhat fixed format. It would indeed have been better to have samples with different format and aiming for a low error rate. If you are scraping a limited amount of sites, you could for each site ask the L
707.
by samus 3y ago
The better way would be to ask it to generate a program that uses CSS selectors to parse the HTML.
708.
by samus 3y ago
You could employ an LLM to give you updated queries when the format changes. This is something where they should shine. And you get something that you can audit and exhaustively test.
709.
by samus 3y ago
This is called Retrieval Augmented Generation (RAG). The LLM driver recognizes a query, it gets send to a vector database or to an external system (could be another LLM...) and the answer is placed in the context. It's a common strateg
710.
by samus 3y ago
They definitely stand their ground if they were aligned to do so.
711.
by samus 3y ago
They simply don't work that way. You are asking it for an answer, it will give you one since all it can do is extrapolate from its training data. Good prompting and certain adjustment to the text generation parameters might help preven
712.
by samus 3y ago
Parsing HTML and tagsoup is IMHO not the right application for LLMs since these are ultimately structured formats. LLM are for NLP tasks, like extracting meaning out of unstructured and ambiguous text. The computational cost of an LLM chewi
713.
by samus 3y ago
Character-level operations are difficult for LLMs. Because of tokenization they don't really "perceive" strings as a list of characters. There are LLMs that ingest bytes, but they are intended to process binary data.
714.
by samus 3y ago
Changing a single pixel is usually not enough to confuse convolutional neuronal networks. Even so, human supervision will probably always be quite important.
715.
by samus 3y ago
They already have superhuman image classification performance.
716.
by samus 3y ago
LLM is just quite compute-intensive. You could send emails to the OpenAI API to classify them as SPAM or HAM, but you'd probably cry at the bill. That should explain why it doesn't exist in free-tier accounts. Worry not, even in t
717.
by samus 3y ago
I recently did a Kaggle experiment and finetuned a LLM to classify SMS as SPAM or HAM. It was not so difficult; following the text classification example in the Hugginface transformers docs was enough. Without trying hard at all, accuracy w
718.
by samus 3y ago
One could also disassemble an executable and build on top of it. Not for the faint of heart and probably illegal, but possible unless it was deliberately obfuscated. Compared to that, it is impossible with state-of-the-art methods to system
719.
by samus 3y ago
There are other ways to achieve that (optimizer features and decisions can be influenced by config settings, or by altering entries in statistics tables), but it's admittedly quite clunky. I agree that a production incident at 3am is d
720.
by samus 3y ago
Accessing such areas is sometimes done deliberately since programmers could rely on the OS telling them what just happened using signals instead of nuking the process wholesale. Doing it without signals is usually slow and/or clunky (n

More ›