born-jre

9 ms·

61.
Exponentially faster language modelling(arxiv.org)
301 points by born-jre 3y ago|137 comments
62.
by born-jre 3y ago
when 100+B model hallucinates, that's a problem but a mistral 7b (qk_4 around 4gb file ) does this have prams to encode information to be hallucination proof, since llm cannot know what they do not know. So maybe we should be building
63.
by born-jre 3y ago
https://wasm4.org/
64.
by born-jre 3y ago
nice, but i had a separate idea. what if u build a wasm runtime that can save and restore memory with and execution states, sounds much more full proof. or i might have misunderstood this idea :D.
65.
by born-jre 3y ago
if someone reputable could maintain a blind benchmark that is not public, that would be great.
66.
by born-jre 3y ago
34B is grouped query attention, right? Does that make it the smallest model with grouped attention? I can see some people fine-tuning it again for general propose instruct.
67.
by born-jre 3y ago
Saw in twitter few days ago, "inventor" had this idea when he visted a zoo and elephant was walking in circle before giving birth and zoo attendant wrong said that elephants do that to help with the childbirth.
68.
by born-jre 3y ago
It's funny when different incarnation of thing people wants happens mainstream functional programming => expected (Ocamal/haskel) => got js libary with functionalness (REACT) Immediate mode GUI => expected (imgui) =>
69.
by born-jre 3y ago
Love the hustle. :D
70.
by born-jre 3y ago
i know, its not about complexity of format. somebody is going run into problem where they have to parse modelfile and parser is not written in their language vs it could be done with yaml/json/toml. Its just me i guess. ziglang is
71.
by born-jre 3y ago
Modelfile cool. Just use JSON or YAML, not custom format please! edit: Adapter support would be really cool. Multiple adapter even better i want somebody to make MoE of adapters.
72.
Room Temp supercondictivity or bust, Experiment WIP(twitter.com)
6 points by born-jre 3y ago|3 comments
73.
by born-jre 3y ago
I have read about Collapse OS (I think it is because it was about a Forth-based end-of-world OS) a while back, and it has made a lasting impression on me. This might be the reason why I enjoy easy bootstrappable/self-contained toolchai
74.
OpenOrca-Preview1(huggingface.co)
2 points by born-jre 3y ago|0 comments
75.
by born-jre 3y ago
if you want to see code you can try kaggle. i think François Chollet talks about this in lex podcast. https://www.kaggle.com/c/abstraction-and-reasoning-challenge https://www.youtube.com/watch?v=Bo8MY4J
76.
by born-jre 3y ago
As a Level 3 plastic hater, it's a wonderful curse. Whenever I see food and plastic together, I automatically think, "Has it been heated until it reached this point?" My mind goes, "Microplastic, microplastic!"
77.
I-JEPA, the Image-Based Joint-Embedding Predictive Architecture (FB)(github.com)
3 points by born-jre 3y ago|0 comments
78.
by born-jre 3y ago
short answer is they have somewhat competent hardware but software sucks or you can watch george hotz rant about how amd driver sucks https://www.youtube.com/watch?v=Mr0rWJhv9jU
79.
Temphia: Platform for Apps(github.com)
3 points by born-jre 3y ago|0 comments
80.
by born-jre 3y ago
that was good one. i wanted to send this to a couple of people of then I realize the meta irony of that. edit: donot know why this reminds of song "Ooh La La - the Face" > Poor young grandson, there's nothing I can say You
81.
Temphia: Platform for Apps(github.com)
3 points by born-jre 3y ago|0 comments
82.
by born-jre 3y ago
sounds cool, i wonder if its possible to do similar with "The Forward-Forward Algorithm" by Geoff Hilton maybe on a few add on layers like with lora [0]: https://arxiv.org/pdf/2212.13345.pdf
83.
by born-jre 3y ago
Split brain patients, maybe? Edit: not agreeing or disagreeing, but that would be the most probable proof or sth
84.
by born-jre 3y ago
at first glance i thought may be its like tinygrad. but looks has many ops than tiny grad but most maps to underlying hardware provided ops? i wonder how well tinygrad's apporach will work out, ops fusion sounds easy, just walk a graph
85.
WebNN: Web Neural Network API(w3.org)
5 points by born-jre 3y ago|1 comments
86.
by born-jre 3y ago
There is so much parallel progress happening left and right at the same time they are not there yet. When things like sparseGPT and models fine-tuned with data with tool ability (not just instruct data) may be soon we get there, as long as
87.
by born-jre 3y ago
Yeah, i hate and admire WordPress at the same, in my brief exposer I quite like its hooks/action/plugin system. In a weird way my current project[0] is inspired by wordpress. (its not a exact CMS thing but), what if someone build
88.
by born-jre 3y ago
lol, i did not know fuzzylogic was seperate field, thats what i call ml/ai . I also named one of my repo fuzzylogic [0]. [0]: https://github.com/temphia/fuzzylogic
89.
by born-jre 3y ago
don't see what platform is supported ? Does it have/will have Linux client?
90.
by born-jre 3y ago
thats why we need wasi/wasm bashed sandboxed plugins scoped bashed on capability it needs. i think lapce supports wasi plugin but overall ux is not there yet when i last tried. https://github.com/lapce/lapce

More ›