fennecfoxy

3 ms·

1.
by fennecfoxy 11d ago
Well yes, there are only so many attention heads (well whatever magical variant frontier models are using these days) that can attend to the context and so as the context grows attention becomes spread thin. But with reasoning enabled I fin
2.
by fennecfoxy 11d ago
Interesting. I agree that it seems that LLMs are wont to fall into recognisable patterns much more readily than humans however I asked a model to evaluate your comment history and it found loads of patterns such as: “I’m not sure…” / “
3.
by fennecfoxy 11d ago
For sure. If you write with a casual conversational tone people don't get triggered. But if you are eloquent and use an extended vocabulary thanks to large amounts of reading then that seems to make people think you're using AI. A
4.
by fennecfoxy 18d ago
Ah right, thank you for clarifying. But I think my point still stands, isn't the geometry information THE information I referred to in the first place? Obviously the vector size gives you the granularity but it's kind of unavoidab
5.
by fennecfoxy 18d ago
Eh people don't care if something is true or false they care about the dopamine hit they get from engaging in tribal behaviours that reinforce their freaky belief systems that often aren't rooted in any kind of reality. Humans are
6.
by fennecfoxy 19d ago
I'm not as heavy on the maths stuff involved in this as other people commenting appear to be. But the idea makes sense, of course there is still recoverable data in embeddings, that's the point. Though as I constantly find the mor
7.
by fennecfoxy 19d ago
IANAL, especially not an American one. But if you're worried just pop it on anon GH. Besides the fact that this sort of protection through obfuscation is dead now anyway. If you can ask an LLM to do it so can I, or anybody else. The on
8.
by fennecfoxy 19d ago
Tbf "is it a crime" is hard for even a single lawyer to answer because it depends on: who you are, your skin colour, how rich you are, your sex, whether it's a white collar crime or not, did you commit the crime on behalf of
9.
by fennecfoxy 23d ago
Talking to members of the opposite sex only? How narrow-minded. Cyberpunk/sci-fi often explores futures where sex/gender is less of an issue (especially Ian. Banks' works). Clearly we're not there yet.
10.
by fennecfoxy 23d ago
So America thinks its above the ICC then? Explains a lot, lmao.
11.
by fennecfoxy 25d ago
The Internet has magnified the "my church leader said that gay people are actually demons" effect. Or "if you throw salt over your shoulder you're protected" or "wolves have alphas of the pack". People don
12.
by fennecfoxy 25d ago
I don't think that that issue is inherent to women. I think it's inherent to everyone and we get affected on different levels in different contexts. Any man who's been told to "man up" (most) has experienced the sam
13.
by fennecfoxy 25d ago
Given what I have seen in recent Hollywood "writing" I would not be surprised if when correctly prompted an LLM can outdo a lot of the disposable, predictable writing in modern shows. Every story boils down to the same predictable
14.
by fennecfoxy 25d ago
I mean a lot of it comes down to: "Give me a poster for my band playing a gig" Versus "Give me a poster for my band playing a gig, it should have x y and z. Use a x' artistic style and include elements of y'. The la
15.
by fennecfoxy 1mo ago
Oh for sure, but I think generally multiple experts are selected in an MoE pass for a token, so presumably it'd select programming related ones as well as general knowledge/language. Only problem would be the routing layer works o
16.
by fennecfoxy 1mo ago
I still personally think that a heavy lean into MoE will be better for that sort of thing. Our brains are subdivided into large parts but I'm sure (and I'm not a brain scientist) that those parts can be subdivided even further int
17.
by fennecfoxy 1mo ago
Fast models is why I was hoping Taalas would get their butts into gear and eventually release a consumer priced card. I'd love to have a pcie card that screams along at 15k t/s even if on a heavily quantized 2026 level model forev
18.
by fennecfoxy 2mo ago
I have found this to some degree (even though I work with amazing people day to day). It's much easier for people to pump out absolute garbage, you know the kind of "just get it done fast" slop that management types cry out f
19.
by fennecfoxy 2mo ago
It's fine. Even re-reading what I'd said it does seem a little extreme, but I still believe the core idea of it. I'm just watching violent criminals in UK get early release while the hapless government and public cannot rub t
20.
by fennecfoxy 2mo ago
Yeah, I always thought the future of this stuff would be hot-pluggable MoE modules or LORAs that are able to be downloaded and applied/used at will like how skills have become a thing. Like, atm most architectures seem limited by a sin
21.
by fennecfoxy 2mo ago
When I lived in Auckland (now in the UK) I used to go to gigs at Whammy bar in St. Kevins arcade, the dive bar part is awesome and I think they stop people doing it now but when I was in uni the walls used to be covered by graffiti and scri
22.
by fennecfoxy 2mo ago
Yes, also because I imagine people generally followed the dog damned rules there.
23.
by fennecfoxy 2mo ago
Yep, as a Kiwi living in UK I actually went back to NZ to redo my visa about December 2019, then that was all sorted and I returned about late Jan-early Feb 2020. I had a stopover in Bangkok and _everyone_ was wearing masks. Not just Asian
24.
by fennecfoxy 2mo ago
No, I'm saying it's only human to retroactively want to do extreme fantastical violence against people who have physically harmed you when you live in a society that a: does not care that you were hurt because you're a man an
25.
by fennecfoxy 2mo ago
I would also like someone to clarify this for me. I (think) I understand what people mean by a neurosymbolic model however common definitions are a bit funky. If it's neuro (llm/transformer similar) symbolic (symbolic with hand cr
26.
by fennecfoxy 2mo ago
I'm curious about this too as I've been working with a 1300 page document on procedures for [industry]. Where every single section is primarily about [industry] with minor differences in verbs actions etc for the procedures. I&#x2
27.
by fennecfoxy 2mo ago
It is nice to have a model that can "do it all", though. And surely that's still the end goal? Like how MoE is still somewhat popular in certain areas even after its heyday. I am wondering if models will end up being some sor
28.
by fennecfoxy 2mo ago
By post-train I presume you mean a finetune? Unless that's wrong (please correct me if so). I haven't looked into model architecture people are working with for this stuff too deeply yet but I presume the core idea is fine-tuning
29.
by fennecfoxy 2mo ago
Thank you for the link! Super interesting and I appreciate how it was written. It seems like a lot of the problems I have been running into with RAG on large/complex documents with generally low contrast in the information is not one t
30.
by fennecfoxy 2mo ago
Hmmmm, this is a problem I have been facing recently. Basic embeddings give decent-ish results (in the top say 20 chunks). Basic agentic retrieval gives slightly better results so long as the agent part of it doesn't go down the wrong

More ›