visarga

52 ms·

961.
by visarga 2y ago
> There are millions of examples of usage of the word red - none of them communicate the subjective experience of seeing red. The trick is that all of them provide perspectives and the model composes those perspectives in the embedding s
962.
by visarga 2y ago
If AI turns out capable of automating human jobs then it will also be a capable assistant to help (jobless) people manage their needs. I am thinking personal automation, or combining human with AI to solve self reliance. You lose jobs but g
963.
by visarga 2y ago
Of course there is. There are millions of examples of usage for the word "red", enough to model its relational semantics. Relational representations don't need external reference systems. LLMs represent words in context of ot
964.
by visarga 2y ago
When we get to that point we will have the right methods to sculpt our bodies as we like. It's a matter of hormones or some other biological hack.
965.
by visarga 2y ago
> training on copyrighted data without a similar licensing agreement is also a type of market harm, because it deprives the copyright holder of a source of revenue I would respond to this by 1. authors don't actually get revenue fro
966.
by visarga 2y ago
You can prompt the model to do that kind of "stream of mind" process. It will maximize modeling uncertainty. This is my prompt: > Write in a raw, real-time stream-of-consciousness style, as if actively solving a problem. Your r
967.
by visarga 2y ago
syntax becomes semantics after sufficient iterative updates in the environment
968.
by visarga 2y ago
This is a good counterexample to "syntax is not sufficient for semantics". It is sufficient when there is no distinction between data and code. Code can reflect on itself as data. Like bootstrapped autocompilers, Godel's Arit
969.
by visarga 2y ago
Even if it's made with AI, it is slop only if you don't add anything original in your prompt, and don't spend much time selecting. The real competition of any new work is the backlog of decades of content that is instantly ac
970.
by visarga 2y ago
When I tested o1 a few hours ago, it seemed like it was losing context. After I asked it to use a specific writing style, and pasting a large reference text, it forgot my demand. I reminded it, and it kept the rule for a few more messages,
971.
by visarga 2y ago
I independently "invented" this 2 months ago, also for parsing 100 page long PDFs. Give each page a title and summary, but prepend as context the title+summary of previous 3 pages. In the end combine all titles+summaries to get a
972.
by visarga 2y ago
use the json mode in ollama
973.
by visarga 2y ago
It feels LLM generated, maybe it's just my imagination.
974.
by visarga 2y ago
A few days ago the QwQ-32B model was released, it uses the same kind of reasoning style. So I took one sample and reverse engineered the prompt with Sonnet 3.5. Now I can just paste this prompt into any LLM. It's all about expressing d
975.
by visarga 2y ago
I rarely ever use o1-preview, almost all usage goes to 4o nowadays. I don't see the point in a model without web search, it's closed off. And the wait time is not worth the result unless you're doing math or code.
976.
by visarga 2y ago
All training data? Even public domain and open source?
977.
by visarga 2y ago
If it is trained on this forum it would learn even worse. We project a lot of doom imagination on AI here.
978.
by visarga 2y ago
> it will circumvent things you do to it that would stop it from completing it's task. I thought you said a supercapable agent not one with long term blindsight. How can a model make its own chips and energy? It needs advanced proc
979.
by visarga 2y ago
No, better to train with all that crap and all the debate around it or you get a stunted model. You think you can find all references that could possibly give this idea to the model, or contexts model could infer it from? Like, how many tim
980.
by visarga 2y ago
Actually, all you need to do is to apply structured randomness to get diversity from a LLM. For example in TinyStories paper, a precursor of the Phi models: > We collected a vocabulary consisting of about 1500 basic words, which try to m
981.
by visarga 2y ago
Generally speaking text only models manage to learn a huge amount about the visual world. So when you put the model train on video it might have less to learn. Video is also less abstract than text, generally. But I am sure we can still ext
982.
by visarga 2y ago
The fact that you can prune a model will not make it smarter, the wall still stands. I think what explains the wall is the fact that we can't scale organic data exponentially, and we have already covered the most useful types. Going fo
983.
by visarga 2y ago
Fully agree, self replication is key. But we can't automate GPU production yet. Current GPU manufacturing is probably one of the most complex human endeavors we've ever created. You need incredibly precise photolithography, ultra-
984.
by visarga 2y ago
> they're in essence, the last company to ever exist, and are building the last product we'll ever need Physical reality is the ultimate rate-limiter. You can train on all of humanity's past experiences, but you can't
985.
by visarga 2y ago
It is just setting the problem space. It doesn't convey anything but a topic.
986.
by visarga 2y ago
You can achieve the same effect by using LLM to do question answering prior to embedding, it's much more flexible but slower, you can use CoT, or even graph rag. Late chunking is a faster implicit alternative.
987.
by visarga 2y ago
I'd love to see a GraphRAG browser that collects the pages I visit automatically.
988.
by visarga 2y ago
You still have to pay for the memory. The Cerebras chip is fast because they use 700x more SRAM than, say, A100 GPUs. Loading the whole model in SRAM every time you compute one token is the expensive bit.
989.
by visarga 2y ago
It's actually a great topic - both humans and LLMs are black boxes. And both rely on patterns and abstractions that are leaky. And in the end it's a matter of trust, like going to the doctor. But we have had extensive experience w
990.
by visarga 2y ago
OCR system use vision models and as such they can make mistakes. They don't sample but they produce a distribution of probability over words like LLMs.

More ›