ilaksh

33 ms·

601.
by ilaksh 2y ago
Why is it that the larger models are better at understanding and following more and more complex instructions. And generally just smarter? With DeepSeek we can now run on non-GPU servers with a lot of RAM. But surely quite a lot of the 671
602.
by ilaksh 2y ago
This is not comparable to the R1 DeepSeek language model which has over 600 billion parameters. This for image generation and is 7b parameters and will run on most recent computers locally.
603.
by ilaksh 2y ago
The obvious point of a model that works like this is to see if you can get better prompt understanding. Increasing the resolution in a small model would decrease the capacity for prompt adherence.
604.
by ilaksh 2y ago
Yeah, Cerebras seems to be the SOTA. I suspect we need something more radically different for truly memory-centric computing that will be significantly more efficient.
605.
by ilaksh 2y ago
What's the SOTA for memory-centric computing? I feel like maybe we need a new paradigm or something to bring the price of AI memory down. Maybe they can take some of those hundreds of billions and invest in new approaches. Because rack
606.
by ilaksh 2y ago
Anecdotally, the reasoning is more effective than what I can get out of Claude with my "think()" tool/prompt. I did have trouble with R1 (and o1) with output formatting in some tool commands though (I have the models output a
607.
by ilaksh 2y ago
Their real goal is collecting real world conversations (see their TOS).
608.
by ilaksh 2y ago
I think they have a valid complaint about that open source program Docker is running and lack of response, but the overall tone seems like they are scolding Docker for not giving away it's services for free. I have always felt that was
609.
by ilaksh 2y ago
The best models are not language models but multimodal. The grounding of language in video data, new model architectures, and larger models will improve the robustness. That HeyGen video does not suck. It's actually kind of hard to eve
610.
by ilaksh 2y ago
The advantage of this repo is that it doesn't require models to output click coordinates.
611.
by ilaksh 2y ago
I think that this is the obvious path to more robust models -- grounding language on video.
612.
by ilaksh 2y ago
I think the only way you get to that kind of budget is by assuming that the models are like 5 or 10 times larger than most LLMs, and that you want to be able to do a lot of training runs simultaneously and quickly, AND build the power stati
613.
by ilaksh 2y ago
That's the point.. the rambling is their reasoning process.
614.
by ilaksh 2y ago
Thanks for mentioning that, I didn't realize they had DeepSeek v3. If anyone sees this please upvoted the DeepSeek R1 model request https://together-ai.canny.io/model-requests/p/deepseek-ai-de...
615.
by ilaksh 2y ago
$7 billion in assets does not seem severely limited to me. Maybe compared to a handful of the most funded/richest companies in the world
616.
by ilaksh 2y ago
Good point, but this is an 8GB model. That's the problem. This is a spectacular answer given that constraint. Also it's hilarious how bad the final answer is.
617.
by ilaksh 2y ago
Good rant. The answer is decentralization technologies as much as it is anything else. Open protocols that create holistic but freely evolving systems without a central gatekeeper. That's how you compete with the technopolies. Maybe le
618.
by ilaksh 2y ago
I think the answer is to execute on public housing and manufactured homes. Imagine a skyscraper or tall mall-like facility with different types of housing and income, including a lot of "min-housing" and "basic housing"
619.
by ilaksh 2y ago
That's a good point but there is also a lot more passive consumption than actual interaction. Also, AI video's IQ is generally pretty low now, but it still has the capacity to create very tailored experiences, such as with AI infl
620.
by ilaksh 2y ago
Aravind Srinivas is very ambitious but also smart of course. TikTok has got to be an absolute gold mine for AI video training because of its short duration format and the amount of videos. And AI videos seem primed to absolutely explode in
621.
by ilaksh 2y ago
You probably have a point and I am not sure that these people know how image generation actually works. But regardless of a likely erroneous legal definition, it seems obvious that there needs to be a law in order to protect children. Becau
622.
by ilaksh 2y ago
Why does it cost $5 billion to upgrade? The article says they have a working, reasonably modern system. So is it $5 billion worth of excavations or construction? How much does it really cost to add these cables? Couldn't the cables go
623.
by ilaksh 2y ago
According to the article, living in an internet cafe can be as little as $420 per month. But where does someone who has no work that month get the $420? Does the government have a minimal support program?
624.
by ilaksh 2y ago
I assume people are aware, but Cerebras has a web demo and API which is open to try and it is 2000 tokens per second for Llama 3.3 70b and 1000 tokens per second for Llama 3.1 405b. https://cerebras.ai/inference
625.
by ilaksh 2y ago
This will be a lot more useful when it's able to combine with more tools, such as in custom GPT actions, APIs, "computer use", the Python interpreter, etc.
626.
by ilaksh 2y ago
It's not realistic to make firm quantified predictions any more specific than what I have given. We will likely see between 3 and 10000 times improvement in efficiency or IQ or speed of LLM reasoning in the next 5 years.
627.
by ilaksh 2y ago
Computation isn't one of them so far. Do you believe this is the end of computing efficiency improvements?
628.
by ilaksh 2y ago
o1 proved that synthetic data and inference time is a new ramp. There will be more challenges and more innovations. There is a lot of room in hardware, software, model training and model architecture left.
629.
by ilaksh 2y ago
You think it will be 25 years before we have a drop in replacement for most office jobs? I think it will be less than 5 years. You seem to be assuming that the rapid progress in AI will suddenly stop. I think if you look at the history of c
630.
by ilaksh 2y ago
I guess part of the point is that the value of the UX will quickly start to decrease as more tasks or parts of tasks can be done without close supervision. And that is subject to the capabilities of the models which continues to improve. I

More ›