cs702

67 ms·

661.
by cs702 2y ago
> Notably, humans also underperform Transformers on recall tasks... Thank you. Yes, that's obviously true... but keep in mind that human beings can pull up information from notes, books, URLs, etc. and store that information in &quo
662.
by cs702 2y ago
Hahahaha. You forgot to mention that when Nvidia's market cap reaches infinity and beyond, by then we would surely be an intergalactic robotic species. Just sayin'. Don't be shy with your forecasts! ;-)
663.
by cs702 2y ago
> AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years. Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year), algorithmic efficiencies (~0.5 OOM
664.
by cs702 2y ago
I've only briefly skimmed the paper, so I can't comment on its information-theoretic aspects yet, but I found the example given on Section 3 very helpful and insightful: The key insight is that if you give a pretrained LLM a quest
665.
by cs702 2y ago
Thank you for sharing this. I've added it to my reading list!
666.
by cs702 2y ago
TL;DR: The authors show that if you simplify Mamba so its state-space layer uses a diagonal matrix A that is a scalar times the identity matrix, the state-space transformation can be expressed as a form of causal linear attention.[a] That&#
667.
by cs702 2y ago
Dao and Gu show that if you simplify Mamba so its state-space layer uses a diagonal matrix A_t that is a scalar times the identity matrix, i.e., A_t = a_t I, the state-space transformation can be expressed as a form of causal linear attenti
668.
by cs702 2y ago
Fantastic work. Deserves to be widely used to teach Maxwell's equations. THANK YOU.
669.
by cs702 2y ago
A powerful example of how well-intentioned regulations -- in this case, a mandate to reduce sulfur content in ship fuel, for environmental reasons -- can lead to unintended consequences that are worse than the original problem -- in this
670.
by cs702 2y ago
Cool and interesting. The authors propose a hybrid digital-analog training loop that takes into account the curvature of the loss landscape (i.e., it uses second-order derivatives), and show with numerical simulations that if their method i
671.
by cs702 2y ago
The whole point is to leverage the laws of nature to train AI models, overcoming the limitations and scaling challenges of digital hardware and existing training methods.
672.
by cs702 2y ago
Apparently this is real. Surely, every other "dear leader" out there will want one of these. They'll say "my loving subjects can now have me ruling forever" or something similar. It's both unbelievable and enti
673.
by cs702 2y ago
Before we can answer that question, don't we need a method for determining whether an entity is conscious? As far as I know, such a method doesn't exist yet. We have no way of verifying if other entities are conscious. I'd se
674.
by cs702 2y ago
Not really. But questions about the work should be asked to the authors. I've only skimmed the paper.
675.
by cs702 2y ago
That sounds rather... serious. Is there anyone here with relevant knowledge who can comment?
676.
Russian Courts Seize €463M from Italian Bank(reuters.com)
3 points by cs702 2y ago|1 comments
677.
by cs702 2y ago
I have nothing against them. Please note, I edited my comment to change "dual axis charts" (common spreadsheet terminology) to "plots with two different scales in the same direction," which more accurately describes the
678.
by cs702 2y ago
I'm 100% in agreement. To anyone here who thinks plots with two different scales in the same direction sometimes are appropriate: Please read this. --- EDIT: Changed "dual axis charts" to "plots with two different scales
679.
by cs702 2y ago
Code: https://github.com/OswaldHe/HMT-pytorch This looks really interesting. I've added the paper to my reading list and look forward to playing with the code. I'm curious to see what kinds of improvements we
680.
by cs702 2y ago
> Large social media platforms appear to be increasingly resistant to third-party tools that give users more command over their experiences. Many of them, in fact, have stopped supporting third-party tools directly. The interests of th
681.
by cs702 2y ago
The hallucinations are a consequence of sampling from a probability distribution over all possible tokens at each step. There are a lot of very smart people trying to figure out how to sample for generative purposes while "grounding&
682.
by cs702 2y ago
Please. No one is ignoring this problem. In fact, a tremendous amount of money is being invested right now to reduce/control hallucinations. It's an active area of research. Everyone with money is trying to solve the problem!
683.
by cs702 2y ago
> Depends on what space you are talking about. The companies constantly building and training new models to keep up with the competition.
684.
by cs702 2y ago
I think the benefits to society are going to be significant, but I suspect that most companies in the space will not recoup their investment. They're losing money or at best breaking even on each token. It's as if the return on
685.
by cs702 2y ago
As I wrote only yesterday, "the usual critics will point out that LLMs like GPT-4o still have a lot of failure modes and suffer from issues that remain unresolved. They will point out that we're reaping diminishing returns from Tr
686.
by cs702 2y ago
Products that are fungible compete on price (what else?). Chat-with-AI services that have similar performance are pretty fungible today. Switching from one to the other is... remarkably easy. The moment Gemini Flash's competitors start
687.
by cs702 2y ago
I never said it was.
688.
by cs702 2y ago
We're witnessing a race to the bottom on pricing as it's happening. Competition based solely or mainly on pricing is a defining characteristic of a commodity market, i.e., a market in which competing products are interchangeable,
689.
Project Astra(theverge.com)
104 points by cs702 2y ago|42 comments
690.
by cs702 2y ago
Starting at $16K. Make sure to watch the video. Given how much capital is being invested right now, and how quickly things are improving, it's hard not to imagine cheap robots taking on many jobs that until now have been done by human

More ›