abernard1

9 ms·

61.
by abernard1 2y ago
> Machine learning is about filling in the gaps based off of a prediction. I think this is a generous interpretation of network-based ML. ML was designed to solve problems. We had lots of data, and we knew large amounts of data could d
62.
by abernard1 2y ago
This is an interesting one because math is doing so much of the heavy lifting. And symbolic math has a far smaller representational space than numerical math. There is one other wonderful thing about symbolic math, the glorious '=&#x2
63.
by abernard1 2y ago
I'll give an argument against this with the caveat it applies only if these are pure LLMs without heuristics or helper models (I do not believe that to be the case with o1). The problem with auditing is not only are the outputs incorre
64.
by abernard1 2y ago
The comparison is flawed though in that humans and LLMs make mistakes for different reasons. Humans forget things. Humans make errors. Humans' train of thought isn't impacted by an errant next token in the statement they're
65.
by abernard1 2y ago
The problem with this line of argumentation is it implies that autoregressive LLMs only hallucinate based upon linguistic fidelity and the quality of the training set. This is not accurate. LLMs will always "hallucinate" because
66.
by abernard1 2y ago
> - Partner countries like Israel (however you feel about that) will eventually use whatever we make. America will not be sole user. Regarding second-order effects, I take great comfort in that. Once upon a time people were able to disc
67.
by abernard1 2y ago
Since people are not reading the paper, here were the conclusions: > Is it really easy to answer semantic reasoning questions over symbolic graphics programs? We provide an intriguing experiment to demonstrate that SVG programs can be qu
68.
by abernard1 2y ago
> I'm no economist, but inflation should reflect an increse in growth and productivity. An increase in growth and productivity should reflect deflation : more goods and services with the same amount of currency. Absent interventio
69.
by abernard1 2y ago
It has nothing to do with "hard money" if you mean physical currency. People want wealth, period. The problem with the idea of inflation (read: "scarcity") to drive monetary velocity is it messes with investment. There
70.
by abernard1 2y ago
> The ideal currency has a constant, predictable 2-4% inflation to promote the velocity of money and prevent stagnation. This fundamentally misunderstands the point of Bitcoin's creation (and currency). The ideal currency does not
71.
by abernard1 2y ago
> “it only predicts the next token” doesn’t (in principle/theory) preclude it having such an understanding of concepts, unless no computational process ever can. In my opinion, this is highly reductive and academic. Whether these m
72.
by abernard1 2y ago
This does not indicate abstract reasoning. I said: > Where they do logical tasks, that can be done with minimal or no understanding, because syllogisms and logical formalisms are highly structured in text arguments. There is an enormous
73.
by abernard1 2y ago
> You're limiting your view of their capabilities on the output format. The "generation" of strings is related to this output format. It's critical to how they work. Legerdemain has been performed to argue that that
74.
by abernard1 2y ago
> Any probability distribution over strings can theoretically be factored into a product of such a “probability that next token is x given that the text so far is y”. And such a probability distribution would not generally understand con
75.
by abernard1 2y ago
> Could a creature that simply evolved to survive and reproduce possibly have a conceptual model underpinning it? Yes. Obviously. I can create plans and think on them. I can think without the need for an internal monologue or talking
76.
by abernard1 2y ago
I agree that's an argument. I would contend that argument is obviously false. If it were true, LLMs could multiply scalar numbers together trivially. It should be the easiest thing in the world for them. The network required to do
77.
by abernard1 2y ago
> LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text. I would
78.
by abernard1 2y ago
> In terms of gdp per capita Soviet russia was 33rd in the world in that year. Modern Russia is 52nd. GDP was also calculated based upon prices in a free market. The problem being, if you don't have a free market, you don't ha
79.
by abernard1 2y ago
> What would it take for the US to lift sanctions (I assume a radical shift within Cuba's government), and for Cuba itself (as a whole) to restructure their government in a way that would benefit them and everyone. It's irrelev
80.
by abernard1 2y ago
Yeah. While I'm sympathetic to the attempt to formalize the representation, there's a psychology to this that a lot of people in tech seem to miss when I see them dancing. The reasoning goes: "if I understand and represent t
81.
by abernard1 2y ago
I enjoy the attempt to analyze the dancing (really choreography). But I'm surprised that in this new world of LLM hype not one word in there was dedicated to what dance really is: a language. Dance is a physical, kinesthetic language.
82.
by abernard1 3y ago
These news sources don't actually care. If you've followed SF news for any length of time, what you'll realize is how extremely thin-skinned they are about any criticism. "Doom loop!? Stop making it worse." As if b
83.
by abernard1 3y ago
I find this a positive development. The people that moved to Austin from California are conformists. The disappointment they experience is what happens when they are not surrounded by the concentrated capital that makes their (relative bor
84.
by abernard1 3y ago
> The rest of your comment has no relevance to the topic at hand. My comment, much like credibility of "expert" testimony in a trial, has to do with large scale failures of so-called expert governance in California. I'm su
85.
by abernard1 3y ago
Except of course, it's not negative to the tune of $21B. Any more than California diverting half its rainwater to the ocean was either environmentally or economically useful to its ecosystem. This is the same state where a good 70% of
86.
by abernard1 3y ago
> They’re the fifth largest global economy Were there perhaps reasons why California was (at least for now) the fifth largest global economy? And could that be partially related to this particular story? I for one encourage California t
87.
by abernard1 3y ago
I would just like to point out that in the space of two decades we've whipsawed between "ulcers aren't caused by stress, they're caused by bacteria" to "no actually ulcers might really be mainly caused by stres
88.
by abernard1 4y ago
> know exactly what these models aren't going to be able to do is pretty foolish at this point This can be said about absolutely any new technology but it does not make it true. It's simply the inscrutability of these tensors
89.
by abernard1 4y ago
> The capabilities of these models are very likely to be radically different five years from now. This is an article of faith. Specifically, it believes that at some point in the future the current paradigm will result in qualitatively
90.
by abernard1 4y ago
Your understanding that problem solving has to do with "steps" is the correct one in my opinion. Also, with data driven approaches, the model isn't necessarily learned in any meaningful way. If you train with certain inputs

More ›