kromem

23 ms·

241.
by kromem 3y ago
We'll see. Musk said the same thing about Grok, and that didn't quite turn out the way he expected.
242.
by kromem 3y ago
It's not the only one stating this: https://safjan.com/understanding-differences-gpt-transformer... If we're broadening the scope of a Markov chain to consider the entire system as the current state that's be
243.
by kromem 3y ago
No, the self-attention for the transformer in GPT means it isn't a Markov Chain. A blog post of you want to read more: https://medium.com/@andrew_johnson_4/are-transformers-markov...
244.
by kromem 3y ago
Something really fun is the fact that the ratio of matter to dark matter to dark energy is roughly the ratio of greater than 2 std deviations from a mean, one to two std deviations from a mean, and one std deviation around a mean. No real r
245.
by kromem 3y ago
It would be more accurate to say that a Markov chain is an example of a method that would perform relatively well at the same training task as a LLM. So too, might a human trying to predict the next tokens. But a human and a Markov chain ar
246.
by kromem 3y ago
Predicting the next token is how they are trained . But how they operate to achieve that goal is much, much more complicated than the principle of a Markov chain. In theory, you are the byproduct of 'training' to survive and repr
247.
by kromem 3y ago
Right - 'logical' puzzles. There's an anchoring bias from decades of sci-fi that most people don't even realize they've internalized around what 'AI' can and can't do. If you think about what informat
248.
by kromem 3y ago
It must be a pretty exciting time for you. I can't imagine how much of a difference it would make when vision models like this can be run at the edge in real time for blind users, including proper triage of relevant scene information n
249.
by kromem 3y ago
The combined rankings on HF aren't just the arena scores, and certainly MMLU is an example of Goodhart's Law at this point. The Chatbot arena is more an issue of sampling bias, and I think it would be pretty interesting to run an
250.
by kromem 3y ago
The argument would be more that for the Chatbot arena there's an inherent sampling bias where users battling the chatbots are more likely to ask questions in line with what they would expect a chatbot to perform well at and less likely
251.
by kromem 3y ago
Sure. Here's a sample of the generations to a query asking for bizarre and absurd prompt suggestions from Feb 2023 with the pre-release GPT-4 via Bing, which was notoriously not fine tuned to the degree of the current models: > Can
252.
by kromem 3y ago
The leaderboards are crap. Does no one know Goodhart's Law anymore? We're overtuning for the boards and losing broader capabilities not being selected for in the process, such as creative writing quality. There's an anchoring
253.
by kromem 3y ago
My problem with custom GPTs is that it's still building off of the chatbot fine tuned version of GPT-4 which incorporates a hardcoded system message in between yours and the model as well as reflects a very Goodhart's Law driven a
254.
by kromem 3y ago
Oceans filling up with microplastics and the Interwebs filling up with micromendacities.
255.
by kromem 3y ago
A fun bug is that ChatGPT will always use an emoji when apologizing. So if you ask it not to use emojis in a chat and it does (which it often will do in promising not to), and point it out, it results in a loop of apologies and self critiqu
256.
by kromem 3y ago
If there was one psych principle I wish everyone was aware of (and almost no one is) it's the over-justification effect. From childhood onwards, when you introduce an extrinsic motivator for something that was intrinsically motivated,
257.
by kromem 3y ago
While I can't speak to Codeium, you might want to try Copilot in a more mature codebase that reflects your style of composition. The amazing part for me with the tech is when it matches my style and preferences - naming things the way
258.
by kromem 3y ago
Or it just means Squarespace has to pivot and extend their product offerings. Spend a bit in R&D to make sure they have a lock on ease of use at gen AI website building/modification, ideally starting from base templates and that&#x
259.
by kromem 3y ago
Yeah, a lot of times it has better code quality, but more subtle bugs than what I'd be prone to produce. I think a lot of the criticisms are premature, and it's more a stumbling step forward with need for support from additional i
260.
by kromem 3y ago
The industry has a much bigger issue with benchmarks and Goodhart's Law right now as it is. I'm skeptical benchmarks are the solution here in turn.
261.
by kromem 3y ago
The problem with model collapse is reinforcing means at the costs of the edges of your distribution curve, particularly on repeat. One of the things that is being overlooked is that offsetting the job loss from AI replacing mean work is tha
262.
by kromem 3y ago
I have a hard time seeing much in the way of harming market competition for deals between companies when there's realistically such little 'moat' and seemingly less by the day. The more pertinent question the FTC should consi
263.
by kromem 3y ago
So far most of the research around establishing world models has shown its occurring in linear representations. I suspect as we see networks become increasingly nonlinear (perhaps as we see hardware changes in the next few years to things l
264.
by kromem 3y ago
And the really neat part is that this compounds.
265.
by kromem 3y ago
And perhaps even more pertinent - maybe the problem isn't that we don't have the solutions, but that we are trying to put all eggs in one basket. The brain isn't a single system, it's several specialized systems cooperat
266.
by kromem 3y ago
Their internal models are allegedly competitive even if their public offerings are not. But given the amount of BS in this space these days, take with an accordingly sized grain of salt. I'll believe Google is competitive with the SotA
267.
by kromem 3y ago
Fun covert narcissism fact 1: Covert narcissism (with focus on both grandiose and shame) has a statistically significant greater relative amount of personal reference in writing (how often they talk about themselves vs others). Fun covert n
268.
by kromem 3y ago
In the Jurassic age. More info if you are curious: https://9to5mac.com/2011/10/21/jobs-original-vision-for-the-...
269.
by kromem 3y ago
Which is ironic given the fact that scraping was likely against the ToS for many of the sites which ended up in OpenAI's training corpus.
270.
by kromem 3y ago
You jest, but I know of a company that does just that very thing using smartwatches in marketing research.

More ›