m_ke

7 ms·

61.
by m_ke 8mo ago
you missed the boat on natural language being the new interface best orgs will own their data and have full history in version control so that it's easier for LLMs and humans to work with, not walled garden traps
62.
by m_ke 8mo ago
also i wasn't concerned about open chinese models till the latest iteration of agentic models. most open claw users have no idea how easy it is to add backdoors to these models and now they're getting free reign on your computer t
63.
by m_ke 8mo ago
you should try some markdown files in git
64.
by m_ke 8mo ago
i'm talking about ubers, airbnbs, amazons, googles and facebooks of the world, marketplace software that aggregates supply and demand > They won't, but this is the actual reason. Nobody likes dealing with support or maintenance
65.
by m_ke 8mo ago
it's not the end of software, there will be infinitely more of it it's the end of 80-90% margins that the valley coasted on for the last 20 years. Salesforces of the world will not lose to an LLM, they will lose to thousands of ti
66.
by m_ke 10mo ago
Was it just me or did Opus start producing incredibly long responses before the crash. I was asking basic questions and it wouldn't stop trying to spit out full codebases worth of unrelated code. For some very simple questions about da
67.
by m_ke 10mo ago
What a lot of people don’t know is that SWE-bench is over 50% Django code, so all of the top labs hyper optimize to perform well on it.
68.
by m_ke 11mo ago
Yes it would require completely new hardware and most likely ditching gradient descent for alternative optimization methods, though I'm not convinced that we'd need to turn to discrete optimization. Some recent works that people m
69.
by m_ke 11mo ago
Before we had proper GPUs everyone said the same thing about Neural Networks. Current model architectures are optimized to get the most out of GPUs, which is why we have transformers dominating as they're mostly large dense matrix mult
70.
by m_ke 11mo ago
yeah it's not happening anytime soon, especially with the whole economy betting trillions of dollars on brute fore scaling of transformers on manhattan sized GPU farms that will use more energy than most mid western states. Brains do i
71.
by m_ke 11mo ago
https://transformer-circuits.pub/2022/toy_model/index.html https://arxiv.org/abs/1803.03635 EDIT: don't have time to write it up, but here's gemini 3 with a short explanation: To si
72.
by m_ke 11mo ago
We really need new hardware optimized for sparse compute. Deep Learning models would work way better with much higher dimensional sparse vectors but current hardware only excels at dense GMMs and structured sparsity.
73.
by m_ke 11mo ago
Soon OpenAI will make its own chips and Nvidia its own foundational models
74.
by m_ke 1y ago
You could say the same about the car manufacturers, lehman and every company in the valley. The whole US economy is getting propped up by AI spending, and to continue on the scaling law ladder each new iteration requires 10x more investment
75.
by m_ke 1y ago
Yes US can fail first and take OpenAI down with it
76.
by m_ke 1y ago
Scaling Laws are the ultimate VC ponzi scheme vehicle. Keep raising 10x more for each round of scaling and very quickly you get large enough to be able to bully anyone into playing with you. Sama got big enough to be able to twist any arm h
77.
by m_ke 1y ago
Worse than the bubble is what's happening to the rest of the economy. If you remove AI related spending the US economy is trending in a really bad direction. This bubble popping will definitely take down crypto with it and rip through
78.
by m_ke 1y ago
I wonder how Google built their empire. Who knows? I’m sure they didn’t scrape every page and piece of media on the internet and train models on it. My point was that the large players have monopoly hold on large swaths of the internet and
79.
by m_ke 1y ago
Yeah we had to roll through a bunch of proxy servers on top of all the other tricks you mentioned to reliably download at a decent pace
80.
by m_ke 1y ago
I used to work on video generation models and was shocked at how hard it was to find any videos online that were not hosted on YouTube, and YouTube has made it impossibly hard to download more than a few videos at a time.
81.
by m_ke 1y ago
I was told that they were actually required to list them even if it’s someone transferring internally. It was for a few specific ML research roles that I was interested in, of which there were very few in NYC and during the interview proces
82.
by m_ke 1y ago
When I was looking for work early this year I was told that most of the Google NYC roles were listed for internal transfers and that most of the actual hiring was in Warsaw (with 1000s of open roles, which I was told by Google recruiters at
83.
by m_ke 1y ago
They’re all already doing this and doing it more will go unnoticed
84.
by m_ke 1y ago
Yeah it's even worse than that. These big cos will be incentivized to move whole teams out of the US since it will be easier to hire from other countries for offices in Paris / Zurich / Warsaw / etc.
85.
by m_ke 1y ago
Exactly like the internet bubble. I've been working in Deep Learning since 2014 and am very bullish on the technology but the trillions of dollars required for the next round of scaling will not be there if GPT-5 is not on the exponent
86.
by m_ke 1y ago
Could all pop today if GPT5 doesn’t benchmark hack hard on some new made up task.
87.
by m_ke 1y ago
No they’re usually done at each attention layer.
88.
by m_ke 1y ago
Would be interesting if this was a coding focused model optimized for Mac inference. Would be a great way to undercut Anthropic. Pretty much give away Sonnet level coding model and have it work with GPT-5 for harder tasks / planning.
89.
by m_ke 1y ago
Judging by the @america feed on twitter it will be all of the fascism with none of the fake MAGA populism. Good luck finding a constituency for that outside of a handful of billionaires and their groupies.
90.
by m_ke 1y ago
I've heard from someone who knows that they're scamming people like crazy. Supposedly they also setup a bunch of LLCs to hire influencers then never paid them.

More ›