musebox35

6 ms·

31.
by musebox35 4mo ago
Could you briefly describe your workflow for doing that or give a pointer to a blog you wrote/like that aligns with the process? Thanks in any case, happy designing ;-)
32.
by musebox35 4mo ago
Please see Ben Evans’ podcast on a good take on this. Coding is just one of the task you do in your job, it is not the job or at least it probably is not. You do not get paid to code, you get paid to make a set of decisions that create valu
33.
by musebox35 5mo ago
I wonder what will be the effect of this on open source software. I do not mean the technical aspects, easy of coding, documenting, explosion of PRs, .. Social aspects scare me a bit more. If everyone is building their own version, will peo
34.
by musebox35 5mo ago
Debating how not to use AI will not get anyone anywhere since negative framing almost never works with humans (it also does not work with llms). Let’s concentrate on how to build closed loop systems that verify the llm output, how to manag
35.
by musebox35 5mo ago
Not all parts of the code is equal in this respect. Those parts pertaining to the user visible portion (API of a library, command args of a CLI, UI of a GUI/TUI app, endpoints in a web service, etc.) are closely related to the spec. Th
36.
by musebox35 5mo ago
I am rereading the Asimov robot novels. A decrease in human to human interaction is a major side effect that he has foreseen. Decreasing interaction and collaboration are some of the core themes.
37.
by musebox35 5mo ago
I think this question is one of the more concrete and practical ways to attack the problem of understanding transformers. Empirically the current architecture is the best to converge training by gradient descent dynamics. Potentially, a dif
38.
by musebox35 5mo ago
Thanks for posting a through and accurate summary of the historical picture. I think it is important to know the past trajectory to extrapolate to the future correctly. For a bit more context: Before 2012 most approaches were based on hand
39.
by musebox35 5mo ago
They say that they did test but the coverage was not enough to pick it up, at least for the prompt change: “ After multiple weeks of internal testing and no regressions in the set of evaluations we ran, we felt confident about the change an
40.
by musebox35 5mo ago
I attended the related session at Next’26 yesterday. From my understanding it is a new backend and they will release the torch tpu source on github in one or two months. It will not support all ops initially but they are moving fast. Still
41.
by musebox35 7mo ago
My ancient boxed copy of Visual Basic for DOS 1.0 that supported mouse clicks on TUI buttons would have found your viewpoint quite offensive if it had any AI in it ;-) Oh boy, good old days.
42.
by musebox35 7mo ago
Similar trend in open text-to-image models: Flux.1 was 12B but now we have 6B models with much better quality. Qwen Image goes from 20B to 7B while merging the edit line and improving quality. Now that the cost of spot H200s at 140GB came d
43.
by musebox35 9mo ago
I guess, the sense of accomplishment is very person dependent. I enjoy programming a lot, but it is easy to find people who would challenge themselves to scale the said website to a million users/X view per day. I don't know the w
44.
by musebox35 9mo ago
That is likely. Another factor that came into my mind is the gpu using less power due to simpler computations. You can store less data for grayscale, so you need to go over less pixel data to do effects etc. Whether accessibility controls a
45.
by musebox35 9mo ago
The bitter lesson here is that if you want to control a business you can not avoid or outsource marketing. It is a huge part of any trade and you have to bear the marketing cost. I totally understand the desire to avoid it and concentrate
46.
by musebox35 10mo ago
If you dig ml/vision papers from old, you will see that formulation-wise they actually did, but they lacked the data, compute, and the mechanistic machinery provided by the transformer architecture. The wheels of progress are slow and
47.
by musebox35 10mo ago
I think this is what is blunted by mass education and most textbooks. We need to discover it again if we want to enjoy our profession with all the signals flowing from social media about all the great things other people are achieving. Stay
48.
by musebox35 10mo ago
I think this is more about mechanistic understanding vs fundamental insight kind of situation. The linear algebra picture is currently very mechanistic since it only tells us what the computations are. There are research groups trying to go
49.
by musebox35 10mo ago
I have deep respect for cuda and Nvidia engineering. However, the arguments above seem to totally ignore Google Search indexing and query software stack. They are the king of distributed software and also hardware that scales. That is way T
50.
by musebox35 10mo ago
That is insightful. Courage to take risks means higher standard deviation in outcomes, more visible successes, but also more hard failures. Risk averse cultures have more stable outcomes, no big successes, but also less financially cripplin
51.
by musebox35 10mo ago
The computations in transformers are actually generalized tensor tensor contractions implemented as matrix multiplications. Their efficient implementation in gpu hardware involves many algebraic gems and is a work of art. You can have a tas
52.
by musebox35 10mo ago
As a 15+ years emacs user the only item on my wishlist is client-server remote editing mode similar to that of vs code. Then I can go back to using emacs on cloud VMs. Does anyone know a solution to this that works as good as VS Code even w
53.
by musebox35 11mo ago
In some commercial contexts with the savings from that 20%, you can buy a lot freedom and then with the freedom you bought you can make more free things :)
54.
by musebox35 11mo ago
In certain contexts 20% is a lot bucks, leaving that on the plate would be very wasteful ;-)
55.
by musebox35 11mo ago
Google Deepmind is the closest lab to that idea because Google is the only entity that is big enough to get close to the scale of AT&T. I was skeptical that the Deepmind and Google Brain merge would be successful but it seems to have wo
56.
by musebox35 11mo ago
I agree, it is another important factor. Pandemic pay and hire rates certainly accentuated this.
57.
by musebox35 11mo ago
Sadly this naturally happens in any field that ends up expanding due to its success. Suddenly the number of new practitioners outnumbers the number of competent educators. I think it is a fundamental human resources problem with no easy fix
58.
by musebox35 11mo ago
I totally agree with your assessment. I just wanted to highlight that “implementation agnostic” is both a blessing and a curse. You can always apply the principles of diataxis but it provides near zero guidance on how to actually build the
59.
by musebox35 11mo ago
waymo already has driverless taxi service in a major us city and is expanding. Tesla is in the process. again this is if they cover the last 5%. Scalability arguments wont matter when they can not launch such a service. And no, cmos cameras
60.
by musebox35 11mo ago
And that last 5% is the toughest nut to crack. There is a reason waymo is way ahead even if they can not scale. Cameras are passive devices with relatively poor dynamic range and low light behavior. They are nowhere near a match/replac

More ›