llm_trw

11 ms·

121.
by llm_trw 2y ago
If only we'd figured out a technology that let us move huge torrents of bits around. If only there was a catchy name for it. Something like bit-torrent perhaps?
122.
by llm_trw 2y ago
You're making an even less charitable set of assumptions: 1). I'm incompetent enough to ignore publicly available table benchmarks. 2). I'm incompetent enough to never look at poor quality data. 3). I'm incompetent enoug
123.
by llm_trw 2y ago
To solve mnist without mathematical tricks like convolutions or attention heads you would nees 2.5e42 weights. Assuming that you're using 16 bit weights that 5e42 bytes. A yotta byte is 10e24. That is you'd need 5 exa yotta bytes
124.
by llm_trw 2y ago
The results we got 18 months ago are still better than the current gemini benchmarks at a fraction the cost. As for markdown, great. Now how do you encode the meta data about the confidence of the model that the text says what it thinks it
125.
by llm_trw 2y ago
>You are assuming you can match Gemini's performance I'm not assuming. We already did, 18 months ago with better performance than the current generation of Gemini for our use case. You're falling into the usual trap of thi
126.
by llm_trw 2y ago
The bitter lesson is very little of the sort. If we had unlimited memory, compute and data we'd use a rank N tensor for an input of length N and call it a day. Unfortunately N^N grows rather fast and we have to do all sorts of interest
127.
by llm_trw 2y ago
I used sxml [0] unironically in this project extensively. The rendering step for reports that humans got to see was a call to pandoc after the sxml was rendered to markdown - look ma we support powerpoint! - but it also allowed us to easily
128.
by llm_trw 2y ago
We had to roll our own from research papers unfortunately. The number one take away we got was to use much larger images than anything that anyone else ever mentioned to get good results. A rule of thumb was that if you print the png of the
129.
by llm_trw 2y ago
It was a financial company that needed a tool that would out perform Bloomberg terminal for traders and quants in markets where their coverage is spotty.
130.
by llm_trw 2y ago
>Not sure what service you're basing your calculation on but with Gemmini The table of costs in the blog post. At 500,000 pages per day the hardware fixed cost overcomes the software variable cost at day 240 and from then on you&#x2
131.
by llm_trw 2y ago
Custom trained yolo v8. I've moved on since then and the work was done in 2023. You'd get better results for much less today.
132.
by llm_trw 2y ago
This is using exactly the wrong tools at every stage of the OCR pipeline, and the cost is astronomical as a result. You don't use multimodal models to extract a wall of text from an image. They hallucinate constantly the second you get
133.
by llm_trw 2y ago
You missed the first sentence of the release: >In December, we kicked off the agentic era by releasing an experimental version of Gemini 2.0 Flash I guess I wasn't building AI agents in February last year.
134.
by llm_trw 2y ago
The issue is that in a scientific experiment you can only change the state of A. If for whatever reason B is stuck to T in all cases no one will think that A implies B, they will think that B is independent of A. Yet that would still fit wi
135.
by llm_trw 2y ago
We are rapidly approaching the point where using an embedding model to run over the whole internet will be the best way to rank search results. At that point all the special sauce Google et al have spend decades mastering will be worth as m
136.
by llm_trw 2y ago
The one area that'd make kagi thousands of dollars from me and the apps I use would be to lower their APO searches to a sane price. Currently they charge 2.5c for an API search. This is between 1,000 to 1,000,000 times more than other
137.
by llm_trw 2y ago
I've come dangerously close to being acquired by one of his companies. His management style is best described as humiliation and abuse. Or to put it another way, you can build a lot of pyramids when you use disposable slaves.
138.
by llm_trw 2y ago
The whole paper depends on the factorization: B = (A or B) and (if A then B) In the following table we have the other three operations which give different truth values to what you call the vacuous case. If "if" is replac
139.
by llm_trw 2y ago
I never thought I'd see a misunderstanding of what "implies" means in science versus in logic be the fundamental mistake made in a paper on logic for science. Here's the truth table for implies (if) in logic. | A |
140.
by llm_trw 2y ago
>Hey Claude, translate this to Swahili from English. Ok, now translate my response from Swahili to English. Thanks. We're close to the point where using a human -> stt -> llm -> tts -> human pipeline you can do real time
141.
by llm_trw 2y ago
Homework is a proxy for your retention of information and a guide to what you should review. That somehow schools started assigning grades to it is as nonsensically barbaric as public bare ass caning was 80 years ago and driven by the same
142.
by llm_trw 2y ago
I come back to this every few months and do some work trying to make sense of how tensors are used in machine learning. Tensors, as used in physics and whose notation these tools inherit, are there for coordinate transforms and nothing else
143.
by llm_trw 2y ago
you.com when you disable their search the internet feature.
144.
by llm_trw 2y ago
Right, so who banned tictok? Like I said, a very bipartisan position.
145.
by llm_trw 2y ago
It wasn't the current administration that banned sale of advanced AI chips to China. Dragging us back to the 1930s seems to be a very bipartisan policy.
146.
by llm_trw 2y ago
https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSNGdNl... But also: https://en.wikipedia.org/wiki/Supergod
147.
by llm_trw 2y ago
Countries like Hungary. Not a place I think anyone should try and emulate.
148.
by llm_trw 2y ago
>The complaint here seems to be that rationalists don't take progressive pieties as axiomatic. A trans vegan gang murdering police officers is what's come out of this milieu. I don't see how anyone can say they aren't
149.
by llm_trw 2y ago
To quote a hot potato of a previous age: all bankers may be Jews, but not all Jews are bankers. We know where that one took us.
150.
by llm_trw 2y ago
R1 is interesting. Since I have access to the thinking tokens I can see where it's going wrong and do prompt surgery. But left to it's own devices it gets thing _stupendously_ wrong about 20% of the time with a huge context blowou

More ›