kolinko

23 ms·

541.
Show HN: Speeding up LLM inference 2x times (possibly)(asciinema.org)
419 points by kolinko 3y ago|114 comments
542.
by kolinko 3y ago
Aside from Karpathy's videos, I'd say: Start by any book on traditional neural networks, so you get a decent understanding of what neural networks are, what is an activation function and what the backpropagation is. From then on,
543.
by kolinko 3y ago
+1. When in Poland I use Allegro all the time - way more reliable than Amazon.pl. Not as good as Amazon in the states though. My friends who moved back from the Bay Area still prefer to use Amazon, but they use Amazon.de instead - similar s
544.
by kolinko 3y ago
That’s already being done, and the taxes/limits on co2 pollution will be increasing. So nothing new here. What you’re implicitly suggesting is government allowing/banning certain uses of technology because they are deemed not wort
545.
by kolinko 3y ago
LLMs are also good at connecting the dots and figuring out patterns in a program. For this case it's ok if they miss even 99% of the times, because you can formally verify which solution is correct, and checking a 100 solutions is orde
546.
by kolinko 3y ago
Formal logic / formal proofs. We have good systems for verifying that. The proper flow is that you use LLM to generate decompilation steps, along with potential proofs, and then use old algorithms from 1970s that verify that the steps
547.
by kolinko 3y ago
What kinds of projects did you build? I'd risk saying that it's more likely you just didn't spot where graphs could be useful, or you implemented a graph not recognising it as such. Even if you work with pure webdev - the lea
548.
by kolinko 3y ago
Expert systems, formal logic, prolog and so on. That was the "AI" of the 70s. The systems failed to grasp real world subtleties, which LLMs finally tackle decently well.
549.
by kolinko 3y ago
As a person from a different language/accent who has to deal with this on a regular basis - having assistants like Siri not understand what I want to say, even though native speakers don't have such problem... Or before an advent
550.
by kolinko 3y ago
A nice story about Xerox discovering this issue in 2003, when their copiers began slightly changing random numbers in copied documents https://www.theverge.com/2013/8/6/4594482/xerox-copiers-rand...
551.
by kolinko 3y ago
Bank Holidays, days of mourning, or a pandemic and war. The systems should be done so that the calendar can be agile like that.
552.
by kolinko 3y ago
Or people from Europe. It’s 50/50 whether an automated station will accept my European card - can’t figure out the pattern there
553.
by kolinko 3y ago
Ouch, bad news for Red Bull drinkers, if true
554.
by kolinko 3y ago
+1. I built a smart contract decompiler (eveem.org), and got deep into formal verification / code analysis path during that time. There was a ton more I could do back then if I had access to gpt-4. Some analysis methods were extremely
555.
by kolinko 3y ago
That's not what the post says if I understand correctly - the post explains why in certain situations the "noise" disappears, and in other cases it doesn't. See comparison between f and g functions.
556.
by kolinko 3y ago
Without systems like this (patriot and the others) people in Ukraine would be without heat and electricity last winter and this winter. I don’t know if you followed the stats - I did (it’s a neigboring country to mine and I have friends the
557.
by kolinko 3y ago
It’s people operating the drones, there is a human in the loop. Although it’s more because of a lack of proper guidance ai than the lack of will. Even with semi-autonomous Switchblades there is always a human choosing the initial targets -
558.
by kolinko 3y ago
Wow, til. Is there anything they can do about the breath? (ditto people on keto diet)
559.
by kolinko 3y ago
But it's not as good with distinguishing complex organic molecules one from another, no? I.e. between various proteins of a similar size.
560.
by kolinko 3y ago
The previous changes didn’t happen so rapidly. Also, there is serious evidence that this one is anthropocenic. And finally, even if it wasn’t, our societies will be affected so much that it is a good investment to work on preventing it. And
561.
by kolinko 3y ago
Is there a platform that managed to implement a decent sandbox without a review process a’la Apple? Android definitely fails in this aspect, if you look at stats on viruses etc.
562.
by kolinko 3y ago
Not exactly what you’re looking fir, but perhaps you’ll find it useful - llama benchmarked on all M-series chips, and in comments there are comparisons with nvidia. https://github.com/ggerganov/llama.cpp/discussion
563.
by kolinko 3y ago
This solution is/should be obvious and easy to anyone who knows algorithms and data structures. I didn't invent op's question, but an answer that it should be a list is obvious right after reading the question. The question i
564.
by kolinko 3y ago
H2 systems and electronics themselves are worth more than $6k probably.
565.
by kolinko 3y ago
It’s the simplest metric to see how much of the model is reasoning and hiw much is just rehashing. If it can’t solve puzzles then it’s more likely to make mistakes when understanding questions that are out of its training set.
566.
by kolinko 3y ago
If I’m not mistaken, for parallel inference requests and for prompt preprocessing it’s compute bound. Also, if you have just a single model you want to optimise (and not the training), you could build an array of asics that do specific matr
567.
by kolinko 3y ago
Also, mirrors that curve near the far end. Each time I visit US I'm frustrated by the american cars having flat mirrors. The blind spots are insane. I guess it's due to some weird regulation? In EU mirrors have their last 2-4cm cu
568.
by kolinko 3y ago
Which is a good thing and an intended behaviour.
569.
by kolinko 3y ago
I would consider a phone as damgerous as a scooter, if not more - considering the amount of private information in it and the access to bank accounts it grants. The damage that can be incurred by malware in a phone can ne greater than a med
570.
by kolinko 3y ago
Interesting thought, but how would that work?

More ›