6 ms·
Mathematicians still don't know the fastest way to multiply numbers
- bombela 3mo agoDoes the article just end after describing the problem for me only? I am left wanting for more.
- kadoban 3mo agoIt's an open question in mathematics/CS, that's all we know. If you want to know more, get to mathing :)
- bombela 3mo agolooks like ads swallowed the content and now I am hitting a paywall.
- woggy 3mo agoAsk Sol 5.6 for the answer
- wolfi1 3mo agoif you want a little more in depth explanation of the whole multiplication thing Ican recommend TAOCP volume 2. it has a section called 'How fast can we multiply?' it should provide more insight. there is a paper from djb (Daniel J Bernstein),which I can also recommemd: https://cr.yp.to/lineartime/multapps-20080515.pdf https://cr.yp.to/lineartime/multapps-20080515.pdf
- bombela 3mo agoThis is one of my favorite section of TAOCP! Thank you for the link.
- Anduia 3mo agoScroll down, there is a huge ad but the article continues.
- bombela 3mo agotrying again, now there is a paywall anyways. I give up.
- deleted 3mo ago[deleted]
- qingcharles 3mo agoAmazed I hadn't heard of this before. Would be interesting to see if they can prove that they have discovered the fastest at O(n × log n) or whether there is more still to come.
- pfdietz 3mo agoLower bounds are really hard.
- pgreenwood 3mo agoI don't think the article did a great job with their two digit illustration. They simply state: (ad + bc) = ((a + b) × (c + d)) – ac – bd. First note this equation is more clearly be written as: ad + bc = (a + b)(c + d) – ac – bd. To see why this is so first expand (a + b)(c + d). (a + b)(c + d) = ac + ad + bc + bd now (a + b)(c + d) − ac − bd = ac + ad + bc + bd − ac − bd Hence ad + bc = (a + b)(c + d) – ac – bd.
- jordigh 3mo agoErm, I'm not sure you clarified anything other than removing one pair of spurious round brackets that who knows why they're there in the source material. There are other weird formatting things in this article, which I blame on AI. I don't think the whole article was written by AI, but the copy-editing and formatting looks like an AI messed up things, such as those pointless round brackets or the inconsistency of multiplication (sometimes there is a × sign, sometimes there isn't), or this: > have suspected that O(n²) was an inherent speed limit for multiplication. The celebrated Soviet math professor Andrey Kolmogorov posed the O(n^2) The AI can't decide on notation.
- estetlinus 3mo agoThe AI is just like me, then.
- entrope 3mo ago> The AI can't decide on notation. It is sadly emblematic of how much SciAm has cheapened itself over the past decade. They used to care about technical details. Now they serve up poorly formatted rehashes of news (largely) from 65 years ago.
- jmalicki 3mo agoI have actually had a ton of success using Strassen matrix multiplication kernels with extra structure in custom CUDA kernels (e.g. a covariance matrix is symmetric positive definite, or can be represented with Cholesky, and that comes up in a ton of useful computation). It's been a couple of years, but IIRC I would find it would start to win over the standard kernels at ~n>2500 or something (and in addition to Strassen was also exploiting the explicit structural constraints of the matrix, so not a completely fair comparison).
- TimorousBestie 3mo agoIf you’re interested, I found https://arxiv.org/abs/2505.09814v1 https://arxiv.org/abs/2505.09814v1 to beat Strassen for medium-sized and larger covariance matrices. YMMV of course. Takes a little adjustment for XX^H but it’s not so bad.
- thx1138jgs 3mo ago[dead]
- ccleve 3mo agoOk, maybe I don't understand the problem, but it seems obvious that it should never be greater than O(min(n1, n2) * 10), where n1 and n2 are the lengths in digits of each argument, and assuming we are multiplying decimal numbers. Take the first digit of the longer number. Multiply it by the shorter number and store the result. Take the second digit of the longer number. If it matches the first digit, do a lookup of the last result and use that, else multiply and store. Repeat. There will be a maximum of 10 * (length of the shorter number) multiplies, because there are only 10 unique digits. After that every operation is a lookup. You could even do a tiny optimization by skipping the multiplication for the zero digit. Worst case, the two numbers are the same length, in which case it's O(n/2 * 10), which is a heck of a lot better than O(n log n). What am I missing here? EDIT to respond to the comments: in the article, they are only counting the number of multiplies in the O() value. They are not including the adds.
- pfdietz 3mo agoAdding up those n1 numbers, each at least n2 digits long, takes O(n1 * n2) time.
- aaronbwebber 3mo agoyour algorithm for multiplication involves doing multiplication?
- eru 3mo agoYou might want to learn about recursion. It'll blow your mind.
- SideQuark 3mo ago… which needs a terminating case, which cannot be not defined as the same recursion.
- eru 3mo agoWhen talking informally, people often omit mentioning the base case when it's obvious or trivial. Btw, your recursion doesn't necessarily need a terminating case. See eg this definition of the list of Fibonacci numbers in Haskell: fibs :: [Integer] fibs = 1 : 1 : zipWith (+) fibs (tail fibs)
- hankbond 3mo agoShout out to a cookie ToS modal on top of an email newsletter modal on top of the article. What a great way to make me immediately click back and leave the site.
- Terr_ 3mo agoShout-out to javascript disabled by default :)
- derwiki 3mo agoAnd Safari Reader Mode!
- aaron695 3mo ago[dead]
- snowwrestler 3mo agoFrom the HN guidelines: > Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting. Tons and tons of sites have annoying popups. As such, complaining about it is usually off topic; that is, the complaint has nothing to do with the substance of the specific article that was posted for discussion. The best place for such a complaint is to send it back to the website publisher. Then at least they will know their audience doesn’t like it.
- MarcelOlsz 3mo agoTerrence Howard already figured this out.
- jdw64 3mo agoI already know about fast multiplication algorithms, but it seems there's still no proof that a faster algorithm absolutely cannot exist. In other words, we don't know where the limit is yet. If that gets proven, would programming multiplication algorithms become faster? I'm curious
- TimTheTinker 3mo agoThe O(n log n) algorithm is galactic (only becomes more efficient when multiplying massive numbers) So for numbers we normally work with, no. Maybe with cryptographic operations though.
- jason_s 3mo agoEven crypto isn't that large: 2^4096 is kind of the norm here. Some mathematical researchers are working in the million, billion, or even trillion-bit range.
- groundzeros2015 3mo agoMatrix multiplication is constantly getting improved but these methods aren’t improvements on practical implementation.
- ChocMontePy 3mo agoMirror in Yahoo News: https://tech.yahoo.com/science/articles/mathematicians-still-don-t-know-130000515.html https://tech.yahoo.com/science/articles/mathematicians-still...
- avmich 3mo agoWe can likely use different number representations for faster results. E.g. numbers in the form of coefficients to prime factors can be multipled at O(n) time, right?
- necovek 3mo agoYou mean like those guaranteed-always-compresses-by-at-least-one-bit algorithm patents gzip page made fun of? In your case, doing prime factoring is where the cost would be, wouldn't it?
- avmich 3mo agoYes, but the point is to look for different representations, not necessarily use this specific one.
- necovek 3mo agoYes, but the point is that getting to a different representation from a natural one (eg. binary one with computers or a decimal one for human use) is going to have some "external" overhead which is likely to negate the savings. Yes, sometimes a neat trick like the one in the article of replacing one multiplication with substraction appears, but that is not as universal.
- zeroonetwothree 3mo agoTrue but addition becomes a lot less efficient in this representation :)
- WCSTombs 3mo agoTo make both addition and multiplication O(n), you can store numbers as their residues modulo a bunch of different primes and appeal to the Chinese Remainder Theorem. However, then size comparison becomes difficult.
- 3mo ago
- charcircuit 3mo agoHow is it measured? A lookup table takes 1 step to find the answer of a multiplication.
- blovescoffee 3mo agobecause you're looking up a result not solving for it
- charcircuit 3mo agoYou solve it by looking it up.
- bmacho 3mo agoThis would be the solution for any problem/algorithm, wouldn't it? Factorize big numbers, sort an array, beat stockfish at chess, create a SOTA microkernel OS from English description. All O(1) with lookup table! It's not how complexity works.
- charcircuit 3mo ago>This would be the solution for any problem/algorithm, wouldn't it? Yes, but it suffers from a large amount of space complexity, and probably would have high constant factors in practice.
- bmacho 3mo agoExactly. So what I wrote debunks your assumptions and proves your argument wrong. This is how conversations normally go.
- charcircuit 3mo agoWhat you wrote agreed with me. >All O(1) with lookup table!
- boomlinde 3mo ago
- JoelJacobson 3mo agoBack in 2024, I was trying to optimize PostgreSQL's NUMERIC data type, which is base-10000, using Karatsuba. The problem of finding the optimal threshold of when to switch to Karatsuba turned out to be really hard, since it depends on the size of both factors combined. After some hundreds of hours, I gave up, and started thinking about if there could be a simpler solution. I came to think about another idea I'd had before but abandoned, about 64-bit modernizing the digit base from 10k to 100M, but that would be a challenge due to existing data on disk. Desperate of finding a solution, I wondered if it could be fast enough to do on-the-fly conversion back and forth between base-10k and base-100M, and then realized that, yes, of course, it will be fast already for quite small N (testing shows already between 3-6 base digits). The trick basically reduced the N in O(N^2) into half, i.e. O((N/2)^2), with some O(2*N) cost for the conversion back and forth. I had a lot of fun hacking on this idea together with the maintainer of the NUMERIC data type, and after two months the patch finally was ready and got committed: https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=8dc28d7eb868b6ce5a51614628bf46fc63c7e90c https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit...
- JoelJacobson 3mo agoHere is the full pgsql-hackers mailing list thread where you can follow our work from initial idea to commit: https://www.postgresql.org/message-id/flat/9d8a4a42-c354-41f3-bbf3-199e1957db97%40app.fastmail.com https://www.postgresql.org/message-id/flat/9d8a4a42-c354-41f...
- sureglymop 3mo agoLove this, thank you for providing the context!
- wormslayer666 3mo agoA bit tangential, but the folks behind the GNU Multiple Precision Library (GMPLib) have the problem of choosing algorithms more or less fleshed out. They've got some fairly approachable manual pages[1] for the various algorithms they use as operand sizes scale up, where Karatsuba is only the second of six options in terms of operational complexity. [1] https://gmplib.org/manual/Multiplication-Algorithms https://gmplib.org/manual/Multiplication-Algorithms
- deleted 3mo ago[deleted]
- NetMageSCW 3mo agoPaywall.
- stfnon 3mo agoback in 2008 I had actually a ton of success using Strassen turboplicattion kernels with some extra custom CUDA lora (its more for Cholay) than anything else but it worked. Obviously I was forbidden to use it due to interest of shana.
- Davidzheng 3mo agoThe complexity is obviously nlogn - it's just hard to prove (this comment is only somewhat serious)
- Davidzheng 3mo agoMaybe they should reduce sorting to multiplication lol
- monster_truck 3mo agoOn some level I feel like we aren't even really trying. This stuff is so damn dry, though. (warning, I refuse to like math and address it on my own terms, proceed further at your own peril) Started looking into exact integer matrix multiplication, wanted to use it for some differential bullshit to find whatever they call the magic numbers that simplify a lot of complicated work into virtually no work for suspension/drivetrain/grip simulations at scale To my surprise rocm didn't even usefully accelerate it! I said there is no fuckin way a 7900XTX is only good for 0.5 TOPS when working with 64 bit integers. I knew RNS/CRT/GEMM was a thing which led me to this https://github.com/RIKEN-RCCS/GEMMul8 https://github.com/RIKEN-RCCS/GEMMul8. Nothing pisses me off more than CUDA having something ROCm doesn't. So I told the models to try and fill the moat in with concrete. Think I got up to almost 3 TOPS before I stopped, and there are some pretty absurd wins for int32/other shapes. Here's the slop https://github.com/doublemover/RNS8 https://github.com/doublemover/RNS8, I haven't cleaned it up or anything. Life has gotten in the way so I had to set it down, and fighting the air conditioning when its "95 feels like 107" and the sky is filled with smoke is... not cool. I will finish it after summer. The HotAisle guy is a legend and hooked it up with some credits so I will be able to do the same for CDNA3, it at least compiles and runs but it has not been optimized/tested much yet. Started with ChatGPT 5.5 but it sucked. I'm not paying $200/mo to play reset bingo while they figure out their bugs, especially without 20x. They lit my last $50 on fire in like 20 minutes with no remediation past "keep paying and you'll get more resets". Don't sleep on Deepseek, V4 Pro was responsible for the biggest leaps and it cost all of $15. It's genuinely great. The only way I'd go back to a closed model is if it was completely free. It will be fun to see how much better models are in a few months.
- 6510 3mo agoIf you do it in binary you only need addition. 12 × 34 = 0xC x 0x22 = 1100 x 100010 Only two 1's! 1100 add 5 zeroes + 1100 add one zero = 110011000 = 408 ta-daa!
- cvoss 3mo agoAddition and conditional bit shifting (which is multiplication by powers of 2).
- ErroneousBosh 3mo agoI don't know what age range "grade school" is, but I remember being taught that method when I was about 7 or 8, although it didn't really "land" properly until I read the short story "The Feeling of Power" by Isaac Asimov. What I'm surprised to see left out here (unless I missed it in the page's horrible formatting) is a mention of the way that computers multiply two integers. They use a technique I saw described in a book when I was about 11 as the "Russian Farmer Method" (or something like that, it was in English and I might have misremembered it). In that you shift the multiplier right and multiplicand left, halving one and doubling the other. If the multiplier is odd, add the multiplicand to the total. It's really doing the same thing as "long multiplication" like you're taught in primary school but in binary so when you add a 0 to the right for the higher order digits you're doubling, not multiplying by ten. If you write code to do it you'd shift the multiplier first then consider whether or not to add by testing the Carry flag, or "Link bit" if like the author of the book I read you're demonstrating it on a PDP8 ;-) But let's have a worked example, picking two numbers at random 205 * 707, use the smaller as the multiplier: 205, 707 odd, add 707 to total 102, 1414 even, disregard 51, 2828 odd, add 2828 to the total 25, 5656 odd, add 5656 to the total 12, 11312 even, disregard 6, 22624 even, disregard 3, 45248 odd, add 45248 to the total 1, 90496 odd, add 90496 to the total -------------------------------------- 144935 If we're disregarding shifts and adds as completing in negligible time, well, this whole thing is just done with shifts and adds, and you can predict how many of them by identifying the leftmost bit set in the multiplier.
- deleted 3mo ago[deleted]
- SkiFire13 3mo ago> If we're disregarding shifts and adds as completing in negligible time When considering multiplication algorithms the parameter N is the number of digits of the two numbers. In that model adds do not complete in negligible time, and instead take O(N) time.
- less_less 3mo agoYeah, that shift-and-add algorithm is sometimes used on microcontrollers, either in software if there's no hardware multiplier, or in hardware if you want the bare minimum in acceleration at a tiny cost in area. Adds are not really considered negligible; the article is just sloppy. (Some shifts might be negligible in some models because a fixed shift requires no logic gates.) The cost of the adds in Karatsuba is significant both theoretically and in practice, and determines the cutoff where Karatsuba is useful. But the exponent in O(n^(log_2 3)) is dominated by the recursive multiplications; the adds only affect the leading constant hidden in the O().
- Kyuren 3mo agoIf the 2019 algorithm is only useful for "galactic numbers" that's really neat because it will help a lot in the future as it seems like computation is only increasing in scale.
- cwillu 3mo ago2 ^ (713 739 807 325 663 489 766 475 852 620 783 120 641) digits. Nope, an implementation of that algorithm will never be directly useful.
- chmod775 3mo agoIf it ever had the slightest chance of being useful on Earth, it would not be called a "galactic algorithm". In this case that algorithm might be more than galactic: there isn't enough matter in the observable universe to build a conventional computer able to execute it. In fact a computer executing it would be so large that speed of light would be the main constraint limiting execution speed, not theoretical algorithmic complexity that ignores data locality.
- nobrains 3mo agoWhy do we make computers multiply single digit numbers, instead of taking the result from a lookup table, like humans do? To answer my own question, I am assuming it would be because multiplying would still be faster than reading from a lookup table? Any ideas?
- sparky_z 3mo agoIn a sense, they do exactly that! But since there are only two single-digit numbers in binary, it makes for a pretty short table.
- less_less 3mo agoHardware multipliers often use a sort of base-4-ish lookup table trick as well, using the Booth-Wallace algorithm. Booth's idea is to rewrite one of the inputs in base (usually) "4", except that the digits go from -2 to +2 instead of 0 to 3. (That's five possible digits! This helps the rewriting stage not have to propagate carries. Carry propagation is very expensive.) You can use Booth in a base higher than 4, especially if you know one of the multiplicands before the other, but you run into tradeoffs pretty quickly. Then for each digit, you select between the other input multiplied by 0 (all zeros), +1 (identity), +2 (shift left by one bit), or -1 or -2 (flip all the bits of +1 or +2, plus a correction). Since a number has about half as many digits in base 4 as in base 2, you have about half as many digits to sum as if you'd done this in base 2. Then you sum up all those results, but since carry propagation is expensive, you mostly use "compressors", e.g. you sum up three intermediates at a time, but you do it bit-by-bit, where three 1-bit numbers add up to a 2-bit number (from 0 to 3). This is called a Wallace Tree. The point is that you are generating carries, but you aren't propagating them, just adding them back into the set of things to be summed. At the end of the tree step, you have just two numbers left, and you add them conventionally. That's the only step that needs full carry propagation. If you are implementing a multiply-add, or multiplying several numbers and adding up all the results or similar, then you usually only need one full carry propagation stage. The overall circuit has quadratic area but only a logarithmic depth in gates. IIRC whether to do Booth or not is a tradeoff: at least in some circumstances the rewrite steps make it slower but smaller. Hardware tool vendors have done a lot of work to tune these circuits very tightly, using e.g. specialized gates like AOI, heuristics for how to set up the tree, etc.
- moi2388 3mo agoFor large numbers, operations like addition don’t matter? Only multiplication? And now we want to find the fewest amount of multiplications? Okay. No problem. (ad + bc) = d + d .. + d + c + c .. + c There we go, zero multiplications.
- entrope 3mo agoThe article is explicit that addition is O(n), with n digits, which is cheaper than multiplication is believed to be. Naive multiplication is O(n*n) -- considerably less than your algorithm.
- moi2388 3mo agoMy algorithm is O(n+n+..n) which is O(n), since there we also ignore addition fortunately :D
- AlotOfReading 3mo agoIt would only be O(n) if the number of additions was constant. Here it varies with the size of the multiplier, giving us O(n*m).
- QuantumNoodle 3mo agowHy DOnT tHeY ASk aI
- tobadzistsini 3mo agoArchive link. Fuck paywalls. Surprised nobody posted yet. https://archive.ph/k3dnD https://archive.ph/k3dnD
- layer8 3mo agoNot everyone is seeing a paywall. Not sure if it’s the adblocker or other factors.
- morpheos137 3mo agoKaratsuba in my understanding only becomes advantageous for very large numbers relative to human scale. Mathematically it is interesting but in engineering terms the overhead usually is not worth it for practical applications. There is a fundamental trade off between factor size and product precision. If you can accept lower precision then floating point works well for large in human scale numbers.
- layer8 3mo agoThe article discusses how Python uses it for numbers above ~2100 bits for that reason. That’s way beyond your regular floating-point type in terms of precision.
- jdhwosnhw 3mo agoI was under the impression that because the grade school technique we learn is really just convolution over the digits, the fastest algorithms achieve o(n logn) via fourier transforms. Is that not the case?
- calf 3mo agoJesus, I took 4 years of Electrical Engineering and at no point did any professor make this brilliant analogy. It would have helped many students, including myself.
- alok-g 3mo agoCan you explain some more? Thanks.
- jdhwosnhw 3mo agoIf you try multiplying two numbers, say 4 digits each, using the standard grade school technique, you'll see that you end up writing the digit-wise product of each number, with a shift (adding a zero at the beginning) for each digit, followed by a sum. This is literally just a convolution operation over the digits (this is because secretly writing a number in base B is equivalent to expressing a polynomial evaluated at the integer B, and multiplying polynomials is a convolution over the coefficients). By the convolution theorem, this O(N^2) time operation can therefore be accomplished in O(N logN) time by doing multiplication in frequency space and then transforming back. This is because the Fourier transform is O(N logN), via the FFT, and multiplication is O(N). I just looked up the answer to my original question - the Fourier trick is notionally only O(N logN), but because the FFT takes you from integers to floating points, as N gets larger you need to encode more and more bits to achieve enough precision to yield absolute errors <1 after doing both Fourier transforms. The need to encode those extra bits tacks on another logN, taking you to O(N log^2 N).
- alok-g 3mo agoThanks! I now understand.
- projectileboy 3mo agoFor anyone interested in digging, check out Karatsuba's algorithm (https://en.wikipedia.org/wiki/Karatsuba_algorithm https://en.wikipedia.org/wiki/Karatsuba_algorithm), Strassen matrix multiplication (https://en.wikipedia.org/wiki/Strassen_algorithm https://en.wikipedia.org/wiki/Strassen_algorithm), and Toom-Cook multiplication (https://en.wikipedia.org/wiki/Toom–Cook_multiplication https://en.wikipedia.org/wiki/Toom–Cook_multiplication). Of course, there are also implementation considerations. For example you can speed up Strassen by recursively breaking down the matrix into sub-matrices in parallel, but only down to a point - once the sub-matrices get small enough, it becomes faster to simply do a straight Strassen computation. And it depends on your hardware. For something seemingly so simple, you can go pretty far down a rabbit hole!
- fractorial 3mo ago> once the sub-matrices get small enough, the next rabbit hole starts when you try to start lifting F_2 into Z for a given <n,n,n> tensor.
- alienbaby 3mo agoWe're going to need a bigger abacus.