7 ms·
You won’t live to see a 128bit CPU (2019)
- Someone 5y agoI don’t think the argumentation in this article is that strong. Replace 64 by 32 and 128 by 64, and you could have said it about the 32-bit to 64-bit transition. That transition _did_ happen, though. I think a better argument is to look at history. Looking at https://en.wikipedia.org/wiki/X86 https://en.wikipedia.org/wiki/X86, 16-bit x86 appeared in 1978, 32-bit in 1985, and 64 bit in 2003. So, we have 7 years from “8 bits isn’t enough” to “16 bits isn’t enough, and 18 from there to “32 bits isn’t enough.” I see something similar to (and probably a consequence of it) Moore’s law there: “we ‘need’ one more bit of address space every year or so”. If that holds and Moore’s law continues, I expect we’ll run out of 64-bit address spaces about 32 years after we ran out of 32-bits, so around 2035. Top-end machines also should have use for 50-bit address spaces by now. There were 64 TB systems six years ago (http://highscalability.com/blog/2015/8/5/how-do-you-program-a-computer-with-10-terabytes-of-ram.html http://highscalability.com/blog/2015/8/5/how-do-you-program-...). That’s 46 bits, so I think my prediction is in the right ballpark.
- vimacs2 5y agoI don't quite buy that you could really have said the same about the 32 bit to 64 bit transition. 4 Gigabytes is a far cry from 18 exabytes! It is going to be a long while till we scale up to that level.
- bombcar 5y agoIf security starts to depend more and more on a random address NOT being real addressable memory the move to 128 bits may come sooner rather than later. I suspect it’ll happen because a 128 bit wide bus will be useful for calculations.
- vlovich123 5y agoThe width of the bus, the width of the register, and the size of the pointer are all different values. A 64-but cpu has registers that are larger than 64-bits (eg 512bits for AVX512) and the memory bus itself is already either 128 or 256bit already. 128bit pointers seems questionable for scalar processing and vector processing already has wider registers/GPU/TPU for acceleration. From a security perspective 64bits of random space (actually 48ish) is pretty darn good and increasing that space for security is not something I’ve seen compelling papers on.
- nwallin 5y agoBus width is orthogonal to the bitness of a CPU. QPI, the bus Intel CPUs use, is a 20 bit bus. HT, the equivalent AMD technology, uses a 32 bit bus, even on 64 bit CPUs. HBM2, used on certain AMD video cards, uses a 1024 bit bus. The Commodore 64 had 8 bit registers, an 8 bit data bus, and a 16 bit address bus. We do not need 128 bit data types in hardware. We barely even need data types with 64 bits of precision. It crops up every now and then- for instance, 32 bit floats cannot store a latitude/longitude value precisely enough, however a 32 bit integer can store a longitude value to about 1 foot of precision if -2^32 is 180 degrees west and 2^32 is 180 degrees east. (this has the nifty benefit of 2s complement integer overflow doing the right thing, although only with longitude, not latitude. but I digress.) There are a handful of situations where you do need more precision than that- RSA and DH come to mind. But in those situations, you'll need arbitrary precision arithmetic anyway. A 128 bit register won't help a whole lot- certainly not enough to offset the cost of doubling the size of all your ALUs, even if the machine is a special purpose one that does nothing but sign SSL certs all day.
- __d 5y agoYeah, but I remember when computers had 1KB RAM, and 64KB was more than the largest mainframe and seemed insanely massive. And then there was the time, a handful of years later, when "No one will ever need more than 640KB". And then 4GB was stupidly massive, and no-one could want, let alone afford, that much RAM. The same with CPU clockspeed (my first computer ran at 2MHz), storage capacity (I paid several thousand dollars for my first 5MB hard drive). Which is not to say that the curve won't flatten out as we approach the number of atoms in the universe, but ... there's never quite enough of anything.
- emteycz 5y agoWouldn't a computer with 18 exabytes of RAM be totally impractically massive, even with maximal possible miniaturization? I agree with warehouse-scale "thinking machines" going beyond 64 bits, but standard hardware is hard to believe.
- __d 5y agoCurrent chips are basically 2D. Not quite, in that they do have 3D features, but those 3D features are essentially still implementing individual cells in a 2D circuit. And those 2D circuits fit, eg. 4Gbits of DRAM into a single layer chip. The actual chip inside the plastic packaging is quite small -- let's say 4x4mm. Now, you can can pack multiple layers of chips into a single package. Samsung is doing at least 12 layers for HBM, for instance, and they're only 60um thick each. So you've got a chip that is roughly 65536 cells wide, and 65536 cells long, but you can only stack it 12 cells high. Let's get that to 65536 layers: roughly a 4mm cube. That's now 256Tbits per package. Put 36 of those on a DIMM, and you've got an 8PB DIMM, and 512 PB in a 64 DIMM slot server. Perhaps we can double the density? That gives us 4EB in a server. This totally glosses over myriad difficult issues, but it's a very, very long way from being inconceivable when you consider the difference between a 512 byte 2114 RAM chip that was popular when I got my first computer, and what we have today.
- 4ad 5y ago64bit didn't come in 2003, it came in 1991 in MIPS, in 1992, in Alpha, and in 1995, in SPARC. Nintendo 64 and Playstation 2 were using 64-bit CPUs before PCs. x86 was really late in the game.
- kasperni 5y agoHe explicitly said x86.
- kergonath 5y agoThey did not; they just quoted a Wikipedia article on x86. But then, if you ignore everything but x86 the argument is much weaker, because technology did not stop and wait for Intel. It would not be that surprising if x86 ceases to be dominant in the future, at which point when the x86 became 64-bit is about as relevant as, say, the release of the PowerPC 620 (first 64-bit PPC chip, released in 1997).
- CalChris 5y agoThe post said "I think a better argument is to look at history" and then left out 12 years of history. x86_64 was on the trailing edge of that history.
- Someone 5y agoYes, you can argue a lot about the years, and about what constitutes a x-bit machine. The first 32-bit CPU was from 1948, for example (https://en.wikipedia.org/wiki/Manchester_Baby https://en.wikipedia.org/wiki/Manchester_Baby), the IBM/360 had 128-bit registers in 1969 (https://en.m.wikipedia.org/wiki/IBM_System/360_Model_85 https://en.m.wikipedia.org/wiki/IBM_System/360_Model_85) and the Pentium Pro is considered 32-bit, but has a 36-bit address bus. I think the analogy with Moore’s law still holds if you pick machines used in comparable roles. Reason is that you can’t make much use of more memory unless your computer gets faster (an exception would be extremely sparse hash tables, but those would need amounts of memory we don’t have yet) Assuming an extremely generous 2 cycles/byte, a 1MHz 6502 would take 2 seconds to clear 1MB of RAM, or 2,000 to clear 1GB. That’s over half an hour. That first Alpha also only had a 34-bit address bus, limiting it to 16GB of physical RAM (https://en.wikipedia.org/wiki/Alpha_21064#Address_unit https://en.wikipedia.org/wiki/Alpha_21064#Address_unit) I picked x86 because it was used in more or less the same role for decades and because it made the 16-32-64 steps.
- JulianMorrison 5y agoI literally already have too much RAM on my computer for a 32 bit processor to address it all.
- userbinator 5y agoMore than 64GB? Because that's the limit of PAE introduced with the Pentium Pro (36-bit address bus), which was otherwise considered a 32-bit processor.
- JulianMorrison 5y agoNope, only 32 GB. But maybe next computer?
- secondcoming 5y agoMy laptop has 128GB
- mrlonglong 5y agoI'll be quadrupling my threadripper to 256GB. Can't have the the riff raff getting to where I am, you know.
- Retric 5y agoWhile that’s 64TB in a single address space, it isn’t a desktop computer. An apples to apples comparison would consider the 1975 Cray-1 which had a 64 bit CPU, and similarly there have been 128 bit CPU’s.
- tomerv 5y agoThe article mentions this (but it is easily missed): while current chips are 64 bits, in practice the memory addressing is only 48 bits, with plans to extend it to 57 bits for server products [1]. Taking this into account, there's a clear slowdown in the growth of the number of bits: 7 years from 16 to 32, then 18 years for anther 16 bits (32 to 48), and now ~18 more years for just 9 more bits. Seems like exponential slowdown, which makes sense given that the size of memory is an exponent of the number of bits. [1] https://en.wikipedia.org/wiki/Intel_5-level_paging https://en.wikipedia.org/wiki/Intel_5-level_paging
- penteract 5y agoYou're counting the years per bit incorrectly, the comment you reply to gets it right - the new generation is needed when the current space runs out, not when the next limit is needed in its entirety. So the ~18 years you mention are the time from needing 33 bit addresses to needing 49 bit addresses. (And 7 years is the time from needing the 9th bit of address space to needing the 17th).
- nwallin 5y agoMoore's law for RAM capacity has begun slowing over the past few years. The 64 TB system you mention is from six years ago, right about when Moore's law began tapering. If Moore's law had continued doubling RAM capacity every eighteen months, we should have seen four doublings, up to 1PB. But that's not a machine we're seeing. Notably, the bandwidth of RAM does not and has not been following Moore's law, improving linearly, and the latency is more or less constant. So as workloads scale up in size, performance will get worse and worse and worse, until "just buy more RAM" no longer solves the problem. Similar to how swap partitions used to be a thing, but performance is now so bad that swap can't be used as cheap RAM anymore. https://www.semanticscholar.org/paper/Understanding-and-Improving-the-Latency-of-Memory-Chang/a5807e24d7f90f398e7ad8afe221f8edaf4e74e3/figure/0 https://www.semanticscholar.org/paper/Understanding-and-Impr...
- Brian_K_White 5y agoThis is not "No one will ever need more than 640k".
- emteycz 5y agoAre the commenters AI?
- jdshs 5y agoA Wonderful and Beautiful comment, Thank you for your precious informative content!
- zzo38computer 5y agoI think I saw an advertisement for a 128-bit computer in an old issue of BYTE magazine. I think it was from IBM.
- mikeytown2 5y agoI'd guess Digital Alpha (DEC) but I can't find anything mentioning it. Other guess would be Cray.
- eesmith 5y agohttps://en.wikipedia.org/wiki/128-bit_computing https://en.wikipedia.org/wiki/128-bit_computing list contenders as IBM System/370, upgraded DEC VAX, and ICL 2900 Series.
- karmakaze 5y agoThe page lists these uses for 128-computing, any of which seem like sufficient motivators for it to come to pass: - The free software used to implement RISC-V architecture is defined for 32, 64 and 128 bits [...] - Universally unique identifiers (UUID) consist of a 128-bit value. - IPv6 routes computer network traffic amongst a 128-bit range of addresses. - ZFS is a 128-bit file system. - 128 bits is a common key size for symmetric ciphers [...] - The IBM i virtual instruction set defines all pointers as 128-bit. - Increasing the word size can speed up multiple precision mathematical libraries [...] - MD5 is a hash function producing a 128-bit hash value. - Apache Avro uses a 128-bit random number as synchronization marker [...]
- eesmith 5y agoThose seem like poor reasons to justify 128-bit CPU hardware, over 128-bit emulation in software. In particular, MD5 use in new software is strongly discouraged. You'll note that some of them are essentially identical with https://en.wikipedia.org/wiki/256-bit_computing https://en.wikipedia.org/wiki/256-bit_computing so they seem more guided by abstract arguments than concrete engineering reasoning.
- jdkee 5y ago“640k ought to be enough for anybody.” -possibly Bill Gates.
- ncmncm 5y ago... in a joke nobody got at the time, and few even much later.
- eesmith 5y agoCould you explain? Quoting https://quoteinvestigator.com/2011/09/08/640k-enough/ https://quoteinvestigator.com/2011/09/08/640k-enough/ : > "Gates has denied the quotation and the evidence is not compelling" and > "the comments by Nancy Andrews and Jerry Pournelle showed that a common mindset of the period was compatible with informal hyperbolic remarks of this type. The 1989 speech by Gates presented a more sophisticated analysis of the growth of computing power and memory." so I interpret the current retelling of the joke as putting words into Gates' mouth to try and knock him down a notch, and the original 1985 editorial by James E. Fawcette more along the lines of truthiness ("the belief or assertion that a particular statement is true based on the intuition or perceptions of some individual or individuals, without regard to evidence, logic"). Note how the editorial starts "Legend has it".
- ncmncm 5y agoHiya Doc -- Anybody who had done any programming on mainframes, which both Gates and Allen did before the IBM or Apple computers came out, would only ever say such a thing ironically. Memory has always been the limiting factor on what you could program. An essential base skill for any programmer was minimizing memory use so you would not run out later in the project. So, when the PC came out, 640k was a lot for an 8-bit micro, but the 8088 was nominally 16-bit and equipped to manage a megabyte without any hardware fiddling, and its contemporary, the 68000, could already address more.
- eesmith 5y agoSure [0], but who said it, when, and what's the joke? There's no evidence that Gates said it, and he denies saying it. I gave citations for how several people at the time said it unironically. Nancy Andrews: "When the IBM PC-1 was introduced in 1981, it came equipped with 64K of memory. Those of us who upgraded our systems to 128K thought we had all the memory we’d ever need." Jerry Pournelle: "My first microcomputer had 12K of memory. When I expanded to a full 64K, I thought I had all the memory I’d ever need. Hah. I know better now." And the earliest reference to that supposed quote was an editorial with many examples of how software wasn't prepared for larger memory use - hardly an example of "a joke nobody got at the time". As I don't have a Dr. or Phd but I do have a MS, you shouldn't call me "Doc". The appropriate academic title is "Master", though that seems excessive for HN. [0] I started with microcomputers in '83 and did some mainframe programming in FORTRAN so I'm aware of the context.
- dehrmann 5y agoReminds me a little of the Nintendo 64. 64 bits was a marketing gimmick, and even now, most applications barely need 64-bit memory or precision.
- stephc_int13 5y agoModern CPU are already able to process chunks larger than 64bits, (eg. AVX-512) I can see that growing larger over time, in practice it already has with ARM SVE. Because of that, memory bus width can also grow in a useful way. But for adressable memory, I don't see it. 64 bits is likely to be the end-game until we use a radically different CPU architecture. (much bigger than the difference between a Z80 and a Xeon) Of course, predicting the future is notoriously hard, but if you want to make an educated guess, it is better to not expect infinite and monotonous growth. There is always an end-game.
- userbinator 5y agoYou don't even have to look as recently as AVX; 128-bit wide operations were introduced with SSE, in the Pentium III over 2 decades ago.
- FpUser 5y agohardware 128 bit wide ops were supported from at least 1970 (IBM 370)
- whatshisface 5y agoWe don't even have 64-bit computers today. Look at the specs of your hardware and you will realize it's actually running on significantly smaller address buses. Evidence: smaller powers of two showing up in maximum support specs. What we have are 64-bit instruction sets, which could have been used in the 16-bit era if we had wanted to for some reason. The real difference between 64 and 32 bit architectures is that when they went to 64, they said "who cares if we waste a few bits in pointers, we'd rather never have another change in pointer sizes." It represents no longer trying to shave every bit off of memory representations of code and data.
- worik 5y agoWhat about the Transmeta chips? Were they not 128 bit?
- pornel 5y agoWe may end up with 128-bit pointers even without a "true" 128-bit architecture. For example, CHERI has 128-bit pointers with 64-bit address space and 64 bits of metadata to crash harder on memory bugs in C and C++: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/ https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
- p_l 5y agoAS/400 architecture has been using 128bit pointers for several decades by now, even if underneath it involves 32bit or 64bit cpus - except for a tiny bit of code provided by IBM, you're not allowed to target them directly.
- fanf2 5y agoAnd ARM is working with the CHERI team to make a hardware prototype SOC (not FPGA-based like the MIPS CHERI research devices) https://www.arm.com/company/news/2020/10/morello-program-one-year-on https://www.arm.com/company/news/2020/10/morello-program-one...
- pclmulqdq 5y ago18 exabytes isn't that much if you want to uniquely address every memory byte in a (future) datacenter. Papers on RDMA-based unified memory systems are currently producing some very compelling performance results in workloads like pub-sub and mostly-read databases. It's conceivable that in the next 15 years, there will be special-purpose datacenter-scale computers networked with a unified memory space that need address spaces greater than 64 bits, especially with the high-capacity optane DIMMs that Intel produces.
- whatshisface 5y agoYou will need something much more exotic than a data center for the individual processors to need to address more than 18 exabytes. An architecture where every node can directly read the RAM of every other node is called a supercomputer. In a data center, dividing global address into node IDs and true addresses would handle everything you'd want to do. If the top N bits of the global addresses are only ever read by routing software they don't need to be represented in the processor hardware.
- remram 5y agoDoes any system marketed as a "supercomputer" with exabytes of memory actually allow unified addressing of the RAM? My understanding is that they are used through interfaces such as MPI which doesn't work that way.
- whatshisface 5y agoI think you are on to something insofar as noticing that supercomputing vs. cluster computing is a spectrum. You could imagine starting with cluster computing at zero hardware support, and slowly adding more hardware interconnects and putting more work into topology, as you climb towards the platonic ideal supercomputer which, fitting its name, would simply be one super computer.
- remram 5y agoI don't know if manufacturers are going up the ladder though. Even desktop computers are looking more and more like independent interconnected cores (eg NUMA and GPUs). Even in software, compute-heavy workloads tend to be written that way regardless, to sidestep locking issues. I just don't think people would want to address random parts of memory even if a hypothetical exabyte memory hardware allowed it.
- ngcazz 5y agoUnless you’re the Atari marketing department, in which case a 64-bit architecture means two 32-bit CPUs working in tandem. Rawr.
- randyrand 5y agoof those 64 bits, many are used for tagging pointers with certain attributes (PAC security codes, allocation flags, etc) We have so many uses for pointer tagging we really ought to have more.
- incrudible 5y agoArguably, the Ethereum VM is a 256 bit computer which could see a hardware implementation.
- hughrr 5y agoIt does make sense to have 128 bit computers. Consider memory mapped sparse datasets. 18PB looks like a whole lot of address space now but in 50 years, who knows…
- ytch 5y agoJust like Bill Gates says "640K ought to be enough for anybody." in 1981.
- cafed00d 5y ago"640 k ought to be enough for anybody"
- danielrpa 5y agoGive me the hardware and I will find you an application for it! Sure, 128bit CPUs make no sense for the current applications, but if our history shows us anything, is that people max out the hardware with something new as soon as it appears. Part of the solution is using the previous generation's computers to create data and algorithms for the next generation's.
- ergot_vacation 5y ago"Some cutting edge devices have 64 Gb or even 128 Gb, but this is required in very few situations like heavy CAT applications or video editing." Or running two copies of Chrome at the same time.