13 ms·

The prospects for 128 bit processors (1995)

85 points by luu 5y ago

hajile 5y ago
He failed to account for the "why" with previous architectures. 8-bit systems offer numbers from 0 to 255. In order to solve most realistic problems, you need 16 or 32-bit numbers. Working with a 32-bit number requires a half-dozen instructions to add two numbers and closer to 30+ instructions to multiply them and that's assuming no register pressure. If you only have a couple registers and an accumulator, you will have many more instruction moving data between registers and memory. 16-bit processors were an almost uniform upgrade here. You halve the number of raw math operations and the amount of MOVs decrease way more than that. 32-bit processors were a mild upgrade in comparison. If you needed 32-bit numbers, they were faster, but any time you could get away with 16-bit numbers, the other half of the ALU was just wasting power and die space. There was still a pretty large advantage to this change though. 64-bit processors have comparatively mild advantages. Unfortunately, increasing cache size (especially L1 cache) explodes the implementation complexity when you try to keep latency cycles down. Actually using a bunch of 64-bit numbers where 32-bit numbers would be sufficient halves the hit rate in cache. The actual gains from a fully 64-bit processor are consequently rather mild in most real-world applications. Most of the things people believe to be inherent advantages of 64-bit processors are actually x86 specific because it could double it's general purpose registers and ditch tons of legacy garbage when in 64-bit mode. The Pentium 4 for example could access 36-bits of memory despite being a 32-bit architecture. Meanwhile 64-bit x86 systems use 40-52 bits for their memory offsets rather than the full 64 and we're nowhere close to saturating that in any commonly-used system. 128-bit processors hit this wall hard. You can't use 128-bit numbers everywhere because the cache hit is too big (you could mitigate this by increasing addressable size from 8 bits to 16 or 32 bits, but that adds another set of problems). To get that full cache usage, you are still going to be using 32-bit integers wherever possible, but that means 3/4 of your pipeline goes to waste for most calculations. For the occasional times where 128-bit or bigint numbers are needed, it's better to have a 64-bit processor and dedicate that extra die area to other things than wider ALUs. There is a special case for very specific supercomputers running highly-detailed simulations requiring high-precision floating-point numbers, but even in those cases, I suspect a 64-bit system with a 128-bit floating-point co-processor is probably a good approach.
- fennecfoxen 5y ago
  > To get that full cache usage, you are still going to be using 32-bit integers wherever possible, but that means 3/4 of your pipeline goes to waste for most calculations. I thought we were in an era of Dark Silicon where we are power/thermal constrained and it’s great to have lots of different special purpose parts of the die as long as you can turn them off when you’re not using them.
  - hajile 5y ago
    Not even close. TSMC 10nm wafer $6,000 TSMC N7 wafer $9,000 TSMC N5 wafer $17,000 TSMC N3 wafer $20,000+ (exact pricing not public) Cutting core size and saving just a few mm^2 per chip means a few more chips per wafer which is critical to the bottom line given the radical price increases for each node. The only company this doesn't seem to be particularly true for is Apple. Apple has 2 large, high-performance cores, 4 relatively large efficiency cores (around A78 performance levels), a large GPU, and a whopping 32MB of SLC (system level cache). Meanwhile, Qualcomm's just released Snapdragon has ONE high-performance core (that is still smaller than Apple's performance cores) and 3 "medium" cores that are only a bit better than Apple's efficiency cores with a smaller GPU and a tiny 6MB SLC. Qualcomm has been so desperate to lower wafer/chip costs that they even moved to a terrible Samsung process. The only specialized units are HIGHLY specialized. They are fixed function (or near fixed function) video decoders or ML units and none of these actually add very much die area. GPUs have actually moved heavily toward general purpose computing (GP-GPU) specifically so they could reuse the die area for other non-graphics things.
ajuc 5y ago
> Of course, if somebody does an operating system that uses 128-bit addressing to address every byte in the world uniquely, and this takes over the world, it might be an impetus for 128-bitters :-) I'd like to see such OS.
- Animats 5y ago
  Symbolics tried that. Scary thought: how many bytes of RAM are there in the world? Probably more than 2^64. There are around 7 billion smartphones, and many have over 4GB of memory. That's over 2^64 right there. I'd like to have 128-bit atomic operations. If you're doing lock-free programming, hardware compare and swap on a structure that contains two pointers is useful. Then you can update lists atomically. In a 64-bit pointer machine, that takes 128 bit atomics. So far, not much hardware offers that.
  - Dylan16807 5y ago
    CMPXCHG16B has been standard for the vast majority of the time x86_64 has existed. If a CPU can run windows 8.1, then it has 128-bit atomic operations.
    - Animats 5y ago
      Rust does not have AtomicU128, for some reason. Need to look into that.
      - Dylan16807 5y ago
        https://internals.rust-lang.org/t/is-atomicu128-gone-for-good/15657 https://internals.rust-lang.org/t/is-atomicu128-gone-for-goo... https://ebarnard.github.io/2019-06-03-rust-smaller-trait-implementers-docs/core/sync/atomic/struct.AtomicU128.html https://ebarnard.github.io/2019-06-03-rust-smaller-trait-imp... https://github.com/rust-lang/rust/issues/32976 https://github.com/rust-lang/rust/issues/32976 A little confusing but it looks like you can get to it.
        5y ago
reincarnate0x14 5y ago
I'd be curious if anyone with understanding of memory transistor density has an idea of what sort of physical footprint and power requirements 128-bit addressable memory would look like, because I'm having a hard time picturing even a distributed, shared virtual memory space realistically exceeding 16 exabytes. It looks like the Fugaku ARM cluster with 158,000 nodes uses < 5 PiB for 40 MW. As an aside, man does reading something from @sgi.com on USENET take me back to a better time. Title should probably note [1995] [0] https://www.fujitsu.com/global/about/innovation/fugaku/specifications/ https://www.fujitsu.com/global/about/innovation/fugaku/speci... [1] Also https://arxiv.org/abs/quant-ph/9908043 https://arxiv.org/abs/quant-ph/9908043 "Ultimate physical limits to computation"
- Laremere 5y ago
  85.8 grams of DNA is enough to overflow 64bits. Not that we're anywhere close to being able to use or process information on those scales, but it at least storage on that scale doesn't outright violate the laws of physics/information theory.
  - littlestymaar 5y ago
    Storage needs don't affect pointer size though. To need bigger pointers, you must need an addressable space this big. That said, how to you make your calculation about DNA's information density? Without counting water, I've arrived at 720g for one “mole of bits”, which is a bunch of orders of magnitude higher than your figure: 2 amino-acids (262g/mole) [1], 2 Deoxyribose (135g/mole each) and 2 Phosphate (94g/mole each). [1]: and it's the same value be it A-T or G-C, that's interesting
    - spqr0a1 5y ago
      The information density is even higher than that (double), because one pair of bases has 4 possible states. The pair A-T is symbolically distinct from T-A, for example.
  - jamal-kumar 5y ago
    Not anywhere close? I've heard things about using DNA itself for the memory storage format which is super interesting. I guess it just has bandwidth limitations and is more for long term storage? [1] [1] https://www.scientificamerican.com/article/dna-the-ultimate-data-storage-solution/ https://www.scientificamerican.com/article/dna-the-ultimate-...
PaulHoule 5y ago
They had 128-bit computing in the 1970s if you count this sort of thing https://en.wikipedia.org/wiki/IBM_AS/400 https://en.wikipedia.org/wiki/IBM_AS/400 which used 128-bit unique identifiers for objects that could be in RAM or kept persistently in storage which was transparently managed by the OS, language runtime, etc.
- guerrilla 5y ago
  Some VAXen could also do 128-bit ALU operations too. I thought DEC had a full 128-bit CPU but I've never been able to find it... I probably just remember wrong.
  - PaulHoule 5y ago
    Of the various claims that something is X-bit I think the most credible is the size of the address space as opposed to the size of the ALU. A 128-bit address space is large enough (UUIDs) that you can name 2^64 objects (about as many iron atoms are as in an iron filing) randomly. That lets you take two separate systems and then merge them into one system with no name conflicts, a handy property for 'distributed systems'
    - kens 5y ago
      That would yield a mess, historically. For instance, the 32-bit IBM System/360 had a 24-bit address space. The 16-bit Intel 8086 had a 20-bit address space. The 4-bit Intel 4004 had a 12-bit address space. It doesn't make sense to call the 4004 a 12-bit processor and the System/360 a 24-bit processor.
    - duskwuff 5y ago
      > Of the various claims that something is X-bit I think the most credible is the size of the address space as opposed to the size of the ALU. But that would make the MOS 6502 a 16-bit processor, and the Intel 8088 20-bit (!) -- neither of which are consistent with common usage. (They're generally considered 8-bit and 16-bit, respectively.) I'd stick with "the size of a general-purpose register". That neatly excludes special-purpose registers from consideration, like 256/512-bit AVX registers in x86_64 systems and the 16-bit program counter in most 8-bit CPUs.
CodeWriter23 5y ago
> Of course, a 32-bit physical addressing machine can get around this with extra external-mapping registers When I came up in this industry, we referred to that as "Bank Switching"
pella 5y ago
https://en.wikipedia.org/wiki/128-bit_computing https://en.wikipedia.org/wiki/128-bit_computing "The RISC-V ISA specification from 2016 includes a reservation for a 128-bit version of the architecture, but the details remain undefined intentionally, because there is yet so little practical experience with such large memory systems"
- monocasa 5y ago
  Fabrice Bellard's TinyEmu actually implements enough of the RV128 spec to run code. So the spec is defined well enough to implement, I just think they don't want to paint themselves into a corner before there's an actual need. I was going to link to his site, but it seems he needs to update his certs.
  - bejelentkezni 5y ago
    https://bellard.org/tinyemu https://bellard.org/tinyemu Looks fine to me.
don-code 5y ago
We've definitely found use cases for higher-width buses, but seemingly less so for operating on 128-bit integers and pointers, like this article is describing. Take for instance VLIW architectures (Itanium used 64-bit registers but 128-bit instructions; you packed two ops into an instruction), vector extensions (AVX-512 uses 512-bit registers to hold multiple integers from 16 to 64 bits long) - we've long known these to be viable strategies for long word lengths in a CPU. But the ability to operate on a 128-bit integer directly doesn't seem to have a use case yet.
- Dylan16807 5y ago
  > (Itanium used 64-bit registers but 128-bit instructions; you packed two ops into an instruction) Three ops.
Someone 5y ago
FTA: Thus, a CPU family intended to address higher-end systems will typically add 2 more bits of physical address* every 3 years.* For me, that loosely follows from the “CPU speed doubles every 18 months” variant on Moore’s law. Having extra address space doesn’t help you if your CPU is too slow to fill it (1), so if your CPU gets twice as fast in 18 months, the amount of memory you can effectively address doubles. That would only change if we had non-volatile RAM. (1) yes, you could have a hash table with effectively zero chance of collisions if you were to use petabytes to store a few hundred items, and that would be useful even with a 1MHz CPU, but if you can spend that money on memory, you’re better of spending it on better CPU. So, that only would help if CPU evolution stopped and memory got drastically cheaper, both in $ and in power usage.
dang 5y ago
One small thread from the past: The prospects for 128 bit processors (1995) - https://news.ycombinator.com/item?id=6300063 https://news.ycombinator.com/item?id=6300063 - Aug 2013 (2 comments)
AdamH12113 5y ago
> For example, 36-bit physical addresses support 16GB memories ... and there already have been shipped single-rack microprocessor boxes with 16GB using just 16Mb DRAMs; there are of course, more in the 4GB-8GB range. Of course, a 32-bit physical addressing machine can get around this with extra external-mapping registers ... assuming one can ignore the moaning from the kernel programmers :-) I remember 16 megabytes of RAM being enough for a good gaming rig in 1995. Who was using gigabytes of RAM back then, and for what?
- hackcasual 5y ago
  Databases. Early high traffic internet services
  - colejohnson66 5y ago
    I’m other words: the same things that, today, are using a terabyte of RAM. Cloud computing and databases.
- monocasa 5y ago
  A couple years later, but in 1997 you could get one of these with 64GB of RAM. https://en.wikipedia.org/wiki/Sun_Enterprise#Enterprise_10000 https://en.wikipedia.org/wiki/Sun_Enterprise#Enterprise_1000...
- hnlmorg 5y ago
  Most likely databases. Pretty sure one of the Sun SPARC boxes I managed in 1995 and that had a few GBs of RAM. It also ran Informix, which I don't miss administrating.
- cesaref 5y ago
  I was building and speccing servers to run banking applications in the mid to late 90s and this was in the days when x86 was still rubbish (or perceived to be so), so it was big Solaris boxes (E10k etc) and beefy HP PA-RISC machines. Applications were written in C++ and COBOL, the DB was Oracle. CPUs were expensive, disks were slow (and expensive - this is when EMC ruled!), so memory was used to boost database performance, and you could put a lot of memory in these machines (and we did). There were often 3 machines at each site, 2 production (one hot standby) and 1 test/qa machine, usually the same chassis but with lower specs (less CPU, less memory, less disk etc). The production machines were often at two close sites (10k apart) with disk replication between them over fibre in the street. It was quite common back then for the clients to specify ridiculous projected growth, and require total resilience from the hardware. I'd spec up the machines, and they'd baulk at the price (and when you get an international bank to baulk at the price you can imagine we're talking multiple millions for hardware let alone support contracts etc). Then, following a slice of realism, they'd relax some of the requirements, and we'd get something for 1/4 of the money. Happy days, lots of travel though.
CalChris 5y ago
Even 64-bit systems today don’t address 64-bits of physical memory. The Arm A78 (a very random example) uses 40 bits of physical address space. The 128 bits would really be for arithmetic and bus widths.
- hajile 5y ago
  I don't know why you were downvoted. This is absolutely true and even the biggest x86 systems don't address more than 52-bits.
- astrange 5y ago
  Some 64-bit systems still choose to use 32-bit pointers because it saves so much memory on storing them.
EdSchouten 5y ago
Relatedly, I hope we ever get CPUs where pointers are bit addressed. I’m not suggesting they should allow byte level access at arbitrary alignments. I just mean that the smallest addressable unit is a bit. That would allow us to finally declare lists of booleans without needing any specialization or losing the possibility of referencing elements.
- esmIII 5y ago
  Some ARM cortex-m chips have a feature where you can address individual bits. https://developer.arm.com/documentation/ddi0439/b/Programmers-Model/Bit-banding https://developer.arm.com/documentation/ddi0439/b/Programmer...
- flohofwoe 5y ago
  This would also go nicely with arbitrary-width integers. Load/store instructions would just need to set aside a few opcode bits to define the number of bits to load or store, and pointers would just get 3 bits wider (basically left-shifted 3 bits, the upper bits in a 64-bit pointer are not used anyway). Pointer alignment could be the same as the bit width, e.g. 2-bit load/store could require 2-bit alignment, byte load/store byte alignment, and so forth...
- gumby 5y ago
  > Relatedly, I hope we ever get CPUs where pointers are bit addressed. You mean like the famous PDP-6 from 1964? The width of a byte was variable and could be as small as 1: http://pdp10.nocrew.org/docs/instruction-set/Byte.html http://pdp10.nocrew.org/docs/instruction-set/Byte.html (That's the PDP-10 instruction set documentation, but the PDP-6's instruction set was essentially identical). I never wrote PDP-6 code but wrote a lot of PDP-10 code and the byte instructions were used heavily. In particular characters were commonly six or seven bits wide so string manipulation used byte pointers and was quite fast, but they were also good for packed arrays of all sorts of sizes.
- cwp 5y ago
  Funny, I've had the opposite thought. If registers are a certain width, and all memory access is aligned anyway, why do we need pointers that can address arbitrary bytes? Heck, why are bytes even a thing in a 64-bit machine?
boznz 5y ago
It is a geniune method to speed up computation if you can put all the operands in the instruction (VLWI)
Findecanor 5y ago
We already have 128-bit processors, of sorts. They are called "vector units". ;) 128-bit integers could be useful for bignums. You can do bignum addition on existing SIMD units but you'd have to use tricks to carry the carry between elements and to the next vector — and that makes the resulting code slower than using GPRs.[1] But I think this would still fit better in vector units than making the integer registers larger: The hw trend (ARM SVE, RISC-V V) is to make vector-width implementation-defined, starting at 128 bits in the low-end. 128-bit floating point is apparently a thing in scientific computing (but I've not seen it myself). 128-bit "pointers" exist in CHERI [2], where they are tagged capabilities that also store the bounds and access rights (rwx) for accessing memory through the pointer. Addresses are still 64-bit though. GPRs are still 64-bit. The caps are stored in a new register file. If you want to scale up address spaces, I think the biggest issue is not the number of bits but rather how to coordinate the address space to avoid collisions. One approach would be to re-introduce segmentation (pointer ⇒ segment ID and offset within segment), but unlike earlier system make each segment have its own local namespace of segment IDs it knows and translate between those when transferring pointers between segments. This idea is used by e.g. the research OS "Twizzer" [3], which was made for a large address space using NVRAM as RAM. I think this "pointer swizzling" could also be helped by hardware support. [1] Integer Addition and Carryout, by Alexander Yee <http://numberworld.org/y-cruncher/internals/addition.html http://numberworld.org/y-cruncher/internals/addition.html> [2] Capability Hardware Enhanced RISC Instructions (CHERI)<https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/ https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/> [3] Twizzler: An Operating System for Next-Generation Memory Hierarchies <https://www.ssrc.ucsc.edu/pub/bittman-ssrctr-17-01.html https://www.ssrc.ucsc.edu/pub/bittman-ssrctr-17-01.html>
- jamal-kumar 5y ago
  Very interesting. Does this have any implications for more accurate precision when floating points are being worked with in big numbers by any chance?
- hajile 5y ago
  128-bit vectors can do some stuff with integers, but most don't work at all for 128-bit floats which are where 128-bit is most important in hardware (because the software fallback is SLOW). I believe IBM POWER chips have both IEEE 128 floats (I think they call them a long double) and IEEE 128-bit decimal too. Outside of some very precise simulations though, I doubt there is very much demand.
__d 5y ago
Great to see posts from John Mashey here. John wrote the UNIX PWB aka Mashey shell, and then had a long career at MIPS and SGI.
Teknoman117 5y ago
> Of course, some kinds of system designs burn physical memory addresses faster than you'd expect. In particular, suppose you build a system with multiple memory systems. A minimal/natural approach is to use the high-order bits of an address to select the memory to be accessed. We're already starting to see this regarding some of the up-and-coming technologies like CXL and Gen-Z (which have now been merged). The idea is for a fabric consisting of multiple machines to share a common address space, and with the transition from 48-bit virtual addresses to 56-bit virtual addresses happening soon-ish, there's not many bits in the address space left to leave a machine identifier in your 56-bit address.
sandworm101 5y ago
Non-volatile memory tech. Once storage and ram become one and the same then the need for addresses will explode.
bombcar 5y ago
Finally we can get to IPv6 and store an address in a register.
bArray 5y ago
I think 128 bit computers will come around eventually, despite it having been declared that 64 bit is "enough". Some pressures may come from: * Memory addressing - As the article suggests, addressing large amount of address space. Not just RAM, but disk control on the bit/byte level (something solid state drives may enable and new filesystems may take advantage of). There may also be applications where you want an etabyte disk as low-speed RAM. * Multi-byte processing - accelerating instructions like AVX have shown the power of processing multiple bytes at a time. One can imagine that wider registers would accelerate these processes and would allow for multi-byte processes to happen in parallel. * Gaming/simulation - We have seen quite a few examples where physics in games and simulations have broken down due to the inaccuracy of `double` for large values. I believe Minecraft physics for example used to become extremely unstable near the world border. * Hashing - With `int32_t` and large amounts of data, you will see a lot of collisions. `int64_t` is lesser, but still likely. `int128_t` is rarer but still possible. `int256_t` (`long long` on a 128 bit processor) would be highly unlikely. Being able to compare hashes in just a few clock cycles would be awesome. * Custom instructions - When programs can define a custom instruction to speed-up computation, 128 bits, or 16 bytes, could even be enough to contain the custom instruction and the payload. These are just things I've noticed. I imagine there are others too. The prediction of 2043 is still quite realistic, I wouldn't be surprised if we beat it. I was quite disappointed to see many Linux distros give up on 32 bit support because it was too much effort to support. It probably points towards some crappy code that is highly dependant on the platform.
- anamax 5y ago
  Multi-issue 64bit machines can compare 256 bits in "just a few cycles". (You do 2-4 compares per cycle and then combine the results.)
- zarzavat 5y ago
  In my opinion none of these are good reasons for 128-bit computers, as they can be easily addressed by using 2x64 bit integers and software arithmetic, or bigints. The issue with 32-bit specifically is that the maximum value (2 billion for a signed int) is really small and hits all kinds of practical limits: there are more humans than that! Whereas 2^63 is big enough for almost all practical purposes. There were costs to the 64 bit transition in terms of higher memory usage so I don’t think we’ll see another transition because the costs at 128-bit would outweigh the benefits.
DonHopkins 5y ago
https://news.ycombinator.com/item?id=26112698 https://news.ycombinator.com/item?id=26112698 >On Nov 8, 2018, I sent Bill Joy a birthday greeting: "Happy 17,179,869,184 MIPS Birthday, Bill"! (2 to the (year - 1984)) https://medium.com/@donhopkins/bill-joys-law-2-year-1984-million-instructions-per-second-3a8c92165cfe https://medium.com/@donhopkins/bill-joys-law-2-year-1984-mil...
pyuser583 5y ago
Notice the physical address they have listed is the current Google headquarters. The Googleplex is built on/around the old SGI campus.
phkamp 5y ago
At the time at least one 128 bit computer had been on the market for several years: The Rational R1000.
trasz 5y ago
CHERI/Morello pointers are 128 bit.

85 points by luu 5y ago

He failed to account for the "why" with previous architectures. 8-bit systems offer numbers from 0 to 255. In order to solve most realistic problems, you need 16 or 32-bit numbers. Working with a 32-bit number requires a half-dozen instructions to add two numbers and closer to 30+ instructions to multiply them and that's assuming no register pressure. If you only have a couple registers and an accumulator, you will have many more instruction moving data between registers and memory. 16-bit processors were an almost uniform upgrade here. You halve the number of raw math operations and the amount of MOVs decrease way more than that. 32-bit processors were a mild upgrade in comparison. If you needed 32-bit numbers, they were faster, but any time you could get away with 16-bit numbers, the other half of the ALU was just wasting power and die space. There was still a pretty large advantage to this change though. 64-bit processors have comparatively mild advantages. Unfortunately, increasing cache size (especially L1 cache) explodes the implementation complexity when you try to keep latency cycles down. Actually using a bunch of 64-bit numbers where 32-bit numbers would be sufficient halves the hit rate in cache. The actual gains from a fully 64-bit processor are consequently rather mild in most real-world applications. Most of the things people believe to be inherent advantages of 64-bit processors are actually x86 specific because it could double it's general purpose registers and ditch tons of legacy garbage when in 64-bit mode. The Pentium 4 for example could access 36-bits of memory despite being a 32-bit architecture. Meanwhile 64-bit x86 systems use 40-52 bits for their memory offsets rather than the full 64 and we're nowhere close to saturating that in any commonly-used system. 128-bit processors hit this wall hard. You can't use 128-bit numbers everywhere because the cache hit is too big (you could mitigate this by increasing addressable size from 8 bits to 16 or 32 bits, but that adds another set of problems). To get that full cache usage, you are still going to be using 32-bit integers wherever possible, but that means 3/4 of your pipeline goes to waste for most calculations. For the occasional times where 128-bit or bigint numbers are needed, it's better to have a 64-bit processor and dedicate that extra die area to other things than wider ALUs. There is a special case for very specific supercomputers running highly-detailed simulations requiring high-precision floating-point numbers, but even in those cases, I suspect a 64-bit system with a 128-bit floating-point co-processor is probably a good approach.

fennecfoxen 5y ago
> To get that full cache usage, you are still going to be using 32-bit integers wherever possible, but that means 3/4 of your pipeline goes to waste for most calculations. I thought we were in an era of Dark Silicon where we are power/thermal constrained and it’s great to have lots of different special purpose parts of the die as long as you can turn them off when you’re not using them.
- hajile 5y ago
  Not even close. TSMC 10nm wafer $6,000 TSMC N7 wafer $9,000 TSMC N5 wafer $17,000 TSMC N3 wafer $20,000+ (exact pricing not public) Cutting core size and saving just a few mm^2 per chip means a few more chips per wafer which is critical to the bottom line given the radical price increases for each node. The only company this doesn't seem to be particularly true for is Apple. Apple has 2 large, high-performance cores, 4 relatively large efficiency cores (around A78 performance levels), a large GPU, and a whopping 32MB of SLC (system level cache). Meanwhile, Qualcomm's just released Snapdragon has ONE high-performance core (that is still smaller than Apple's performance cores) and 3 "medium" cores that are only a bit better than Apple's efficiency cores with a smaller GPU and a tiny 6MB SLC. Qualcomm has been so desperate to lower wafer/chip costs that they even moved to a terrible Samsung process. The only specialized units are HIGHLY specialized. They are fixed function (or near fixed function) video decoders or ML units and none of these actually add very much die area. GPUs have actually moved heavily toward general purpose computing (GP-GPU) specifically so they could reuse the die area for other non-graphics things.

ajuc 5y ago

> Of course, if somebody does an operating system that uses 128-bit addressing to address every byte in the world uniquely, and this takes over the world, it might be an impetus for 128-bitters :-) I'd like to see such OS.

Animats 5y ago
Symbolics tried that. Scary thought: how many bytes of RAM are there in the world? Probably more than 2^64. There are around 7 billion smartphones, and many have over 4GB of memory. That's over 2^64 right there. I'd like to have 128-bit atomic operations. If you're doing lock-free programming, hardware compare and swap on a structure that contains two pointers is useful. Then you can update lists atomically. In a 64-bit pointer machine, that takes 128 bit atomics. So far, not much hardware offers that.
- Dylan16807 5y ago
  CMPXCHG16B has been standard for the vast majority of the time x86_64 has existed. If a CPU can run windows 8.1, then it has 128-bit atomic operations.
  - Animats 5y ago
    Rust does not have AtomicU128, for some reason. Need to look into that.
    - Dylan16807 5y ago
      https://internals.rust-lang.org/t/is-atomicu128-gone-for-good/15657 https://internals.rust-lang.org/t/is-atomicu128-gone-for-goo... https://ebarnard.github.io/2019-06-03-rust-smaller-trait-implementers-docs/core/sync/atomic/struct.AtomicU128.html https://ebarnard.github.io/2019-06-03-rust-smaller-trait-imp... https://github.com/rust-lang/rust/issues/32976 https://github.com/rust-lang/rust/issues/32976 A little confusing but it looks like you can get to it.
      - 5y ago

reincarnate0x14 5y ago

I'd be curious if anyone with understanding of memory transistor density has an idea of what sort of physical footprint and power requirements 128-bit addressable memory would look like, because I'm having a hard time picturing even a distributed, shared virtual memory space realistically exceeding 16 exabytes. It looks like the Fugaku ARM cluster with 158,000 nodes uses < 5 PiB for 40 MW. As an aside, man does reading something from @sgi.com on USENET take me back to a better time. Title should probably note [1995] [0] https://www.fujitsu.com/global/about/innovation/fugaku/specifications/ https://www.fujitsu.com/global/about/innovation/fugaku/speci... [1] Also https://arxiv.org/abs/quant-ph/9908043 https://arxiv.org/abs/quant-ph/9908043 "Ultimate physical limits to computation"

Laremere 5y ago
85.8 grams of DNA is enough to overflow 64bits. Not that we're anywhere close to being able to use or process information on those scales, but it at least storage on that scale doesn't outright violate the laws of physics/information theory.
- littlestymaar 5y ago
  Storage needs don't affect pointer size though. To need bigger pointers, you must need an addressable space this big. That said, how to you make your calculation about DNA's information density? Without counting water, I've arrived at 720g for one “mole of bits”, which is a bunch of orders of magnitude higher than your figure: 2 amino-acids (262g/mole) [1], 2 Deoxyribose (135g/mole each) and 2 Phosphate (94g/mole each). [1]: and it's the same value be it A-T or G-C, that's interesting
  - spqr0a1 5y ago
    The information density is even higher than that (double), because one pair of bases has 4 possible states. The pair A-T is symbolically distinct from T-A, for example.
- jamal-kumar 5y ago
  Not anywhere close? I've heard things about using DNA itself for the memory storage format which is super interesting. I guess it just has bandwidth limitations and is more for long term storage? [1] [1] https://www.scientificamerican.com/article/dna-the-ultimate-data-storage-solution/ https://www.scientificamerican.com/article/dna-the-ultimate-...

PaulHoule 5y ago

They had 128-bit computing in the 1970s if you count this sort of thing https://en.wikipedia.org/wiki/IBM_AS/400 https://en.wikipedia.org/wiki/IBM_AS/400 which used 128-bit unique identifiers for objects that could be in RAM or kept persistently in storage which was transparently managed by the OS, language runtime, etc.

guerrilla 5y ago
Some VAXen could also do 128-bit ALU operations too. I thought DEC had a full 128-bit CPU but I've never been able to find it... I probably just remember wrong.
- PaulHoule 5y ago
  Of the various claims that something is X-bit I think the most credible is the size of the address space as opposed to the size of the ALU. A 128-bit address space is large enough (UUIDs) that you can name 2^64 objects (about as many iron atoms are as in an iron filing) randomly. That lets you take two separate systems and then merge them into one system with no name conflicts, a handy property for 'distributed systems'
  - kens 5y ago
    That would yield a mess, historically. For instance, the 32-bit IBM System/360 had a 24-bit address space. The 16-bit Intel 8086 had a 20-bit address space. The 4-bit Intel 4004 had a 12-bit address space. It doesn't make sense to call the 4004 a 12-bit processor and the System/360 a 24-bit processor.
  - duskwuff 5y ago
    > Of the various claims that something is X-bit I think the most credible is the size of the address space as opposed to the size of the ALU. But that would make the MOS 6502 a 16-bit processor, and the Intel 8088 20-bit (!) -- neither of which are consistent with common usage. (They're generally considered 8-bit and 16-bit, respectively.) I'd stick with "the size of a general-purpose register". That neatly excludes special-purpose registers from consideration, like 256/512-bit AVX registers in x86_64 systems and the 16-bit program counter in most 8-bit CPUs.

CodeWriter23 5y ago

> Of course, a 32-bit physical addressing machine can get around this with extra external-mapping registers When I came up in this industry, we referred to that as "Bank Switching"

pella 5y ago

https://en.wikipedia.org/wiki/128-bit_computing https://en.wikipedia.org/wiki/128-bit_computing "The RISC-V ISA specification from 2016 includes a reservation for a 128-bit version of the architecture, but the details remain undefined intentionally, because there is yet so little practical experience with such large memory systems"

monocasa 5y ago
Fabrice Bellard's TinyEmu actually implements enough of the RV128 spec to run code. So the spec is defined well enough to implement, I just think they don't want to paint themselves into a corner before there's an actual need. I was going to link to his site, but it seems he needs to update his certs.
- bejelentkezni 5y ago
  https://bellard.org/tinyemu https://bellard.org/tinyemu Looks fine to me.

don-code 5y ago

We've definitely found use cases for higher-width buses, but seemingly less so for operating on 128-bit integers and pointers, like this article is describing. Take for instance VLIW architectures (Itanium used 64-bit registers but 128-bit instructions; you packed two ops into an instruction), vector extensions (AVX-512 uses 512-bit registers to hold multiple integers from 16 to 64 bits long) - we've long known these to be viable strategies for long word lengths in a CPU. But the ability to operate on a 128-bit integer directly doesn't seem to have a use case yet.

Dylan16807 5y ago
> (Itanium used 64-bit registers but 128-bit instructions; you packed two ops into an instruction) Three ops.

Someone 5y ago

FTA: Thus, a CPU family intended to address higher-end systems will typically add 2 more bits of physical address* every 3 years.* For me, that loosely follows from the “CPU speed doubles every 18 months” variant on Moore’s law. Having extra address space doesn’t help you if your CPU is too slow to fill it (1), so if your CPU gets twice as fast in 18 months, the amount of memory you can effectively address doubles. That would only change if we had non-volatile RAM. (1) yes, you could have a hash table with effectively zero chance of collisions if you were to use petabytes to store a few hundred items, and that would be useful even with a 1MHz CPU, but if you can spend that money on memory, you’re better of spending it on better CPU. So, that only would help if CPU evolution stopped and memory got drastically cheaper, both in $ and in power usage.

dang 5y ago

One small thread from the past: The prospects for 128 bit processors (1995) - https://news.ycombinator.com/item?id=6300063 https://news.ycombinator.com/item?id=6300063 - Aug 2013 (2 comments)

AdamH12113 5y ago

> For example, 36-bit physical addresses support 16GB memories ... and there already have been shipped single-rack microprocessor boxes with 16GB using just 16Mb DRAMs; there are of course, more in the 4GB-8GB range. Of course, a 32-bit physical addressing machine can get around this with extra external-mapping registers ... assuming one can ignore the moaning from the kernel programmers :-) I remember 16 megabytes of RAM being enough for a good gaming rig in 1995. Who was using gigabytes of RAM back then, and for what?

hackcasual 5y ago
Databases. Early high traffic internet services
- colejohnson66 5y ago
  I’m other words: the same things that, today, are using a terabyte of RAM. Cloud computing and databases.
monocasa 5y ago
A couple years later, but in 1997 you could get one of these with 64GB of RAM. https://en.wikipedia.org/wiki/Sun_Enterprise#Enterprise_10000 https://en.wikipedia.org/wiki/Sun_Enterprise#Enterprise_1000...
hnlmorg 5y ago
Most likely databases. Pretty sure one of the Sun SPARC boxes I managed in 1995 and that had a few GBs of RAM. It also ran Informix, which I don't miss administrating.
cesaref 5y ago
I was building and speccing servers to run banking applications in the mid to late 90s and this was in the days when x86 was still rubbish (or perceived to be so), so it was big Solaris boxes (E10k etc) and beefy HP PA-RISC machines. Applications were written in C++ and COBOL, the DB was Oracle. CPUs were expensive, disks were slow (and expensive - this is when EMC ruled!), so memory was used to boost database performance, and you could put a lot of memory in these machines (and we did). There were often 3 machines at each site, 2 production (one hot standby) and 1 test/qa machine, usually the same chassis but with lower specs (less CPU, less memory, less disk etc). The production machines were often at two close sites (10k apart) with disk replication between them over fibre in the street. It was quite common back then for the clients to specify ridiculous projected growth, and require total resilience from the hardware. I'd spec up the machines, and they'd baulk at the price (and when you get an international bank to baulk at the price you can imagine we're talking multiple millions for hardware let alone support contracts etc). Then, following a slice of realism, they'd relax some of the requirements, and we'd get something for 1/4 of the money. Happy days, lots of travel though.

CalChris 5y ago

Even 64-bit systems today don’t address 64-bits of physical memory. The Arm A78 (a very random example) uses 40 bits of physical address space. The 128 bits would really be for arithmetic and bus widths.

hajile 5y ago
I don't know why you were downvoted. This is absolutely true and even the biggest x86 systems don't address more than 52-bits.
astrange 5y ago
Some 64-bit systems still choose to use 32-bit pointers because it saves so much memory on storing them.

EdSchouten 5y ago

Relatedly, I hope we ever get CPUs where pointers are bit addressed. I’m not suggesting they should allow byte level access at arbitrary alignments. I just mean that the smallest addressable unit is a bit. That would allow us to finally declare lists of booleans without needing any specialization or losing the possibility of referencing elements.

esmIII 5y ago
Some ARM cortex-m chips have a feature where you can address individual bits. https://developer.arm.com/documentation/ddi0439/b/Programmers-Model/Bit-banding https://developer.arm.com/documentation/ddi0439/b/Programmer...
flohofwoe 5y ago
This would also go nicely with arbitrary-width integers. Load/store instructions would just need to set aside a few opcode bits to define the number of bits to load or store, and pointers would just get 3 bits wider (basically left-shifted 3 bits, the upper bits in a 64-bit pointer are not used anyway). Pointer alignment could be the same as the bit width, e.g. 2-bit load/store could require 2-bit alignment, byte load/store byte alignment, and so forth...
gumby 5y ago
> Relatedly, I hope we ever get CPUs where pointers are bit addressed. You mean like the famous PDP-6 from 1964? The width of a byte was variable and could be as small as 1: http://pdp10.nocrew.org/docs/instruction-set/Byte.html http://pdp10.nocrew.org/docs/instruction-set/Byte.html (That's the PDP-10 instruction set documentation, but the PDP-6's instruction set was essentially identical). I never wrote PDP-6 code but wrote a lot of PDP-10 code and the byte instructions were used heavily. In particular characters were commonly six or seven bits wide so string manipulation used byte pointers and was quite fast, but they were also good for packed arrays of all sorts of sizes.
cwp 5y ago
Funny, I've had the opposite thought. If registers are a certain width, and all memory access is aligned anyway, why do we need pointers that can address arbitrary bytes? Heck, why are bytes even a thing in a 64-bit machine?

boznz 5y ago

It is a geniune method to speed up computation if you can put all the operands in the instruction (VLWI)

Findecanor 5y ago

We already have 128-bit processors, of sorts. They are called "vector units". ;) 128-bit integers could be useful for bignums. You can do bignum addition on existing SIMD units but you'd have to use tricks to carry the carry between elements and to the next vector — and that makes the resulting code slower than using GPRs.[1] But I think this would still fit better in vector units than making the integer registers larger: The hw trend (ARM SVE, RISC-V V) is to make vector-width implementation-defined, starting at 128 bits in the low-end. 128-bit floating point is apparently a thing in scientific computing (but I've not seen it myself). 128-bit "pointers" exist in CHERI [2], where they are tagged capabilities that also store the bounds and access rights (rwx) for accessing memory through the pointer. Addresses are still 64-bit though. GPRs are still 64-bit. The caps are stored in a new register file. If you want to scale up address spaces, I think the biggest issue is not the number of bits but rather how to coordinate the address space to avoid collisions. One approach would be to re-introduce segmentation (pointer ⇒ segment ID and offset within segment), but unlike earlier system make each segment have its own local namespace of segment IDs it knows and translate between those when transferring pointers between segments. This idea is used by e.g. the research OS "Twizzer" [3], which was made for a large address space using NVRAM as RAM. I think this "pointer swizzling" could also be helped by hardware support. [1] Integer Addition and Carryout, by Alexander Yee <http://numberworld.org/y-cruncher/internals/addition.html http://numberworld.org/y-cruncher/internals/addition.html> [2] Capability Hardware Enhanced RISC Instructions (CHERI)<https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/ https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/> [3] Twizzler: An Operating System for Next-Generation Memory Hierarchies <https://www.ssrc.ucsc.edu/pub/bittman-ssrctr-17-01.html https://www.ssrc.ucsc.edu/pub/bittman-ssrctr-17-01.html>

jamal-kumar 5y ago
Very interesting. Does this have any implications for more accurate precision when floating points are being worked with in big numbers by any chance?
hajile 5y ago
128-bit vectors can do some stuff with integers, but most don't work at all for 128-bit floats which are where 128-bit is most important in hardware (because the software fallback is SLOW). I believe IBM POWER chips have both IEEE 128 floats (I think they call them a long double) and IEEE 128-bit decimal too. Outside of some very precise simulations though, I doubt there is very much demand.

__d 5y ago

Great to see posts from John Mashey here. John wrote the UNIX PWB aka Mashey shell, and then had a long career at MIPS and SGI.

Teknoman117 5y ago

> Of course, some kinds of system designs burn physical memory addresses faster than you'd expect. In particular, suppose you build a system with multiple memory systems. A minimal/natural approach is to use the high-order bits of an address to select the memory to be accessed. We're already starting to see this regarding some of the up-and-coming technologies like CXL and Gen-Z (which have now been merged). The idea is for a fabric consisting of multiple machines to share a common address space, and with the transition from 48-bit virtual addresses to 56-bit virtual addresses happening soon-ish, there's not many bits in the address space left to leave a machine identifier in your 56-bit address.

sandworm101 5y ago

Non-volatile memory tech. Once storage and ram become one and the same then the need for addresses will explode.

bombcar 5y ago

Finally we can get to IPv6 and store an address in a register.

bArray 5y ago

I think 128 bit computers will come around eventually, despite it having been declared that 64 bit is "enough". Some pressures may come from: * Memory addressing - As the article suggests, addressing large amount of address space. Not just RAM, but disk control on the bit/byte level (something solid state drives may enable and new filesystems may take advantage of). There may also be applications where you want an etabyte disk as low-speed RAM. * Multi-byte processing - accelerating instructions like AVX have shown the power of processing multiple bytes at a time. One can imagine that wider registers would accelerate these processes and would allow for multi-byte processes to happen in parallel. * Gaming/simulation - We have seen quite a few examples where physics in games and simulations have broken down due to the inaccuracy of `double` for large values. I believe Minecraft physics for example used to become extremely unstable near the world border. * Hashing - With `int32_t` and large amounts of data, you will see a lot of collisions. `int64_t` is lesser, but still likely. `int128_t` is rarer but still possible. `int256_t` (`long long` on a 128 bit processor) would be highly unlikely. Being able to compare hashes in just a few clock cycles would be awesome. * Custom instructions - When programs can define a custom instruction to speed-up computation, 128 bits, or 16 bytes, could even be enough to contain the custom instruction and the payload. These are just things I've noticed. I imagine there are others too. The prediction of 2043 is still quite realistic, I wouldn't be surprised if we beat it. I was quite disappointed to see many Linux distros give up on 32 bit support because it was too much effort to support. It probably points towards some crappy code that is highly dependant on the platform.

anamax 5y ago
Multi-issue 64bit machines can compare 256 bits in "just a few cycles". (You do 2-4 compares per cycle and then combine the results.)
zarzavat 5y ago
In my opinion none of these are good reasons for 128-bit computers, as they can be easily addressed by using 2x64 bit integers and software arithmetic, or bigints. The issue with 32-bit specifically is that the maximum value (2 billion for a signed int) is really small and hits all kinds of practical limits: there are more humans than that! Whereas 2^63 is big enough for almost all practical purposes. There were costs to the 64 bit transition in terms of higher memory usage so I don’t think we’ll see another transition because the costs at 128-bit would outweigh the benefits.

DonHopkins 5y ago

https://news.ycombinator.com/item?id=26112698 https://news.ycombinator.com/item?id=26112698 >On Nov 8, 2018, I sent Bill Joy a birthday greeting: "Happy 17,179,869,184 MIPS Birthday, Bill"! (2 to the (year - 1984)) https://medium.com/@donhopkins/bill-joys-law-2-year-1984-million-instructions-per-second-3a8c92165cfe https://medium.com/@donhopkins/bill-joys-law-2-year-1984-mil...

pyuser583 5y ago

Notice the physical address they have listed is the current Google headquarters. The Googleplex is built on/around the old SGI campus.

phkamp 5y ago

At the time at least one 128 bit computer had been on the market for several years: The Rational R1000.

trasz 5y ago

CHERI/Morello pointers are 128 bit.