4 ms·

In the era of "home computers", the serial connections were usually for low-speed peripherals like modems, and a you had a parallel bus for things like hard dri

by hakfoo 6y ago

In the era of "home computers", the serial connections were usually for low-speed peripherals like modems, and a you had a parallel bus for things like hard drives. This seemed to carry well into the modern PC/Mac era. Then in rapid succession starting in the early 2000s, we saw USB and SATA/SAS replace most IDE and SCSI use cases, and PCI Express replace PCI and AGP, along with Rambus taking a swing at conventional SDRAM. Was there some fundamental breakthrough that made it possible to do high-speed serial busses, or was it that the costs of managing them finally undercut the hassles of signaling difficulties on a parallel bus? The other thing I wonder about is what happened to SRAM? I know it always used a price concern-- "64K of DRAM is $10, and 64K of SRAM is $50", but given that DRAM is so cheap now, I'm surprised there's no SRAM product for enthusiasts/cost-no-object performance seekers. They'll already pay 5-10x the price of commodity DDR4 for the highest-binned overclockable stuff.

freeone3000 6y ago
Intel sells Optane, which is SRAM for the modern era. It's kind of tricky to get anything out of it (edit: besides using it as a really fast disk, which is what it's currently treated like) without bespoke software, as it's neither disk (byte-addressable!) nor RAM (persistent!). Linux kernel support is experimental and ongoing. It's coming, but it's not here yet. It's also $3000 per TB, triple the price of DDR4, and not much faster.
- selimthegrim 6y ago
  https://www.tomshardware.com/news/intel-kills-off-all-optane-only-ssds-for-consumers-no-replacements-planned https://www.tomshardware.com/news/intel-kills-off-all-optane...
  - freeone3000 6y ago
    They're killing the "consumer" line, not the product entirely. That same announcement unveils the P5800X, which hooks up to a PCI-e x4 slot just the same. There are resellers available for single-unit quantities.
- hnuser123456 6y ago
  It's about $1200 per TB for the now-discontinued consumer models. The real benefit vs. NAND is that it doesn't care about read/write mix and has max IOPS with any mix with consistently very, very low latency.
toast0 6y ago
> Was there some fundamental breakthrough that made it possible to do high-speed serial busses, or was it that the costs of managing them finally undercut the hassles of signaling difficulties on a parallel bus? As I understand it, the switch to serial and point to point are because of three things. a) pin/wire count reduction --- you can't have a small chip that uses a pc parallel port, cause it needs a lot of leads, etc. b) bus termination gets weird with potentially variable reflection and distortion at high speed c) clock skew is hard to manage on wide busses at high speed. You gotta make sure all the leads are the same length and/or add compensation logic, and it's a lot harder than a serial connection which might have a clock right next to the data line, or might be self-clocking, so the data line includes the clock signal and it can't be out of sync. About the only parallel bus left in modern computers is RAM. Rambus was a disaster, and probably soured the industry on a serial ram interface for some time. Note though that a RAM channel now usually has a max of two slots, but boards with only one slot per channel are common too; 4 slots per channel was common many years ago, when speeds were slower.
- jleahy 6y ago
  It's really about b/c rather than a, and it's not really about 'serial', it's about having differential signaling, an embedded clock (eg. 8B10B coding) and having multiple lanes that tolerate a length mismatch (and realigning them dynamically using some kind of marker in the data). You can have any one of those three things and get some of the benefits. Note that one thing that isn't special is having 1 lane, and in fact few 'serial' protocols (eg. PCIe) actually use 1 lane. If you think about PCIe we moved from a 32-bit bus (32 wires bidirectional) to x16 slots with 64 wires in total (16 lanes, 2 directions each, +ve/-ve for each). Maybe you consider AGP as the true precursor of x16 PCIE, but that was also 32 I/O lanes. So there wasn't a wire saving. The truth is if you want very high data rates over medium or long distances then you have to go with 'serial' (the three points above) and you can get anywhere between 2.5Gbps and 54Gbps per differential pair depending on the technology. If you are willing to have shorter distances then you can get up to something like 6.4Gbps per pin and it'll use less power (vastly less power if the data rate goes down to <1Gbps per pin), but you won't be able to reach as far. Technologies like HBM use single-ended (non-differential) signaling and make up for this with huge amounts of very short wires (up to a thousand or so). Given the constraints of the technology if I was designing RAM right now with no legacy baggage I'd chose single-ended signaling, and if I was designing a high-speed I/O slot I would choose differential signaling. So it's all quite sensible and rational.
ohazi 6y ago
Amusingly, things sort of went the other way in microcontroller land. You had SPI, which has a clock and data pin. That got faster and faster. Then there was QSPI (quad SPI) with four data pins, so you could shovel more data per clock, e.g. when loading an initial FPGA configuration, or when a fast CPU wanted to XIP from external flash. Now apparently there's Octal SPI? Which is... arguably a parallel bus!
- jleahy 6y ago
  Sure, but SPI is basically a parallel bus that's 1-bit wide. It has all the hallmarks (separate clock and single-ended signaling), so adding extra bits was an easy thing.
namibj 6y ago
Well, high speed SerDes transceiver only became "worth it" fairly recently (around when PCIe/SATA/SAS landed). I think this is due to switching speeds for the required SerDes logic only becoming fast enough (compared to parallel line drivers and the frequencies you can run through FR4). DRAM consumes less power than SRAM. Far less. My understanding is, that simply changing the parameters of DDR4 towards lower latency would yield better results.
Accujack 6y ago
>I'm surprised there's no SRAM product for enthusiasts/cost-no-object performance seekers. I suspect it's because the trade-off for the increased performance would be more than just the price, and that's not even considering the CPU changes and motherboard design necessary to implement SRAM main memory - no CPU maker is going to go to the expense of a separate production line just for enthusiast CPUs that support SRAM core memory. SRAM density is significantly lower (cells/mm^2) then DRAM, because a DRAM cell can by implemented as a single capacitor, where a SRAM cell is six MOSFETs. So very generally speaking the amount of SRAM you can pack into a chip is about 1/6 (or fewer) the amount of DRAM cells in the same space. Given that modern CPUs typically have only four DIMM slots populated in pairs and you can't add more without changing the memory interface, that means you've cut the total amount of memory possible in a system by about 80%, so max 6.4GB instead of 32GB. So.. a memory type for "enthusiasts" that costs e.g. 5x as much as DIMMS, only allows you 20% of the memory size, and also needs a motherboard and CPU design that's incompatible with DIMM based motherboards. Not attractive to too many people. If you want an easier way to make an enthusiast product with the performance benefits of SRAM, just put the memory on die on a multi-chip module so it's a very short distance from the processor cores (like Apple's M1). Short distance equals good signal integrity, low latency, and on die means simpler motherboard design. The trade off is that you can't add memory later, you have to buy a whole new chip.

by hakfoo 6y ago

freeone3000 6y ago

Intel sells Optane, which is SRAM for the modern era. It's kind of tricky to get anything out of it (edit: besides using it as a really fast disk, which is what it's currently treated like) without bespoke software, as it's neither disk (byte-addressable!) nor RAM (persistent!). Linux kernel support is experimental and ongoing. It's coming, but it's not here yet. It's also $3000 per TB, triple the price of DDR4, and not much faster.

selimthegrim 6y ago
https://www.tomshardware.com/news/intel-kills-off-all-optane-only-ssds-for-consumers-no-replacements-planned https://www.tomshardware.com/news/intel-kills-off-all-optane...
- freeone3000 6y ago
  They're killing the "consumer" line, not the product entirely. That same announcement unveils the P5800X, which hooks up to a PCI-e x4 slot just the same. There are resellers available for single-unit quantities.
hnuser123456 6y ago
It's about $1200 per TB for the now-discontinued consumer models. The real benefit vs. NAND is that it doesn't care about read/write mix and has max IOPS with any mix with consistently very, very low latency.

toast0 6y ago

> Was there some fundamental breakthrough that made it possible to do high-speed serial busses, or was it that the costs of managing them finally undercut the hassles of signaling difficulties on a parallel bus? As I understand it, the switch to serial and point to point are because of three things. a) pin/wire count reduction --- you can't have a small chip that uses a pc parallel port, cause it needs a lot of leads, etc. b) bus termination gets weird with potentially variable reflection and distortion at high speed c) clock skew is hard to manage on wide busses at high speed. You gotta make sure all the leads are the same length and/or add compensation logic, and it's a lot harder than a serial connection which might have a clock right next to the data line, or might be self-clocking, so the data line includes the clock signal and it can't be out of sync. About the only parallel bus left in modern computers is RAM. Rambus was a disaster, and probably soured the industry on a serial ram interface for some time. Note though that a RAM channel now usually has a max of two slots, but boards with only one slot per channel are common too; 4 slots per channel was common many years ago, when speeds were slower.

jleahy 6y ago
It's really about b/c rather than a, and it's not really about 'serial', it's about having differential signaling, an embedded clock (eg. 8B10B coding) and having multiple lanes that tolerate a length mismatch (and realigning them dynamically using some kind of marker in the data). You can have any one of those three things and get some of the benefits. Note that one thing that isn't special is having 1 lane, and in fact few 'serial' protocols (eg. PCIe) actually use 1 lane. If you think about PCIe we moved from a 32-bit bus (32 wires bidirectional) to x16 slots with 64 wires in total (16 lanes, 2 directions each, +ve/-ve for each). Maybe you consider AGP as the true precursor of x16 PCIE, but that was also 32 I/O lanes. So there wasn't a wire saving. The truth is if you want very high data rates over medium or long distances then you have to go with 'serial' (the three points above) and you can get anywhere between 2.5Gbps and 54Gbps per differential pair depending on the technology. If you are willing to have shorter distances then you can get up to something like 6.4Gbps per pin and it'll use less power (vastly less power if the data rate goes down to <1Gbps per pin), but you won't be able to reach as far. Technologies like HBM use single-ended (non-differential) signaling and make up for this with huge amounts of very short wires (up to a thousand or so). Given the constraints of the technology if I was designing RAM right now with no legacy baggage I'd chose single-ended signaling, and if I was designing a high-speed I/O slot I would choose differential signaling. So it's all quite sensible and rational.

ohazi 6y ago

Amusingly, things sort of went the other way in microcontroller land. You had SPI, which has a clock and data pin. That got faster and faster. Then there was QSPI (quad SPI) with four data pins, so you could shovel more data per clock, e.g. when loading an initial FPGA configuration, or when a fast CPU wanted to XIP from external flash. Now apparently there's Octal SPI? Which is... arguably a parallel bus!

jleahy 6y ago
Sure, but SPI is basically a parallel bus that's 1-bit wide. It has all the hallmarks (separate clock and single-ended signaling), so adding extra bits was an easy thing.

namibj 6y ago

Well, high speed SerDes transceiver only became "worth it" fairly recently (around when PCIe/SATA/SAS landed). I think this is due to switching speeds for the required SerDes logic only becoming fast enough (compared to parallel line drivers and the frequencies you can run through FR4). DRAM consumes less power than SRAM. Far less. My understanding is, that simply changing the parameters of DDR4 towards lower latency would yield better results.

Accujack 6y ago

>I'm surprised there's no SRAM product for enthusiasts/cost-no-object performance seekers. I suspect it's because the trade-off for the increased performance would be more than just the price, and that's not even considering the CPU changes and motherboard design necessary to implement SRAM main memory - no CPU maker is going to go to the expense of a separate production line just for enthusiast CPUs that support SRAM core memory. SRAM density is significantly lower (cells/mm^2) then DRAM, because a DRAM cell can by implemented as a single capacitor, where a SRAM cell is six MOSFETs. So very generally speaking the amount of SRAM you can pack into a chip is about 1/6 (or fewer) the amount of DRAM cells in the same space. Given that modern CPUs typically have only four DIMM slots populated in pairs and you can't add more without changing the memory interface, that means you've cut the total amount of memory possible in a system by about 80%, so max 6.4GB instead of 32GB. So.. a memory type for "enthusiasts" that costs e.g. 5x as much as DIMMS, only allows you 20% of the memory size, and also needs a motherboard and CPU design that's incompatible with DIMM based motherboards. Not attractive to too many people. If you want an easier way to make an enthusiast product with the performance benefits of SRAM, just put the memory on die on a multi-chip module so it's a very short distance from the processor cores (like Apple's M1). Short distance equals good signal integrity, low latency, and on die means simpler motherboard design. The trade off is that you can't add memory later, you have to buy a whole new chip.