4 ms·

The Talos II, Blackbird POWER9 systems support tagged memory

125 points by hlandau 4y ago

tony-allan 4y ago
I like the first 2 footnotes from the post: 1. This point is worth drawing attention to; the abstraction of the machine's ISA was so effective in the case of the AS/400 that IBM never even documented the CPU's ISA, nor did they need to. Supposedly, the ISA was System 360-like. 2. The fact that AS/400 and its predecessor System/38 significantly predate Java demonstrates that there really is no “new” idea in computer science that IBM wasn't doing in the 70s, an observation I've become fond of. The explosion of “virtualisation” in IT in the 2000s/2010s is another example; to my knowledge, hardware virtualisation was first invented by IBM in the System 370.
- KerrAvon 4y ago
  The predecessor to Java was really the UCSD P-system, which was probably contemporary with the System/38.
  - pjmlp 4y ago
    Burroughs was already doing such ideas in 1961.
- KerrAvon 4y ago
  IIRC System/38 was the one product to actually ship that was based on concepts from IBM’s FS effort. FS was an attempt to produce a fully virtualized — at every layer — System/370 successor in the mid to late 70’s; it failed.
- vmcms 4y ago
  Having cut my teeth on VM/CMS, I really appreciate the crystallized wisdom in "there really is no 'new' idea in computer science that IBM wasn't doing in the 70s" - though I'd simplify it to "there really is no 'new' idea in computer science that wasn't implemented in the 70s." Not every innovation happened at IBM first back in those days, any more than they happened at Microsoft first in the '90s, even if IBM (and Microsoft) were more successful at capitalizing on the innovations of others due to the advantage of being well-capitalized second movers.
- pjmlp 4y ago
  Burroughs did it first, in what concerns Java like systems, in 1961. https://en.m.wikipedia.org/wiki/Burroughs_large_systems https://en.m.wikipedia.org/wiki/Burroughs_large_systems
markphip 4y ago
Seeing the headline, I wondered if the article would be about the IBM iSeries. Spent first 15 years of my career working on those, always a fond memory. Great machines to develop traditional database apps on.
- zasdffaa 4y ago
  > Great machines to develop traditional database apps on Hi, could you explain what it was that made them so good to do this, thanks.
  - lproven 4y ago
    Single-level store is a big thing. No other modern OS has or can do that. The integral database, too? https://en.wikipedia.org/wiki/IBM_i#SLIC https://en.wikipedia.org/wiki/IBM_i#SLIC
    - zasdffaa 4y ago
      A search for Single-level store gets me https://en.wikipedia.org/wiki/Single-level_store https://en.wikipedia.org/wiki/Single-level_store "The term originally referred to what is now usually called virtual memory" And seems to lead on to memory mapped files, so nothing apparently significant AFAICS. I'll check out your slic link, thanks.
      - lproven 4y ago
        Single-level store is _nothing_ to do with virtual memory. That's a bad edit and can be ignored. Read the rest of the article.
        zasdffaa 4y ago
        I don't understand that. Quite literally it is (or seems to be). In the 'design' section: "With a single-level storage the entire storage of a computer is thought of as a single two-dimensional plane of addresses, pointing to pages. Pages may be in primary storage (RAM) or in secondary storage (disk); however, the current location of an address is unimportant to a process. The operating system takes on the responsibility of locating pages and making them available for processing. If a page is in primary storage, it is immediately available. If a page is on disk, a page fault occurs and the operating system brings the page into primary storage. No explicit I/O to secondary storage is done by processes: instead, reads from secondary storage are done as the result of page faults; writes to secondary storage are done when pages that have been modified since being read from secondary storage into primary storage are written back to their location in secondary storage." IOW this is classic VM behaviour. Which bit of the wiki article is revelatory? (NB. I've actually used multics).
edwintorok 4y ago
Interesting that it does ECC on blocks of 64 byte instead of 64-bit, freeing up some of the bits in ECC RAM to be used for other things like extra tag bits.
- addaon 4y ago
  Yeah. I've mentioned this before, but in DDR5 and beyond the nominal channel width goes from 64 bit to 32 bit -- and an ECC protected channel goes from 72 bit to 40 bit. (It would be possible to ECC protect a 32-bit channel with 39 bits, but the actual DRAM ICs come in multiples of 8 bits.) This means that applications that bond channels together and read/write/ECC-check in 64-bit chunks can have 8 unassigned bits per 80-bit read, while still ECC protecting the full width. I haven't seen any discussion of this capability, but it's super interesting to me.
temptemptemp111 4y ago
twoodfin 4y ago
The title undersells the content here: This is an excellent summary of what pointer tagging is, how AS/400 / iSeries uses it, and the likely implementation built atop unconventional uses of ECC.
the-rc 4y ago
Neither this page nor the one about the AS extensions seem to describe what happens when storing data to disk. AFAIR, IBM used drives with 520 byte sectors. 512 bytes were the actual data. The remaining 64 bits were used to set tags in memory accordingly (you can fit 32 16-byte pointers in the sector, so each pointer had two tag bits and both had to be on for the pointer to be deemed valid). I thought that there was a special instruction to set all of a 512 byte memory region's tags at once. I don't see it mentioned. Maybe it's only a virtual instruction and it's implemented by just a bunch of SETTAG/STQ, which would destroy performance or negate most of the benefits of DMA I/O (you could skip this step if you know the block has no pointers...). You need to extract the tags from memory, pack them and save them separately when writing to persistent storage, of course.
- hlandau 4y ago
  Author here; yes, the sole focus here is on the ISA-relevant parts. Obviously disk formats are out of scope. It's correct that IBM originally used custom IBM disk drives with 520-byte sectors. I believe 528 byte sectors were also seen where both tags and parity (for some RAID-like technology, I don't recall the details) were desired. On devices with a 4096 byte sector size, this leads to equivalent sector sizes of 4160 or 4224 bytes respectively. Nowadays you can also run IBM i on virtualised disks provided by IBM's Virtual I/O System (VIOS) under PowerVM. VIOS is AIUI basically a customised AIX install which runs as a VM under PowerVM and provides I/O functionality similar to that provided by a Xen dom0. In this case things are virtualised on top of normal hard drives with 512 byte sectors, so it's possible for these weird sector sizes to be emulated nowadays. I believe you can still pass a hard drive directly to an IBM i VM, though, and that IBM still offers hard drives with these special sector sizes for this purpose. Of course IBM doesn't make drives nowadays, so these are obviously an IBM rebrand of some other manufacturer's hard drives. What's really interesting though is that if you read the manuals for modern hard drives (yes, hard drives have manuals, who knew?), many of them advertise a selectable sector size, where you can reformat the drive to a new sector size. For example, see this manual, which says you can select a sector size of 512, 520, or 528 bytes (or 4096, 4160 or 4224 bytes): https://www.seagate.com/www-content/product-content/enterprise-hdd-fam/exos-7e10/_shared/docs/100870879e.pdf#page=10 https://www.seagate.com/www-content/product-content/enterpri... So theoretically most modern hard drives can be reformatted to support IBM i. I believe this functionality is also maintained to support modern hardware RAID systems, though.

125 points by hlandau 4y ago

I like the first 2 footnotes from the post: 1. This point is worth drawing attention to; the abstraction of the machine's ISA was so effective in the case of the AS/400 that IBM never even documented the CPU's ISA, nor did they need to. Supposedly, the ISA was System 360-like. 2. The fact that AS/400 and its predecessor System/38 significantly predate Java demonstrates that there really is no “new” idea in computer science that IBM wasn't doing in the 70s, an observation I've become fond of. The explosion of “virtualisation” in IT in the 2000s/2010s is another example; to my knowledge, hardware virtualisation was first invented by IBM in the System 370.

KerrAvon 4y ago
The predecessor to Java was really the UCSD P-system, which was probably contemporary with the System/38.
- pjmlp 4y ago
  Burroughs was already doing such ideas in 1961.
KerrAvon 4y ago
IIRC System/38 was the one product to actually ship that was based on concepts from IBM’s FS effort. FS was an attempt to produce a fully virtualized — at every layer — System/370 successor in the mid to late 70’s; it failed.
vmcms 4y ago
Having cut my teeth on VM/CMS, I really appreciate the crystallized wisdom in "there really is no 'new' idea in computer science that IBM wasn't doing in the 70s" - though I'd simplify it to "there really is no 'new' idea in computer science that wasn't implemented in the 70s." Not every innovation happened at IBM first back in those days, any more than they happened at Microsoft first in the '90s, even if IBM (and Microsoft) were more successful at capitalizing on the innovations of others due to the advantage of being well-capitalized second movers.
pjmlp 4y ago
Burroughs did it first, in what concerns Java like systems, in 1961. https://en.m.wikipedia.org/wiki/Burroughs_large_systems https://en.m.wikipedia.org/wiki/Burroughs_large_systems

markphip 4y ago

Seeing the headline, I wondered if the article would be about the IBM iSeries. Spent first 15 years of my career working on those, always a fond memory. Great machines to develop traditional database apps on.

zasdffaa 4y ago
> Great machines to develop traditional database apps on Hi, could you explain what it was that made them so good to do this, thanks.
- lproven 4y ago
  Single-level store is a big thing. No other modern OS has or can do that. The integral database, too? https://en.wikipedia.org/wiki/IBM_i#SLIC https://en.wikipedia.org/wiki/IBM_i#SLIC
  - zasdffaa 4y ago
    A search for Single-level store gets me https://en.wikipedia.org/wiki/Single-level_store https://en.wikipedia.org/wiki/Single-level_store "The term originally referred to what is now usually called virtual memory" And seems to lead on to memory mapped files, so nothing apparently significant AFAICS. I'll check out your slic link, thanks.
    - lproven 4y ago
      Single-level store is _nothing_ to do with virtual memory. That's a bad edit and can be ignored. Read the rest of the article.
      - zasdffaa 4y ago
        I don't understand that. Quite literally it is (or seems to be). In the 'design' section: "With a single-level storage the entire storage of a computer is thought of as a single two-dimensional plane of addresses, pointing to pages. Pages may be in primary storage (RAM) or in secondary storage (disk); however, the current location of an address is unimportant to a process. The operating system takes on the responsibility of locating pages and making them available for processing. If a page is in primary storage, it is immediately available. If a page is on disk, a page fault occurs and the operating system brings the page into primary storage. No explicit I/O to secondary storage is done by processes: instead, reads from secondary storage are done as the result of page faults; writes to secondary storage are done when pages that have been modified since being read from secondary storage into primary storage are written back to their location in secondary storage." IOW this is classic VM behaviour. Which bit of the wiki article is revelatory? (NB. I've actually used multics).

edwintorok 4y ago

Interesting that it does ECC on blocks of 64 byte instead of 64-bit, freeing up some of the bits in ECC RAM to be used for other things like extra tag bits.

addaon 4y ago
Yeah. I've mentioned this before, but in DDR5 and beyond the nominal channel width goes from 64 bit to 32 bit -- and an ECC protected channel goes from 72 bit to 40 bit. (It would be possible to ECC protect a 32-bit channel with 39 bits, but the actual DRAM ICs come in multiples of 8 bits.) This means that applications that bond channels together and read/write/ECC-check in 64-bit chunks can have 8 unassigned bits per 80-bit read, while still ECC protecting the full width. I haven't seen any discussion of this capability, but it's super interesting to me.

temptemptemp111 4y ago

twoodfin 4y ago

The title undersells the content here: This is an excellent summary of what pointer tagging is, how AS/400 / iSeries uses it, and the likely implementation built atop unconventional uses of ECC.

the-rc 4y ago

Neither this page nor the one about the AS extensions seem to describe what happens when storing data to disk. AFAIR, IBM used drives with 520 byte sectors. 512 bytes were the actual data. The remaining 64 bits were used to set tags in memory accordingly (you can fit 32 16-byte pointers in the sector, so each pointer had two tag bits and both had to be on for the pointer to be deemed valid). I thought that there was a special instruction to set all of a 512 byte memory region's tags at once. I don't see it mentioned. Maybe it's only a virtual instruction and it's implemented by just a bunch of SETTAG/STQ, which would destroy performance or negate most of the benefits of DMA I/O (you could skip this step if you know the block has no pointers...). You need to extract the tags from memory, pack them and save them separately when writing to persistent storage, of course.

hlandau 4y ago
Author here; yes, the sole focus here is on the ISA-relevant parts. Obviously disk formats are out of scope. It's correct that IBM originally used custom IBM disk drives with 520-byte sectors. I believe 528 byte sectors were also seen where both tags and parity (for some RAID-like technology, I don't recall the details) were desired. On devices with a 4096 byte sector size, this leads to equivalent sector sizes of 4160 or 4224 bytes respectively. Nowadays you can also run IBM i on virtualised disks provided by IBM's Virtual I/O System (VIOS) under PowerVM. VIOS is AIUI basically a customised AIX install which runs as a VM under PowerVM and provides I/O functionality similar to that provided by a Xen dom0. In this case things are virtualised on top of normal hard drives with 512 byte sectors, so it's possible for these weird sector sizes to be emulated nowadays. I believe you can still pass a hard drive directly to an IBM i VM, though, and that IBM still offers hard drives with these special sector sizes for this purpose. Of course IBM doesn't make drives nowadays, so these are obviously an IBM rebrand of some other manufacturer's hard drives. What's really interesting though is that if you read the manuals for modern hard drives (yes, hard drives have manuals, who knew?), many of them advertise a selectable sector size, where you can reformat the drive to a new sector size. For example, see this manual, which says you can select a sector size of 512, 520, or 528 bytes (or 4096, 4160 or 4224 bytes): https://www.seagate.com/www-content/product-content/enterprise-hdd-fam/exos-7e10/_shared/docs/100870879e.pdf#page=10 https://www.seagate.com/www-content/product-content/enterpri... So theoretically most modern hard drives can be reformatted to support IBM i. I believe this functionality is also maintained to support modern hardware RAID systems, though.