5 ms·

Hmmm... I vaguely recall the term "shards" when referring to DB/2 running on S36, S38, and later newfangled AS/400 across the world. When presenting the data i

Hmmm... I vaguely recall the term "shards" when referring to DB/2 running on S36, S38, and later newfangled AS/400 across the world. When presenting the data in a single pane, some "shards" would come in late, or be broken, and require reconnecting.

brandall10 2y ago
The wikipedia entry for this confirms two sources depending on context, and in your case it came from a late 80s paper: Sarin, DeWitt & Rosenberg, Overview of SHARD: A System for Highly Available Replicated Data, Technical Report CCA-88-01, Computer Corporation of America, May 1988 https://en.wikipedia.org/wiki/Shard_(database_architecture)#Etymology https://en.wikipedia.org/wiki/Shard_(database_architecture)#...
- codethief 2y ago
  I seem to remember a conversation here on HN not too long ago where people tried to reconstruct the history behind "sharding" and, in particular, tried to find that 1980s paper you mentioned – without success. I believe they even contacted one of the authors.
  - mindcrime 2y ago
    Yes, that was this discussion: https://news.ycombinator.com/item?id=36848605 https://news.ycombinator.com/item?id=36848605
    - renewiltord 2y ago
      Actual paper did exist but no one has a copy anymore according to the contacted author. It was an internal Xerox memo.
      - edent 2y ago
        FWIW, I do now have a copy. I'm unsure of the copyright status so I'm a bit reluctant to share it.
        codethief 2y ago
        Congratulations! How did you end up getting the copy? Also, maybe university libraries and/or Archive.org could help you with the copyright question?
        edent 2y ago
        A commenter on my blog made contact with someone who had a copy.
        mindcrime 2y ago
        > I'm unsure of the copyright status so I'm a bit reluctant to share it. I can totally understand that, but at the same time I'd really love to have a look at that as well. Maybe you could reach out to archive.org and see if they'd be interested in hosting a copy?
- BoiledCabbage 2y ago
  From a quick search right now, the term seems to have come from that same system, but the first reference appears to be older than 1988. It looks to be possibly 1985. Following this link here: https://shkspr.mobi/blog/2021/06/where-is-the-original-overview-of-shard-paper/ https://shkspr.mobi/blog/2021/06/where-is-the-original-overv... In a comment at the bottom it references a paper published by a few people working jointly with Computer Corporation of America, MIT & Boston University. If you view that referenced paper "Correctness Conditions for Highly Available Databases" by N. Lynch, B. Blaustein & M. Siegel (https://apps.dtic.mil/sti/pdfs/ADA171427.pdf https://apps.dtic.mil/sti/pdfs/ADA171427.pdf), and look at section 1.2 it clearly describes "SHARD: (System for Highly Available Replicated Data)" as being underdevelopment at CCA. It also says if you want to learn more about Shard, see the paper's reference [SBK]. Checking out the references section of that paper it lists the following for [SBK]: Sarin, S. K., Blaustein, B. T., and Kaufman, C. W., "System Architecture for Partition Tolerant Distributed Databases," IEEE Transactions on Computers C-34, 12 (December 1985). pp. 1158-1163. Which means there was a paper published in 1985, describing the in development Shard system. It is possible that in 1985 they hadn't yet named the system "Shard", and it only got that name by 1988 - but it'd be interesting to check out that 1985 paper and see if they used the term Shard at all.
  - brandall10 2y ago
    Great work. It's esp. interesting that this is an acronym. You should submit a correction.
    - roberttod 2y ago
      HN post here claims this paper may not exist https://news.ycombinator.com/item?id=36848605 https://news.ycombinator.com/item?id=36848605
      - BoiledCabbage 2y ago
        That comment is referring to a different paper than the one I mentioned.
- scott_s 2y ago
  I found that paper title as well when looking into this exact question. That paper does not have the number of citations I would expect if it is the source of the term. It's possibly the source, but it's not obviously the source.
crawfishphase 2y ago
I worked with db2 people back in 98 and they talked about shards. They worked down in some cave in the basement that smelled like folgers coffee, feet, and camel filter bbq sauce. I would offer them a roll of TP each and every time they said the word. Their office chairs had seat-cushions that looked like they were pulled out of a dumpster fire, worn out and melty looking. With all the red meat, nicotine, and caffeine, Im assuming someone sharded there at least once. Maybe that is what got those seat cushions all worn out.

brandall10 2y ago

The wikipedia entry for this confirms two sources depending on context, and in your case it came from a late 80s paper: Sarin, DeWitt & Rosenberg, Overview of SHARD: A System for Highly Available Replicated Data, Technical Report CCA-88-01, Computer Corporation of America, May 1988 https://en.wikipedia.org/wiki/Shard_(database_architecture)#Etymology https://en.wikipedia.org/wiki/Shard_(database_architecture)#...

codethief 2y ago
I seem to remember a conversation here on HN not too long ago where people tried to reconstruct the history behind "sharding" and, in particular, tried to find that 1980s paper you mentioned – without success. I believe they even contacted one of the authors.
- mindcrime 2y ago
  Yes, that was this discussion: https://news.ycombinator.com/item?id=36848605 https://news.ycombinator.com/item?id=36848605
  - renewiltord 2y ago
    Actual paper did exist but no one has a copy anymore according to the contacted author. It was an internal Xerox memo.
    - edent 2y ago
      FWIW, I do now have a copy. I'm unsure of the copyright status so I'm a bit reluctant to share it.
      - codethief 2y ago
        Congratulations! How did you end up getting the copy? Also, maybe university libraries and/or Archive.org could help you with the copyright question?
        edent 2y ago
        A commenter on my blog made contact with someone who had a copy.
      - mindcrime 2y ago
        > I'm unsure of the copyright status so I'm a bit reluctant to share it. I can totally understand that, but at the same time I'd really love to have a look at that as well. Maybe you could reach out to archive.org and see if they'd be interested in hosting a copy?
BoiledCabbage 2y ago
From a quick search right now, the term seems to have come from that same system, but the first reference appears to be older than 1988. It looks to be possibly 1985. Following this link here: https://shkspr.mobi/blog/2021/06/where-is-the-original-overview-of-shard-paper/ https://shkspr.mobi/blog/2021/06/where-is-the-original-overv... In a comment at the bottom it references a paper published by a few people working jointly with Computer Corporation of America, MIT & Boston University. If you view that referenced paper "Correctness Conditions for Highly Available Databases" by N. Lynch, B. Blaustein & M. Siegel (https://apps.dtic.mil/sti/pdfs/ADA171427.pdf https://apps.dtic.mil/sti/pdfs/ADA171427.pdf), and look at section 1.2 it clearly describes "SHARD: (System for Highly Available Replicated Data)" as being underdevelopment at CCA. It also says if you want to learn more about Shard, see the paper's reference [SBK]. Checking out the references section of that paper it lists the following for [SBK]: Sarin, S. K., Blaustein, B. T., and Kaufman, C. W., "System Architecture for Partition Tolerant Distributed Databases," IEEE Transactions on Computers C-34, 12 (December 1985). pp. 1158-1163. Which means there was a paper published in 1985, describing the in development Shard system. It is possible that in 1985 they hadn't yet named the system "Shard", and it only got that name by 1988 - but it'd be interesting to check out that 1985 paper and see if they used the term Shard at all.
- brandall10 2y ago
  Great work. It's esp. interesting that this is an acronym. You should submit a correction.
  - roberttod 2y ago
    HN post here claims this paper may not exist https://news.ycombinator.com/item?id=36848605 https://news.ycombinator.com/item?id=36848605
    - BoiledCabbage 2y ago
      That comment is referring to a different paper than the one I mentioned.
scott_s 2y ago
I found that paper title as well when looking into this exact question. That paper does not have the number of citations I would expect if it is the source of the term. It's possibly the source, but it's not obviously the source.

crawfishphase 2y ago

I worked with db2 people back in 98 and they talked about shards. They worked down in some cave in the basement that smelled like folgers coffee, feet, and camel filter bbq sauce. I would offer them a roll of TP each and every time they said the word. Their office chairs had seat-cushions that looked like they were pulled out of a dumpster fire, worn out and melty looking. With all the red meat, nicotine, and caffeine, Im assuming someone sharded there at least once. Maybe that is what got those seat cushions all worn out.