5 ms·

Seattle library network outage nears a month

87 points by dangle1 2y ago

rolph 2y ago
re-lend from home ? "Book checkouts are being done by spreadsheet. Column A: the library user’s account number. Column B: the book’s bar code number. The low-tech inventory will be integrated with the library’s normal account system at some future, unknown date." - this is how we did it not too long ago; and before that, we had a signout card, held by library until return. before that was a handwritten ledger.
- chuckadams 2y ago
  Add an automatic timestamp column and you're mostly golden, you could rebuild most of the inventory state from that. Complex systems should be able to be rebuilt from simple logs. I'm not saying event sourcing makes it trivial to rebuild from scratch, but I suspect their currently ransomware-encrypted system was designed such that it's not even possible.
  - rolph 2y ago
    quite a few years ago, there was a punchclock-like device, the librarian punched the card, time stamp ; and patron number punched on a card [ that was kept in the volume and transfered to the borrowed stack on loan]
donkeybeer 2y ago
̶C̶o̶m̶p̶u̶t̶e̶r̶s̶ ̶d̶o̶n̶'̶t̶ ̶n̶e̶e̶d̶ ̶n̶e̶t̶w̶o̶r̶k̶s̶ ̶t̶o̶ ̶f̶u̶n̶c̶t̶i̶o̶n̶.̶ ̶I̶t̶ ̶i̶s̶ ̶p̶e̶r̶h̶a̶p̶s̶ ̶a̶n̶ ̶i̶n̶d̶i̶c̶a̶t̶i̶o̶n̶ ̶o̶f̶ ̶h̶o̶w̶ ̶t̶h̶e̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶i̶n̶d̶u̶s̶t̶r̶y̶ ̶i̶m̶p̶l̶e̶m̶e̶n̶t̶e̶d̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶e̶n̶f̶o̶r̶c̶e̶m̶e̶n̶t̶ ̶i̶n̶ ̶t̶h̶e̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶w̶o̶r̶l̶d̶ ̶t̶h̶a̶t̶ ̶l̶i̶b̶r̶a̶r̶y̶ ̶n̶e̶e̶d̶e̶d̶ ̶t̶o̶ ̶b̶e̶ ̶c̶o̶n̶n̶e̶c̶t̶e̶d̶ ̶t̶o̶ ̶t̶h̶e̶ ̶n̶e̶t̶w̶o̶r̶k̶ ̶t̶o̶ ̶l̶e̶n̶d̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶c̶o̶p̶i̶e̶s̶ ̶o̶f̶ ̶b̶o̶o̶k̶s̶.̶ ̶ ̶T̶h̶a̶t̶ ̶s̶a̶i̶d̶,̶ ̶I̶ ̶s̶t̶i̶l̶l̶ ̶t̶h̶i̶n̶k̶ ̶(̶i̶f̶ ̶t̶h̶e̶y̶ ̶a̶r̶e̶n̶'̶t̶ ̶a̶l̶r̶e̶a̶d̶y̶ ̶d̶o̶i̶n̶g̶ ̶i̶t̶)̶ ̶t̶h̶e̶y̶ ̶s̶h̶o̶u̶l̶d̶ ̶h̶a̶v̶e̶ ̶k̶e̶p̶t̶ ̶a̶ ̶l̶o̶c̶a̶l̶ ̶c̶o̶p̶y̶ ̶o̶f̶ ̶f̶o̶r̶ ̶e̶x̶a̶m̶p̶l̶e̶ ̶P̶r̶o̶j̶e̶c̶t̶ ̶G̶u̶t̶e̶n̶b̶e̶r̶g̶ ̶a̶n̶d̶ ̶o̶t̶h̶e̶r̶ ̶s̶u̶c̶h̶ ̶a̶r̶c̶h̶i̶v̶e̶s̶ ̶o̶f̶ ̶o̶u̶t̶ ̶o̶f̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶c̶o̶n̶t̶e̶n̶t̶.̶ ̶ ̶E̶d̶i̶t̶:̶ ̶s̶o̶ ̶I̶ ̶r̶e̶a̶d̶ ̶t̶h̶e̶ ̶a̶r̶t̶i̶c̶l̶e̶ ̶a̶n̶d̶ ̶t̶h̶e̶ ̶p̶r̶o̶b̶l̶e̶m̶ ̶s̶e̶e̶m̶s̶ ̶t̶o̶ ̶b̶e̶ ̶m̶o̶r̶e̶ ̶a̶f̶f̶e̶c̶t̶i̶n̶g̶ ̶t̶h̶e̶i̶r̶ ̶a̶c̶c̶o̶u̶n̶t̶i̶n̶g̶ ̶a̶n̶d̶ ̶l̶o̶g̶g̶i̶n̶g̶ ̶o̶f̶ ̶b̶o̶o̶k̶s̶ ̶l̶e̶n̶t̶ ̶e̶t̶c̶.̶ ̶T̶h̶e̶r̶e̶ ̶i̶s̶ ̶n̶o̶t̶h̶i̶n̶g̶ ̶s̶a̶i̶d̶ ̶a̶b̶o̶u̶t̶ ̶a̶n̶y̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶l̶i̶b̶r̶a̶r̶y̶ ̶b̶e̶i̶n̶g̶ ̶d̶o̶w̶n̶ ̶e̶x̶p̶l̶i̶c̶i̶t̶l̶y̶ ̶b̶u̶t̶ ̶i̶'̶d̶ ̶s̶a̶y̶ ̶i̶t̶ ̶s̶t̶i̶l̶l̶ ̶a̶p̶p̶l̶i̶e̶s̶,̶ ̶t̶h̶a̶t̶ ̶k̶e̶e̶p̶i̶n̶g̶ ̶l̶o̶c̶a̶l̶ ̶c̶o̶p̶i̶e̶s̶ ̶o̶f̶ ̶s̶t̶u̶f̶f̶ ̶i̶s̶ ̶u̶s̶e̶f̶u̶l̶.̶ So my comment is not relevant, and should be ignored. Managing the metadata of this scale is certainly no easy task without computer systems.
- shiandow 2y ago
  One could of course try to keep the library, its 27 branches and its website up to date without connecting anything to the internet. But I think it's understandable why they chose not to.
  - pinewurst 2y ago
    I think they're rightfully suspicious of what remains in their computers even not connected (knowingly) to the Internet.
  - donkeybeer 2y ago
    Yes I apologize, I had misunderstood. The computers main purpose was being used with the management of the metadata, there is no mention of book content itself on the network. Sorry.
- Arainach 2y ago
  The Seattle Library has 27 locations and a million books, serving a city with 700,000+ people (and a larger metro area - I forget if you need to be a Seattle resident to get a card). To be able to query materials across the library system, handle reservations and item transfers, and so much more these computers absolutely need to be networked. Perhaps you meant "don't need to be connected to the Internet", but I would wager a majority of the library's patrons search the library's catalog online rather than coming in person. In the case of digital loans, moat patrons want to download their digital books onto their personal devices - over the internet, so clearly the library services need to talk to an online service.
  - pinewurst 2y ago
    No - pretty much anyone in the state can get a card via reciprocality.
  - donkeybeer 2y ago
    I apologize, I had misunderstood the problem. The problem is the metadata not the actual books were managed on the computer systems. Sorry.
deleted 2y ago
[deleted]
KineticLensman 2y ago
The British Library - the UK's National Library - was devastated in October 2023 by a ransomware attack [0] which had a massive knock-on effect on academic institutions, students and also 20,000 authors who derive income from Public Lending Rights. One of the reasons that services still haven't been restored is that the Library relied heavily on ancient bespoke software running on old versions of OS. New IT is being installed that is more proof against modern cyber attacks, but the older library software simply doesn't run in modern environments. So they can't simply restore from backup, they have to port / reimplement some of their operational software. [0] https://en.wikipedia.org/wiki/British_Library_cyberattack https://en.wikipedia.org/wiki/British_Library_cyberattack
- tmpz22 2y ago
  It's a shame the same passion for porting old video games doesn't exist for library software. If we can get doom to run on a refrigerator surely we can get a less computationally intensive software to run - if only in a virtual machine on a more modern OS.
  - sneak 2y ago
    Library operations are simple CRUD, a competent engineer could replace most library systems with a webapp in a week. Running webapps on machines immune to ransomware is a solved problem. These organizations are simply incompetent and are now paying the price for being run by fools who don’t know what they’re doing.
    - KineticLensman 2y ago
      > Library operations are simple CRUD Not the ones that manage royalties payments to 1000s of authors depending on data collected by a national system of library IT, or the Secure (!) electronic delivery service that makes more than 100 million items available to researchers worldwide. See [1] for a full list of the massive range of non-trivial services operated by the National Library. > These organizations are simply incompetent and are now paying the price for being run by fools who don’t know what they’re doing Well Information Management is literally what some of their staff are trained in, but the National Library is publicly funded, so can't update software on its own terms. [1] https://en.wikipedia.org/wiki/British_Library https://en.wikipedia.org/wiki/British_Library
AndrewKemendo 2y ago
I would bet good money they some “whiz technology person” is now rabidly trying to figure out how to turn the card catalog into an “offline saas digital lending service” and pitch venture for a $2M pre-seed to pitch libraries on it
- Kon-Peki 2y ago
  Rabidly is probably a good word to use. This whiz technology person is also going to discover, too late, that most libraries will refuse to touch it unless it can be proven that a patron's lending history is not recorded unless the patron opts-in.
- tmpz22 2y ago
  The "whiz technology person" after a few months of toying with NextJS will fully embrace the vendor-lock-in with a poorly optimized and rushed implementation using a team of college drop outs who themselves have only a few months (claimed as years) experience with SPA development. The resulting monstrosity will not be properly reviewed by government purchasers and will become the absolute bane of existence for every end user. Several years in the future a massive multimillion dollar contract will be issued by the government to overhaul the software. It will arrive years late and achieve exactly the same result. 2 new homes will be purchased in Atherton.
- balls187 2y ago
  I would think that instead of offline, they would move to using the blockchain.
  - AndrewKemendo 2y ago
    Good call, totally missed that opportunity
mlekoszek 2y ago
This happened in Toronto last year and knocked out the system entirely for four months. It seems that libraries are a big target for these actors. I wonder who they are, and what their agenda is?
- maximilianburke 2y ago
  Their agenda is to make money and/or create chaos.
cptcobalt 2y ago
https://archive.is/yVa8c https://archive.is/yVa8c
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]
johnklos 2y ago
Whenever I hear about ransomware attacks, I wonder how anyone could pay for IT services to set up a system where: 1) there are no meaningful, clean backups and 2) regular users can overwrite many / most files 3) and/or administrators use machines that are easily compromisible 4) and/or the system is built on insecure technology The fact that 1) could negate 2), 3) and 4) means people are doing IT and are taking good money from corporations or taxpayers when they absolutely should not. So when there are things like this in the news where large numbers of users are affected, I can't help but wonder why we don't see IT companies getting strung up (figuratively, of course) and publicly embarrassed for each and every one of these incidences.
ncr100 2y ago
More than just books, this is internet for people without home internet: > “A lot of people come here for internet access, so it’s been quite a blow to the community,” a librarian at the Greenwood library said. > Some individual library branches have put together lists of nearby places that could offer similar services. But they’re not great. > The downtown library recommends a FedEx, a quarter mile away, but internet access costs 39 cents a minute. There is a public law library in the King County Courthouse, but computers there are intended for legal matters only. The nearest branches of the King County Library System are all about 40 minutes, by bus, from downtown. What an enormously awful impact this weasel hacker / group has imposed upon innocent people. It's not bloviating to say this is what evil looks like: is causing suffering and further dividing a group of people from those that do have home PC/internet, for no good reason.

87 points by dangle1 2y ago

re-lend from home ? "Book checkouts are being done by spreadsheet. Column A: the library user’s account number. Column B: the book’s bar code number. The low-tech inventory will be integrated with the library’s normal account system at some future, unknown date." - this is how we did it not too long ago; and before that, we had a signout card, held by library until return. before that was a handwritten ledger.

chuckadams 2y ago
Add an automatic timestamp column and you're mostly golden, you could rebuild most of the inventory state from that. Complex systems should be able to be rebuilt from simple logs. I'm not saying event sourcing makes it trivial to rebuild from scratch, but I suspect their currently ransomware-encrypted system was designed such that it's not even possible.
- rolph 2y ago
  quite a few years ago, there was a punchclock-like device, the librarian punched the card, time stamp ; and patron number punched on a card [ that was kept in the volume and transfered to the borrowed stack on loan]

donkeybeer 2y ago

̶C̶o̶m̶p̶u̶t̶e̶r̶s̶ ̶d̶o̶n̶'̶t̶ ̶n̶e̶e̶d̶ ̶n̶e̶t̶w̶o̶r̶k̶s̶ ̶t̶o̶ ̶f̶u̶n̶c̶t̶i̶o̶n̶.̶ ̶I̶t̶ ̶i̶s̶ ̶p̶e̶r̶h̶a̶p̶s̶ ̶a̶n̶ ̶i̶n̶d̶i̶c̶a̶t̶i̶o̶n̶ ̶o̶f̶ ̶h̶o̶w̶ ̶t̶h̶e̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶i̶n̶d̶u̶s̶t̶r̶y̶ ̶i̶m̶p̶l̶e̶m̶e̶n̶t̶e̶d̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶e̶n̶f̶o̶r̶c̶e̶m̶e̶n̶t̶ ̶i̶n̶ ̶t̶h̶e̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶w̶o̶r̶l̶d̶ ̶t̶h̶a̶t̶ ̶l̶i̶b̶r̶a̶r̶y̶ ̶n̶e̶e̶d̶e̶d̶ ̶t̶o̶ ̶b̶e̶ ̶c̶o̶n̶n̶e̶c̶t̶e̶d̶ ̶t̶o̶ ̶t̶h̶e̶ ̶n̶e̶t̶w̶o̶r̶k̶ ̶t̶o̶ ̶l̶e̶n̶d̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶c̶o̶p̶i̶e̶s̶ ̶o̶f̶ ̶b̶o̶o̶k̶s̶.̶ ̶ ̶T̶h̶a̶t̶ ̶s̶a̶i̶d̶,̶ ̶I̶ ̶s̶t̶i̶l̶l̶ ̶t̶h̶i̶n̶k̶ ̶(̶i̶f̶ ̶t̶h̶e̶y̶ ̶a̶r̶e̶n̶'̶t̶ ̶a̶l̶r̶e̶a̶d̶y̶ ̶d̶o̶i̶n̶g̶ ̶i̶t̶)̶ ̶t̶h̶e̶y̶ ̶s̶h̶o̶u̶l̶d̶ ̶h̶a̶v̶e̶ ̶k̶e̶p̶t̶ ̶a̶ ̶l̶o̶c̶a̶l̶ ̶c̶o̶p̶y̶ ̶o̶f̶ ̶f̶o̶r̶ ̶e̶x̶a̶m̶p̶l̶e̶ ̶P̶r̶o̶j̶e̶c̶t̶ ̶G̶u̶t̶e̶n̶b̶e̶r̶g̶ ̶a̶n̶d̶ ̶o̶t̶h̶e̶r̶ ̶s̶u̶c̶h̶ ̶a̶r̶c̶h̶i̶v̶e̶s̶ ̶o̶f̶ ̶o̶u̶t̶ ̶o̶f̶ ̶c̶o̶p̶y̶r̶i̶g̶h̶t̶ ̶c̶o̶n̶t̶e̶n̶t̶.̶ ̶ ̶E̶d̶i̶t̶:̶ ̶s̶o̶ ̶I̶ ̶r̶e̶a̶d̶ ̶t̶h̶e̶ ̶a̶r̶t̶i̶c̶l̶e̶ ̶a̶n̶d̶ ̶t̶h̶e̶ ̶p̶r̶o̶b̶l̶e̶m̶ ̶s̶e̶e̶m̶s̶ ̶t̶o̶ ̶b̶e̶ ̶m̶o̶r̶e̶ ̶a̶f̶f̶e̶c̶t̶i̶n̶g̶ ̶t̶h̶e̶i̶r̶ ̶a̶c̶c̶o̶u̶n̶t̶i̶n̶g̶ ̶a̶n̶d̶ ̶l̶o̶g̶g̶i̶n̶g̶ ̶o̶f̶ ̶b̶o̶o̶k̶s̶ ̶l̶e̶n̶t̶ ̶e̶t̶c̶.̶ ̶T̶h̶e̶r̶e̶ ̶i̶s̶ ̶n̶o̶t̶h̶i̶n̶g̶ ̶s̶a̶i̶d̶ ̶a̶b̶o̶u̶t̶ ̶a̶n̶y̶ ̶d̶i̶g̶i̶t̶a̶l̶ ̶l̶i̶b̶r̶a̶r̶y̶ ̶b̶e̶i̶n̶g̶ ̶d̶o̶w̶n̶ ̶e̶x̶p̶l̶i̶c̶i̶t̶l̶y̶ ̶b̶u̶t̶ ̶i̶'̶d̶ ̶s̶a̶y̶ ̶i̶t̶ ̶s̶t̶i̶l̶l̶ ̶a̶p̶p̶l̶i̶e̶s̶,̶ ̶t̶h̶a̶t̶ ̶k̶e̶e̶p̶i̶n̶g̶ ̶l̶o̶c̶a̶l̶ ̶c̶o̶p̶i̶e̶s̶ ̶o̶f̶ ̶s̶t̶u̶f̶f̶ ̶i̶s̶ ̶u̶s̶e̶f̶u̶l̶.̶ So my comment is not relevant, and should be ignored. Managing the metadata of this scale is certainly no easy task without computer systems.

shiandow 2y ago
One could of course try to keep the library, its 27 branches and its website up to date without connecting anything to the internet. But I think it's understandable why they chose not to.
- pinewurst 2y ago
  I think they're rightfully suspicious of what remains in their computers even not connected (knowingly) to the Internet.
- donkeybeer 2y ago
  Yes I apologize, I had misunderstood. The computers main purpose was being used with the management of the metadata, there is no mention of book content itself on the network. Sorry.
Arainach 2y ago
The Seattle Library has 27 locations and a million books, serving a city with 700,000+ people (and a larger metro area - I forget if you need to be a Seattle resident to get a card). To be able to query materials across the library system, handle reservations and item transfers, and so much more these computers absolutely need to be networked. Perhaps you meant "don't need to be connected to the Internet", but I would wager a majority of the library's patrons search the library's catalog online rather than coming in person. In the case of digital loans, moat patrons want to download their digital books onto their personal devices - over the internet, so clearly the library services need to talk to an online service.
- pinewurst 2y ago
  No - pretty much anyone in the state can get a card via reciprocality.
- donkeybeer 2y ago
  I apologize, I had misunderstood the problem. The problem is the metadata not the actual books were managed on the computer systems. Sorry.

deleted 2y ago

[deleted]

KineticLensman 2y ago

The British Library - the UK's National Library - was devastated in October 2023 by a ransomware attack [0] which had a massive knock-on effect on academic institutions, students and also 20,000 authors who derive income from Public Lending Rights. One of the reasons that services still haven't been restored is that the Library relied heavily on ancient bespoke software running on old versions of OS. New IT is being installed that is more proof against modern cyber attacks, but the older library software simply doesn't run in modern environments. So they can't simply restore from backup, they have to port / reimplement some of their operational software. [0] https://en.wikipedia.org/wiki/British_Library_cyberattack https://en.wikipedia.org/wiki/British_Library_cyberattack

tmpz22 2y ago
It's a shame the same passion for porting old video games doesn't exist for library software. If we can get doom to run on a refrigerator surely we can get a less computationally intensive software to run - if only in a virtual machine on a more modern OS.
- sneak 2y ago
  Library operations are simple CRUD, a competent engineer could replace most library systems with a webapp in a week. Running webapps on machines immune to ransomware is a solved problem. These organizations are simply incompetent and are now paying the price for being run by fools who don’t know what they’re doing.
  - KineticLensman 2y ago
    > Library operations are simple CRUD Not the ones that manage royalties payments to 1000s of authors depending on data collected by a national system of library IT, or the Secure (!) electronic delivery service that makes more than 100 million items available to researchers worldwide. See [1] for a full list of the massive range of non-trivial services operated by the National Library. > These organizations are simply incompetent and are now paying the price for being run by fools who don’t know what they’re doing Well Information Management is literally what some of their staff are trained in, but the National Library is publicly funded, so can't update software on its own terms. [1] https://en.wikipedia.org/wiki/British_Library https://en.wikipedia.org/wiki/British_Library

AndrewKemendo 2y ago

I would bet good money they some “whiz technology person” is now rabidly trying to figure out how to turn the card catalog into an “offline saas digital lending service” and pitch venture for a $2M pre-seed to pitch libraries on it

Kon-Peki 2y ago
Rabidly is probably a good word to use. This whiz technology person is also going to discover, too late, that most libraries will refuse to touch it unless it can be proven that a patron's lending history is not recorded unless the patron opts-in.
tmpz22 2y ago
The "whiz technology person" after a few months of toying with NextJS will fully embrace the vendor-lock-in with a poorly optimized and rushed implementation using a team of college drop outs who themselves have only a few months (claimed as years) experience with SPA development. The resulting monstrosity will not be properly reviewed by government purchasers and will become the absolute bane of existence for every end user. Several years in the future a massive multimillion dollar contract will be issued by the government to overhaul the software. It will arrive years late and achieve exactly the same result. 2 new homes will be purchased in Atherton.
balls187 2y ago
I would think that instead of offline, they would move to using the blockchain.
- AndrewKemendo 2y ago
  Good call, totally missed that opportunity

mlekoszek 2y ago

This happened in Toronto last year and knocked out the system entirely for four months. It seems that libraries are a big target for these actors. I wonder who they are, and what their agenda is?

maximilianburke 2y ago
Their agenda is to make money and/or create chaos.

cptcobalt 2y ago

https://archive.is/yVa8c https://archive.is/yVa8c

deleted 2y ago

[deleted]

deleted 2y ago

[deleted]

johnklos 2y ago

Whenever I hear about ransomware attacks, I wonder how anyone could pay for IT services to set up a system where: 1) there are no meaningful, clean backups and 2) regular users can overwrite many / most files 3) and/or administrators use machines that are easily compromisible 4) and/or the system is built on insecure technology The fact that 1) could negate 2), 3) and 4) means people are doing IT and are taking good money from corporations or taxpayers when they absolutely should not. So when there are things like this in the news where large numbers of users are affected, I can't help but wonder why we don't see IT companies getting strung up (figuratively, of course) and publicly embarrassed for each and every one of these incidences.

ncr100 2y ago

More than just books, this is internet for people without home internet: > “A lot of people come here for internet access, so it’s been quite a blow to the community,” a librarian at the Greenwood library said. > Some individual library branches have put together lists of nearby places that could offer similar services. But they’re not great. > The downtown library recommends a FedEx, a quarter mile away, but internet access costs 39 cents a minute. There is a public law library in the King County Courthouse, but computers there are intended for legal matters only. The nearest branches of the King County Library System are all about 40 minutes, by bus, from downtown. What an enormously awful impact this weasel hacker / group has imposed upon innocent people. It's not bloviating to say this is what evil looks like: is causing suffering and further dividing a group of people from those that do have home PC/internet, for no good reason.