21 ms·

Upgrading from Debian Jessie to Bullseye after nearly 30 years

292 points by jklinger410 4y ago

foobarian 4y ago
Slightly clickbaity: "last major OS upgrade was to jessie (Debian 8, released in April 2015). That was in 2016." "after nearly 30 years running Debian i386" But impressive nonetheless!
- joeyh 4y ago
  The actual title is not the inspid word soup deposited atop this thread.
  - bscphil 4y ago
    Yeah @dang this title is terrible. Maybe "Upgrading from x86 Debian Jessie directly to x64 Bullseye" would be better?
  - jklinger410 4y ago
    The original title is: chiark’s skip-skip-cross-up-grade Since that isn't very informative, I tried to simplify the first sentence: Two weeks ago I upgraded chiark from Debian jessie i386 to bullseye amd64, after nearly 30 years running Debian i386. This went really quite well, in fact! Realizing now the feet is going from i386 to amd64, rather than an old debian version upgrade. Unfortunately I can't change the title.
- GuB-42 4y ago
  Clickbait indeed, I actually clicked because I knew it was wrong. The true story is that the server started in 1993 as Debian 0.93R5, getting upgrades from time to time, the last one being Jessie to Bullseye. I expected some bullshit or parody, but here, it is genuinely impressive.
- josephcsible 4y ago
  But i386 is the architecture, and until now, none of the upgrades have changed that.
- rob_c 4y ago
  Yeah, my big feeling is it's become a philosophical question as to where it's the same server after so many updates/upgrades.
hprotagonist 4y ago
a domain name i know primarily from about 20 years of downloading PuTTY and friends …
superkuh 4y ago
Too bad dreamwidth hosted blogs are behind cloudflare and set to block non-corporate (or older) browsers. Try to read this post from a Debian Jessie system and you'll just get a cloudflare page that doesn't work.
- jonathantf2 4y ago
  Reading this from a Debian bullseye system just fine.
- dvfjsdhgfv 4y ago
  This is very wrong. If someone at DW is reading this, please don't do that.
  - acdha 4y ago
    I’d want confirmation that it’s true first. People can configure custom UA blocking rules on Cloudflare but based on this I’d bet the problem is some custom configuration or plug-in interfering with the normal human activity challenge. https://www.webpagetest.org/result/220727_AiDcRG_FMH/ https://www.webpagetest.org/result/220727_AiDcRG_FMH/
- 1vuio0pswjnm7 4y ago
  Here is the response I got. the route "/11840.html" is not recognized Internet Archive works for Dreamwidth sites. For me, I add one line to a text file and the localhost forward proxy prefixes the URLs automatically. https://web.archive.org/web/20220719195142if_/https://diziet.dreamwidth.org/11840.html https://web.archive.org/web/20220719195142if_/https://diziet... FWIW, I use a non-corporate browser.
  - marttt 4y ago
    Thanks for your comment. I realized now that achive.org's "archive string" (here 20220719195142if_) is updated automatically. So if I use this string + some other URL, then I get redirected to a current snapshot of that other site, e.g. https://web.archive.org/web/20220719195142if_/http://ranprieur.com https://web.archive.org/web/20220719195142if_/http://ranprie... points me to https://web.archive.org/web/20220723235055if_/https://ranprieur.com https://web.archive.org/web/20220723235055if_/https://ranpri... I suppose the string consists of date + time in hhmmss format + if_? Anyhow, looks like arbitrary strings (e.g. 19991230225818if_) also get redirected to the next existing snapshot counting from that string. This is really nice and simple for text browser scripts. Is there some straightforward way to list all of archive.org's snapshots (of a particular site) without a javascript-enabled browser?
fanf2 4y ago
chiark hosts Simon Tatham’s homepage (including PuTTY, his puzzle collection, his notes on C preprocessing hacks, all of which often appear on this site) and my homepage (https://dotat.at https://dotat.at) amongst many other things. Here is Markus Kuhn commenting on my tweet about this article: https://twitter.com/mgk25/status/1550386849071390721 https://twitter.com/mgk25/status/1550386849071390721
dekhn 4y ago
I stopped using debian for a while after a botched upgrade (bo->hamm, some sort of libc4->glibc issue) and switched away. But that was almost 24 years ago, so I guess this system predated that.
- samtheprogram 4y ago
  What have you used since then? One mistake can really turn someone away for good, but also wondering if you have had another poor experience with whatever you (initially) replaced Debian with?
  - dekhn 4y ago
    Before debian I used TAMU distribution. After Debian I switched to Red Hat 5.something and stuck with it through the extremely weird Red Hat 9 era, which had its ups and downs. Next I switched to ubuntu and have mostly stuck with it, mainly because the packages are fairly well done and it gets an LTS every few years. I have dabbled with debian (missing some features in ubuntu), and several other OSes, but these days, i stick with what represents the least-effort path for dev compatibility (ubuntu). I admire arch but it's not the standard.
jancsika 4y ago
How far back could one start and still do a practical upgrade path on commodity hardware?
- lmm 4y ago
  The article says "chiark’s OS install dates to 1993, when I installed Debian 0.93R5, the first version of Debian to advertise the ability to be upgraded without reinstalling." So that's probably the starting point.
- jeroenhd 4y ago
  On the Intel side, Debian will not run on anything without the i686 instruction set. In theory this should include the Pentium II Pro and up. You can try it for yourself without the hardware, grab https://pcem-emulator.co.uk/index.html https://pcem-emulator.co.uk/index.html, set it up for a Pentium II with contemporary hardware and see if it works. It'll be slow as molasses but it should boot!
  - rahimnathwani 4y ago
    It seems like Debian Bullseye is available for the i386 architecture (i.e. Intel 486 and above): https://cdimage.debian.org/debian-cd/current/i386/iso-cd/ https://cdimage.debian.org/debian-cd/current/i386/iso-cd/
    - jeroenhd 4y ago
      It's listed as i386, but current Debian actually requires a more recent instruction set: > 2.1.2.1. CPU >Nearly all x86-based (IA-32) processors still in use in personal computers are supported. This also includes 32-bit AMD and VIA (former Cyrix) processors, and processors like the Athlon XP and Intel P4 Xeon. >However, Debian GNU/Linux bullseye will not run on 586 (Pentium) or earlier processors. (https://www.debian.org/releases/bullseye/i386/ch02s01.en.html https://www.debian.org/releases/bullseye/i386/ch02s01.en.htm...)
      - rahimnathwani 4y ago
Nux 4y ago
Ha, been visiting this famous machine for decades to download Putty. Small world. https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.ht...
- GekkePrutser 4y ago
  Yeah I was wondering whether it was that same chiark. I didn't have time to get to the end of the article. A bit worrying to hear that important security software was hosted on a 30 year old OS.
  - beermonster 4y ago
    The HN title is misleading as it was actually upgraded from Debian 8 "Jessie" to Bullseye. Debian 8 "jessie" support reached its end-of-life on June 30, 2020, five years after its initial release on April 26, 2015. However that does mean it was potentially vulnerable to critical vulnerabilities for more than two years whilst world+dog used it to download PuTTY for secure access to their servers. Eek Would be good to know how this was managed if at all.
    - adw 4y ago
      TFA covers this (extended security coverage from a third party provider).
    - blibble 4y ago
      I'd be more worried about the 200 shell accounts that machine has definitely been hacked
  - xnorswap 4y ago
    For many years it was only a http site and didn't even have https.
- tecleandor 4y ago
  Oooooh, the name sounded familiar but I didn't know why, as it's been some years since I used Windows regularly. Nice :)
deleted 4y ago
[deleted]
neilv 4y ago
As a Debian user for 20+ years, I've been impressed how well the upgrades of the `stable` major versions go for my workstations. (Servers just get rebuilt.)
whalesalad 4y ago
Knock on wood I’ve had tremendous success with big upgrades like this on Debian. It’s not the most glamorous distro but the stability is well worth it. My go to base VM is Debian 11. I usually use it for my base Dockerfile unless there’s a compelling reason not to. I tend to run Ubuntu AMI’s on EC2 but methinks I should give vanilla Debian a shot there too.
layer8 4y ago
I’ve been upgrading the same Debian VPS since 2009 (starting with Etch IIRC, currently on Bullseye as well), with very few issues. Merging configurations like for Exim is usually most of the work, and it’s often not even strictly necessary. Would love to read the INN story.
marcodiego 4y ago
Upgrading from Jessie to Bullseye compiling apt and having working using just apt and dpkg, that would be a feat.
antod 4y ago
I found that a 'refreshing' read - reveling in the old-school sysadmin wizardry, after so many years of livestock-over-pets thinking. This server sounds like the ultimate pet. It's just nostalgia though, I can't imagine managing systems that way (even at the less skilled level I had) any more. But at the same time it won't stop me admiring those skills - it's like watching a traditional craftsman using old tools.
- Nextgrid 4y ago
  I can't imagine managing lots of such systems, but a handful sounds doable, which is all you need sometimes. I'm currently speccing out a system for an internal application that processes huge amounts of data. So far the plan is to just use standard Postgres on Debian on a huge "pet" server with hundreds of GBs of RAM and redundant 4TB NVME drives and call it a day. It's sized for peak load so no need for any kind of scaling beyond that, and it's a single machine using well-tested software (and default configs whenever possible) so maintenance should be minimal (it'll also be isolated onto its own network and only ever accessed by trusted users so the need for timely updates is minimal too).
  - metadat 4y ago
    Do Debian package versions still lag far behind the current public stable version (i.e. by years)? Postgres gets solid updates seemingly bi-annually or at least annually.
    - Wowfunhappy 4y ago
      They should only be years behind given very unlucky timing, ie, upstream releases a new version right after the stable feature freeze. Whatever the latest version is at the time of Debian's feature freeze, that will be the version for the life of that Debian release. That's basically the point of Debian—the world will never change out from under you.
      - sigio 4y ago
        And with debian's release cycle being a lot shorter these days, most packages should be more then new enough. If needed you can get kernel-packages and stuff like browsers from backports or some 3rd party repositories tailored to bullseye.
pabs3 4y ago
I like that this included a cross-grade from i386 to amd64. I wonder if anyone has ever migrated an install across different CPU types, like x86 to Arm. I guess it would be easyish with multiarch, qemu-user-static and the new crossgrading tool. https://wiki.debian.org/CrossGrading https://wiki.debian.org/CrossGrading
- heywoodlh 4y ago
  Huh, thanks for this link! Very interesting read, did not know tooling like this existed (I assumed that crossgrading in the post was a manual/unsupported process).
  - pabs3 4y ago
    I think the crossgrading in the post was definitely a more manual process than the automated crossgrade tool, but yeah the tool is relatively new in Debian terms.
    - stevekemp 4y ago
      Yes I performed an i386 -> amd64 migration, of a Debian system, back in 2012: https://blog.steve.fi/today_i_migrated_from_32_bit_to_64_bit__in_place.html https://blog.steve.fi/today_i_migrated_from_32_bit_to_64_bit... It wasn't hugely difficult, but it required a bit of attention to details and I think some forced-replacements of packages.
jjice 4y ago
I like reading these kinds of stories because it's impressive to see system administration pros out there. For me though (and I've never had a server this old), container images that have a declarative build structure that I can look at and modify in pieces is such a blessing to have. I've had to wrestle with some weird config stuff that someone changed once without even noticing too many times. I'm a livestock kind of guy, but I respect those with pets :)
mjg59 4y ago
mgarrett@chiark:~$ file /lib/libc.so.4.6.27 /lib/libc.so.4.6.27: Linux/i386 demand-paged executable (QMAGIC), stripped mgarrett@chiark:~$ dpkg -S /lib/libc.so.4.6.27 libc4:i386: /lib/libc.so.4.6.27 Unfortunately the installed kernel has no support for a.out binaries, so nothing can actually run against it any more :(
- fanf2 4y ago
  There were (are?) some old a.out binaries lurking in /usr/local which occasionally caused amused remarks on irc when someone found one :-)
- hobo_mark 4y ago
  TIL, looks like a.out support was removed just a few months ago: https://news.ycombinator.com/item?id=30792059 https://news.ycombinator.com/item?id=30792059
kqr 4y ago
I actually had a similar upgrade to do that I've been postponing for too long. I ended up rebuilding all functionality on the server on a different server -- this time also creating the necessary automation for both provisioning and migrating, so I can rebuild it much cheaper next time. Did it take longer than slogging through a dist-upgrade? Probably. Does it cause some more inconvenience for users, like changed host keys and upgrades of software that might not have needed upgrades? Yup, definitely. Does it let me sleep well at night knowing I'm no longer dependent on a specific machine staying un-broken in the near future? Sure as hell. ---- (I also ended up splitting apart the independent functions the machine provides onto multiple machines, one for each purpose. This increases my exposure to machine failure, but simultaneously limits the blast radius. That's a good trade-off, for the most part.)
- forgetfulness 4y ago
  Did you use a configuration management system like Puppet or Chef, or did you roll your own with scripts?
  - kqr 4y ago
    Configuration management, mainly to make it slightly more cross-platform in case I would use a different Linux distro or whatnot next time around.
- m463 4y ago
  What has helped me most of all is just keeping a log - nothing special maybe readme.txt - of what I did when creating a system, or configuring a package.
denton-scratch 4y ago
I believe chiark is the server that is the home of PuTTY. This is an amusing article about getting a known-good copy of PuTTY. https://noncombatant.org/2014/03/03/downloading-software-safely-is-nearly-impossible/ https://noncombatant.org/2014/03/03/downloading-software-saf... (They have HTTPS now, so perhaps the problems are mitigated)
- tux2bsd 4y ago
  It was a matter of who gives a fuck, noone on windows that's for sure (as trained by Microsoft).
Too 4y ago
“recommendation to change the meaning of #!/usr/bin/python from Python 2, to Python 3.” This isn’t exactly true. The recommendation was for a long time that it should alias python2, later it changed to be ambiguous and up for distros to decide, which is possibly even worse, but I guess good in the long run once py2 gets completely eliminated. It can be found in https://peps.python.org/pep-0394/ https://peps.python.org/pep-0394/ Only way to be sure is to use more specific shebang or a venv, which is a good idea anyway. Especially when trying to run some really old sw, like in tfa, that is not up to date with other packages.
- bayindirh 4y ago
  In current state, Debian does not contain a /usr/bin/python file, and since 2.7 is eliminated from system relatively long time ago, everything is python3. If you really in dire need, you can install "python-is-python3", which just symlinks /usr/bin/python to /usr/bin/python3. However, no package can depend on it in Debian. It's strictly verboten.
deleted 4y ago
[deleted]
moomin 4y ago
I remember when chiark was chiark.culture.org. Yes, really, iwj was that far ahead of the curve in 1993.
codedokode 4y ago
I think that it would be better if I could use 64-bit kernel with 32-bit applications. 32-bit applications are better in almost every aspect: they use smaller ints and pointers and use less memory, they can be used on both 32-bit and 64-bit systems, they don't waste energy on computing extra bits. The only disadvantage is that they cannot use more than 2Gb of RAM, but almost every aplication I use doesn't need so much RAM. On x86, 32-bit apps also have slightly less GPRs. So the perfect solution would be a 64-bit kernel with mostly 32-bit applications except for those cases when you need more than 2Gb of RAM. But instead many developers refuse to provide 32-bit versions of applications for Linux, and provide worse 64-bit versions. And Linux distributions compile everything for 64-bit, even if the application is never supposed to use even 1 Gb of RAM. So everything is wrong with 64-bit support in Linux.
- bayindirh 4y ago
  You can multi-arch a distribution, also there's x64_32 (aka x32) which does exactly what you propose [0]. Running the processor in x64 mode allows wider swaths of data to be processed per clock, which really allows higher performance in data-heavy/CPU bound cases, but horses for courses, definitely. [0]: https://en.wikipedia.org/wiki/X32_ABI https://en.wikipedia.org/wiki/X32_ABI
  - codedokode 4y ago
    > You can multi-arch a distribution I use multi-arch, but it is buggy. Several times, when trying to install a 64-bit library, apt was suggesting to remove tens or hundreds packages to satisfy the dependencies. That included the packages that I have installed manually and it is strange that apt tries to remove them. Also, when using multi-arch, you can accidentally install a 32-bit kernel and then half of applications stop working.
  - ranma42 4y ago
    x64_32 never really took off and there was some discussion of removing the support in the past: https://lwn.net/Articles/774734/ https://lwn.net/Articles/774734/
rob_c 4y ago
I assume this involved imaging a disk or whipping it from one server into another because the amd64 spec didn't exist that far back...
BrandoElFollito 4y ago
I used to constantly upgrade Linux (usually Debian/Ubuntu, but also Fedora at some point) since 1994. It was an exercise I hated. Some 5 or 8 years ago I discovered Docker and it is truly a game changer for self hosted stuff. I do not care about the OS anymore, I can upgrade/reinstall in a breeze. I actually now run Arch to be independent of upgrades and just roll. if the thing breaks it is not a problem - I can reinstall and have everything running in an hour. This changed my life of a home sysadmin.
- jcynix 4y ago
  > I do not care about the OS anymore, I can upgrade/reinstall in a breeze. I actually now run Arch to be independent of upgrades and just roll. Interesting. Do you have any links to howtos or beginner's guides to such a setup?
  - BrandoElFollito 4y ago
    I do not have a documentation, but I do disaster recovery tests from time to time. This is how you can try it out: - download the ISO of a linux distribution, Arch is good because you have continuous updates (there is no "version") - start it on a VM engine (VirtualBox, Hyper-V on Windows, VMWare, ...) - from that point on - START DOCUMENTING - try to on docker install a program you know that is not too complex network-wise or just start with "hello-world" (https://hub.docker.com/_/hello-world https://hub.docker.com/_/hello-world) - you will find that when running "docker pull hello-world", docker is not installed - install docker on Arch according to Arch docs. DOCUMENT that step - retry, hello-world works - now try something like https://github.com/mendhak/docker-http-https-echo https://github.com/mendhak/docker-http-https-echo - you will learn the basics of docker networking, read some docs or just try until you have a curl call working - at that point you can try a program you know (nextcloud, syncting, ...), pulling it from docker hub and make it work. pay attention to two things: the network and the persistent volumes (I recommend, at least for the start, the file-based ones, not the docker ones) - grab a beer, you are 90% done, good work - have a close look at Caddy - this is a web server similar to apache, nginx but MUCH much better. So much better that I have no words. - you will use it as a proxy server for your containers, so that you can get to them via https://nextcloud.yourdomain.com https://nextcloud.yourdomain.com. It os worthwhile to get your domain even if you do not expose anything because things are much easier that way (caddy will manage the TLS part) - now learn docker-compose and add all your dockers to it (it is a YAML description of your containers). - add backup, this will be easier if you add this program on the OS itself (it can sure be in a container but I preferred having that part independent). I recommend Borg despite its few poor choices in the design (that are not likely to bite you at that point) TADAM! you are done. You are independent of the OS, if you want to install fedora or whatever it just doe snot matter because i) all your programs are maintained by someone else (than you to the maintainers, it is nice to donate sometimes) and ii) your backup is data that is easily pluggable back to a ne instance of the software Testing new programs is super easy (you just add them to the doclker compose YAML). I truly recommend you try with a VM and you will quickly realize it is time to reformat your server and put everything under docker :) AMA if you have questions. PS. The documentation you wrote (likely 10 lines or so) is now your DRP, you can test it from time to time just to make sure you can recover everything (= the data) from the backup
sexy_panda 4y ago
I just realised that updating an outdated Ubuntu 21.10 system is just kind of impossible without dirty hacks. I should have gone with bullseye from the beginning.
- tmottabr 4y ago
  You should be able to upgrade without any hacks. 21.10 reaching EOL just means that it stopped receiving updates, everything else should still work as is. Just install the update-manager-core and run "do-release-upgrade -d" as sudo.
  - sexy_panda 4y ago
    Well this didn't work, but this worked / is currently running: https://fedingo.com/how-to-install-or-upgrade-software-from-unsupported-release-in-ubuntu/ https://fedingo.com/how-to-install-or-upgrade-software-from-...

292 points by jklinger410 4y ago

Slightly clickbaity: "last major OS upgrade was to jessie (Debian 8, released in April 2015). That was in 2016." "after nearly 30 years running Debian i386" But impressive nonetheless!

joeyh 4y ago
The actual title is not the inspid word soup deposited atop this thread.
- bscphil 4y ago
  Yeah @dang this title is terrible. Maybe "Upgrading from x86 Debian Jessie directly to x64 Bullseye" would be better?
- jklinger410 4y ago
  The original title is: chiark’s skip-skip-cross-up-grade Since that isn't very informative, I tried to simplify the first sentence: Two weeks ago I upgraded chiark from Debian jessie i386 to bullseye amd64, after nearly 30 years running Debian i386. This went really quite well, in fact! Realizing now the feet is going from i386 to amd64, rather than an old debian version upgrade. Unfortunately I can't change the title.
GuB-42 4y ago
Clickbait indeed, I actually clicked because I knew it was wrong. The true story is that the server started in 1993 as Debian 0.93R5, getting upgrades from time to time, the last one being Jessie to Bullseye. I expected some bullshit or parody, but here, it is genuinely impressive.
josephcsible 4y ago
But i386 is the architecture, and until now, none of the upgrades have changed that.
rob_c 4y ago
Yeah, my big feeling is it's become a philosophical question as to where it's the same server after so many updates/upgrades.

hprotagonist 4y ago

a domain name i know primarily from about 20 years of downloading PuTTY and friends …

superkuh 4y ago

Too bad dreamwidth hosted blogs are behind cloudflare and set to block non-corporate (or older) browsers. Try to read this post from a Debian Jessie system and you'll just get a cloudflare page that doesn't work.

jonathantf2 4y ago
Reading this from a Debian bullseye system just fine.
dvfjsdhgfv 4y ago
This is very wrong. If someone at DW is reading this, please don't do that.
- acdha 4y ago
  I’d want confirmation that it’s true first. People can configure custom UA blocking rules on Cloudflare but based on this I’d bet the problem is some custom configuration or plug-in interfering with the normal human activity challenge. https://www.webpagetest.org/result/220727_AiDcRG_FMH/ https://www.webpagetest.org/result/220727_AiDcRG_FMH/
1vuio0pswjnm7 4y ago
Here is the response I got. the route "/11840.html" is not recognized Internet Archive works for Dreamwidth sites. For me, I add one line to a text file and the localhost forward proxy prefixes the URLs automatically. https://web.archive.org/web/20220719195142if_/https://diziet.dreamwidth.org/11840.html https://web.archive.org/web/20220719195142if_/https://diziet... FWIW, I use a non-corporate browser.
- marttt 4y ago
  Thanks for your comment. I realized now that achive.org's "archive string" (here 20220719195142if_) is updated automatically. So if I use this string + some other URL, then I get redirected to a current snapshot of that other site, e.g. https://web.archive.org/web/20220719195142if_/http://ranprieur.com https://web.archive.org/web/20220719195142if_/http://ranprie... points me to https://web.archive.org/web/20220723235055if_/https://ranprieur.com https://web.archive.org/web/20220723235055if_/https://ranpri... I suppose the string consists of date + time in hhmmss format + if_? Anyhow, looks like arbitrary strings (e.g. 19991230225818if_) also get redirected to the next existing snapshot counting from that string. This is really nice and simple for text browser scripts. Is there some straightforward way to list all of archive.org's snapshots (of a particular site) without a javascript-enabled browser?

fanf2 4y ago

chiark hosts Simon Tatham’s homepage (including PuTTY, his puzzle collection, his notes on C preprocessing hacks, all of which often appear on this site) and my homepage (https://dotat.at https://dotat.at) amongst many other things. Here is Markus Kuhn commenting on my tweet about this article: https://twitter.com/mgk25/status/1550386849071390721 https://twitter.com/mgk25/status/1550386849071390721

dekhn 4y ago

I stopped using debian for a while after a botched upgrade (bo->hamm, some sort of libc4->glibc issue) and switched away. But that was almost 24 years ago, so I guess this system predated that.

samtheprogram 4y ago
What have you used since then? One mistake can really turn someone away for good, but also wondering if you have had another poor experience with whatever you (initially) replaced Debian with?
- dekhn 4y ago
  Before debian I used TAMU distribution. After Debian I switched to Red Hat 5.something and stuck with it through the extremely weird Red Hat 9 era, which had its ups and downs. Next I switched to ubuntu and have mostly stuck with it, mainly because the packages are fairly well done and it gets an LTS every few years. I have dabbled with debian (missing some features in ubuntu), and several other OSes, but these days, i stick with what represents the least-effort path for dev compatibility (ubuntu). I admire arch but it's not the standard.

jancsika 4y ago

How far back could one start and still do a practical upgrade path on commodity hardware?

lmm 4y ago
The article says "chiark’s OS install dates to 1993, when I installed Debian 0.93R5, the first version of Debian to advertise the ability to be upgraded without reinstalling." So that's probably the starting point.
jeroenhd 4y ago
On the Intel side, Debian will not run on anything without the i686 instruction set. In theory this should include the Pentium II Pro and up. You can try it for yourself without the hardware, grab https://pcem-emulator.co.uk/index.html https://pcem-emulator.co.uk/index.html, set it up for a Pentium II with contemporary hardware and see if it works. It'll be slow as molasses but it should boot!
- rahimnathwani 4y ago
  It seems like Debian Bullseye is available for the i386 architecture (i.e. Intel 486 and above): https://cdimage.debian.org/debian-cd/current/i386/iso-cd/ https://cdimage.debian.org/debian-cd/current/i386/iso-cd/
  - jeroenhd 4y ago
    It's listed as i386, but current Debian actually requires a more recent instruction set: > 2.1.2.1. CPU >Nearly all x86-based (IA-32) processors still in use in personal computers are supported. This also includes 32-bit AMD and VIA (former Cyrix) processors, and processors like the Athlon XP and Intel P4 Xeon. >However, Debian GNU/Linux bullseye will not run on 586 (Pentium) or earlier processors. (https://www.debian.org/releases/bullseye/i386/ch02s01.en.html https://www.debian.org/releases/bullseye/i386/ch02s01.en.htm...)
    - rahimnathwani 4y ago

Nux 4y ago

Ha, been visiting this famous machine for decades to download Putty. Small world. https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.ht...

GekkePrutser 4y ago
Yeah I was wondering whether it was that same chiark. I didn't have time to get to the end of the article. A bit worrying to hear that important security software was hosted on a 30 year old OS.
- beermonster 4y ago
  The HN title is misleading as it was actually upgraded from Debian 8 "Jessie" to Bullseye. Debian 8 "jessie" support reached its end-of-life on June 30, 2020, five years after its initial release on April 26, 2015. However that does mean it was potentially vulnerable to critical vulnerabilities for more than two years whilst world+dog used it to download PuTTY for secure access to their servers. Eek Would be good to know how this was managed if at all.
  - adw 4y ago
    TFA covers this (extended security coverage from a third party provider).
  - blibble 4y ago
    I'd be more worried about the 200 shell accounts that machine has definitely been hacked
- xnorswap 4y ago
  For many years it was only a http site and didn't even have https.
tecleandor 4y ago
Oooooh, the name sounded familiar but I didn't know why, as it's been some years since I used Windows regularly. Nice :)

deleted 4y ago

[deleted]

neilv 4y ago

As a Debian user for 20+ years, I've been impressed how well the upgrades of the `stable` major versions go for my workstations. (Servers just get rebuilt.)

whalesalad 4y ago

Knock on wood I’ve had tremendous success with big upgrades like this on Debian. It’s not the most glamorous distro but the stability is well worth it. My go to base VM is Debian 11. I usually use it for my base Dockerfile unless there’s a compelling reason not to. I tend to run Ubuntu AMI’s on EC2 but methinks I should give vanilla Debian a shot there too.

layer8 4y ago

I’ve been upgrading the same Debian VPS since 2009 (starting with Etch IIRC, currently on Bullseye as well), with very few issues. Merging configurations like for Exim is usually most of the work, and it’s often not even strictly necessary. Would love to read the INN story.

marcodiego 4y ago

Upgrading from Jessie to Bullseye compiling apt and having working using just apt and dpkg, that would be a feat.

antod 4y ago

I found that a 'refreshing' read - reveling in the old-school sysadmin wizardry, after so many years of livestock-over-pets thinking. This server sounds like the ultimate pet. It's just nostalgia though, I can't imagine managing systems that way (even at the less skilled level I had) any more. But at the same time it won't stop me admiring those skills - it's like watching a traditional craftsman using old tools.

Nextgrid 4y ago
I can't imagine managing lots of such systems, but a handful sounds doable, which is all you need sometimes. I'm currently speccing out a system for an internal application that processes huge amounts of data. So far the plan is to just use standard Postgres on Debian on a huge "pet" server with hundreds of GBs of RAM and redundant 4TB NVME drives and call it a day. It's sized for peak load so no need for any kind of scaling beyond that, and it's a single machine using well-tested software (and default configs whenever possible) so maintenance should be minimal (it'll also be isolated onto its own network and only ever accessed by trusted users so the need for timely updates is minimal too).
- metadat 4y ago
  Do Debian package versions still lag far behind the current public stable version (i.e. by years)? Postgres gets solid updates seemingly bi-annually or at least annually.
  - Wowfunhappy 4y ago
    They should only be years behind given very unlucky timing, ie, upstream releases a new version right after the stable feature freeze. Whatever the latest version is at the time of Debian's feature freeze, that will be the version for the life of that Debian release. That's basically the point of Debian—the world will never change out from under you.
    - sigio 4y ago
      And with debian's release cycle being a lot shorter these days, most packages should be more then new enough. If needed you can get kernel-packages and stuff like browsers from backports or some 3rd party repositories tailored to bullseye.

pabs3 4y ago

I like that this included a cross-grade from i386 to amd64. I wonder if anyone has ever migrated an install across different CPU types, like x86 to Arm. I guess it would be easyish with multiarch, qemu-user-static and the new crossgrading tool. https://wiki.debian.org/CrossGrading https://wiki.debian.org/CrossGrading

heywoodlh 4y ago
Huh, thanks for this link! Very interesting read, did not know tooling like this existed (I assumed that crossgrading in the post was a manual/unsupported process).
- pabs3 4y ago
  I think the crossgrading in the post was definitely a more manual process than the automated crossgrade tool, but yeah the tool is relatively new in Debian terms.
  - stevekemp 4y ago
    Yes I performed an i386 -> amd64 migration, of a Debian system, back in 2012: https://blog.steve.fi/today_i_migrated_from_32_bit_to_64_bit__in_place.html https://blog.steve.fi/today_i_migrated_from_32_bit_to_64_bit... It wasn't hugely difficult, but it required a bit of attention to details and I think some forced-replacements of packages.

jjice 4y ago

I like reading these kinds of stories because it's impressive to see system administration pros out there. For me though (and I've never had a server this old), container images that have a declarative build structure that I can look at and modify in pieces is such a blessing to have. I've had to wrestle with some weird config stuff that someone changed once without even noticing too many times. I'm a livestock kind of guy, but I respect those with pets :)

mjg59 4y ago

mgarrett@chiark:~$ file /lib/libc.so.4.6.27 /lib/libc.so.4.6.27: Linux/i386 demand-paged executable (QMAGIC), stripped mgarrett@chiark:~$ dpkg -S /lib/libc.so.4.6.27 libc4:i386: /lib/libc.so.4.6.27 Unfortunately the installed kernel has no support for a.out binaries, so nothing can actually run against it any more :(

fanf2 4y ago
There were (are?) some old a.out binaries lurking in /usr/local which occasionally caused amused remarks on irc when someone found one :-)
hobo_mark 4y ago
TIL, looks like a.out support was removed just a few months ago: https://news.ycombinator.com/item?id=30792059 https://news.ycombinator.com/item?id=30792059

kqr 4y ago

I actually had a similar upgrade to do that I've been postponing for too long. I ended up rebuilding all functionality on the server on a different server -- this time also creating the necessary automation for both provisioning and migrating, so I can rebuild it much cheaper next time. Did it take longer than slogging through a dist-upgrade? Probably. Does it cause some more inconvenience for users, like changed host keys and upgrades of software that might not have needed upgrades? Yup, definitely. Does it let me sleep well at night knowing I'm no longer dependent on a specific machine staying un-broken in the near future? Sure as hell. ---- (I also ended up splitting apart the independent functions the machine provides onto multiple machines, one for each purpose. This increases my exposure to machine failure, but simultaneously limits the blast radius. That's a good trade-off, for the most part.)

forgetfulness 4y ago
Did you use a configuration management system like Puppet or Chef, or did you roll your own with scripts?
- kqr 4y ago
  Configuration management, mainly to make it slightly more cross-platform in case I would use a different Linux distro or whatnot next time around.
m463 4y ago
What has helped me most of all is just keeping a log - nothing special maybe readme.txt - of what I did when creating a system, or configuring a package.

denton-scratch 4y ago

I believe chiark is the server that is the home of PuTTY. This is an amusing article about getting a known-good copy of PuTTY. https://noncombatant.org/2014/03/03/downloading-software-safely-is-nearly-impossible/ https://noncombatant.org/2014/03/03/downloading-software-saf... (They have HTTPS now, so perhaps the problems are mitigated)

tux2bsd 4y ago
It was a matter of who gives a fuck, noone on windows that's for sure (as trained by Microsoft).

Too 4y ago

“recommendation to change the meaning of #!/usr/bin/python from Python 2, to Python 3.” This isn’t exactly true. The recommendation was for a long time that it should alias python2, later it changed to be ambiguous and up for distros to decide, which is possibly even worse, but I guess good in the long run once py2 gets completely eliminated. It can be found in https://peps.python.org/pep-0394/ https://peps.python.org/pep-0394/ Only way to be sure is to use more specific shebang or a venv, which is a good idea anyway. Especially when trying to run some really old sw, like in tfa, that is not up to date with other packages.

bayindirh 4y ago
In current state, Debian does not contain a /usr/bin/python file, and since 2.7 is eliminated from system relatively long time ago, everything is python3. If you really in dire need, you can install "python-is-python3", which just symlinks /usr/bin/python to /usr/bin/python3. However, no package can depend on it in Debian. It's strictly verboten.

deleted 4y ago

[deleted]

moomin 4y ago

I remember when chiark was chiark.culture.org. Yes, really, iwj was that far ahead of the curve in 1993.

codedokode 4y ago

I think that it would be better if I could use 64-bit kernel with 32-bit applications. 32-bit applications are better in almost every aspect: they use smaller ints and pointers and use less memory, they can be used on both 32-bit and 64-bit systems, they don't waste energy on computing extra bits. The only disadvantage is that they cannot use more than 2Gb of RAM, but almost every aplication I use doesn't need so much RAM. On x86, 32-bit apps also have slightly less GPRs. So the perfect solution would be a 64-bit kernel with mostly 32-bit applications except for those cases when you need more than 2Gb of RAM. But instead many developers refuse to provide 32-bit versions of applications for Linux, and provide worse 64-bit versions. And Linux distributions compile everything for 64-bit, even if the application is never supposed to use even 1 Gb of RAM. So everything is wrong with 64-bit support in Linux.

bayindirh 4y ago
You can multi-arch a distribution, also there's x64_32 (aka x32) which does exactly what you propose [0]. Running the processor in x64 mode allows wider swaths of data to be processed per clock, which really allows higher performance in data-heavy/CPU bound cases, but horses for courses, definitely. [0]: https://en.wikipedia.org/wiki/X32_ABI https://en.wikipedia.org/wiki/X32_ABI
- codedokode 4y ago
  > You can multi-arch a distribution I use multi-arch, but it is buggy. Several times, when trying to install a 64-bit library, apt was suggesting to remove tens or hundreds packages to satisfy the dependencies. That included the packages that I have installed manually and it is strange that apt tries to remove them. Also, when using multi-arch, you can accidentally install a 32-bit kernel and then half of applications stop working.
- ranma42 4y ago
  x64_32 never really took off and there was some discussion of removing the support in the past: https://lwn.net/Articles/774734/ https://lwn.net/Articles/774734/

rob_c 4y ago

I assume this involved imaging a disk or whipping it from one server into another because the amd64 spec didn't exist that far back...

BrandoElFollito 4y ago

I used to constantly upgrade Linux (usually Debian/Ubuntu, but also Fedora at some point) since 1994. It was an exercise I hated. Some 5 or 8 years ago I discovered Docker and it is truly a game changer for self hosted stuff. I do not care about the OS anymore, I can upgrade/reinstall in a breeze. I actually now run Arch to be independent of upgrades and just roll. if the thing breaks it is not a problem - I can reinstall and have everything running in an hour. This changed my life of a home sysadmin.

jcynix 4y ago
> I do not care about the OS anymore, I can upgrade/reinstall in a breeze. I actually now run Arch to be independent of upgrades and just roll. Interesting. Do you have any links to howtos or beginner's guides to such a setup?
- BrandoElFollito 4y ago
  I do not have a documentation, but I do disaster recovery tests from time to time. This is how you can try it out: - download the ISO of a linux distribution, Arch is good because you have continuous updates (there is no "version") - start it on a VM engine (VirtualBox, Hyper-V on Windows, VMWare, ...) - from that point on - START DOCUMENTING - try to on docker install a program you know that is not too complex network-wise or just start with "hello-world" (https://hub.docker.com/_/hello-world https://hub.docker.com/_/hello-world) - you will find that when running "docker pull hello-world", docker is not installed - install docker on Arch according to Arch docs. DOCUMENT that step - retry, hello-world works - now try something like https://github.com/mendhak/docker-http-https-echo https://github.com/mendhak/docker-http-https-echo - you will learn the basics of docker networking, read some docs or just try until you have a curl call working - at that point you can try a program you know (nextcloud, syncting, ...), pulling it from docker hub and make it work. pay attention to two things: the network and the persistent volumes (I recommend, at least for the start, the file-based ones, not the docker ones) - grab a beer, you are 90% done, good work - have a close look at Caddy - this is a web server similar to apache, nginx but MUCH much better. So much better that I have no words. - you will use it as a proxy server for your containers, so that you can get to them via https://nextcloud.yourdomain.com https://nextcloud.yourdomain.com. It os worthwhile to get your domain even if you do not expose anything because things are much easier that way (caddy will manage the TLS part) - now learn docker-compose and add all your dockers to it (it is a YAML description of your containers). - add backup, this will be easier if you add this program on the OS itself (it can sure be in a container but I preferred having that part independent). I recommend Borg despite its few poor choices in the design (that are not likely to bite you at that point) TADAM! you are done. You are independent of the OS, if you want to install fedora or whatever it just doe snot matter because i) all your programs are maintained by someone else (than you to the maintainers, it is nice to donate sometimes) and ii) your backup is data that is easily pluggable back to a ne instance of the software Testing new programs is super easy (you just add them to the doclker compose YAML). I truly recommend you try with a VM and you will quickly realize it is time to reformat your server and put everything under docker :) AMA if you have questions. PS. The documentation you wrote (likely 10 lines or so) is now your DRP, you can test it from time to time just to make sure you can recover everything (= the data) from the backup

sexy_panda 4y ago

I just realised that updating an outdated Ubuntu 21.10 system is just kind of impossible without dirty hacks. I should have gone with bullseye from the beginning.

tmottabr 4y ago
You should be able to upgrade without any hacks. 21.10 reaching EOL just means that it stopped receiving updates, everything else should still work as is. Just install the update-manager-core and run "do-release-upgrade -d" as sudo.
- sexy_panda 4y ago
  Well this didn't work, but this worked / is currently running: https://fedingo.com/how-to-install-or-upgrade-software-from-unsupported-release-in-ubuntu/ https://fedingo.com/how-to-install-or-upgrade-software-from-...