8 ms·

How does Linux handle writes?

150 points by dmazin 3y ago

sltkr 3y ago
As a corollary, it's not possible to read a single byte from a file (with regular usermode I/O). When you call e.g. `read(fd, buf, 1)` the Linux kernel will read an entire 4096 page into the page table cache, and then copy 1 byte from that into your buffer. This might sound inefficient but in practice consecutive reads are so much more efficient than random reads that the difference between reading 1 byte and 4096 bytes is negligible when the data is stored consecutively. Consequently, when designing on-disk data structures that are accessed randomly (like Btree pages in a database, for example), it usually doesn't make sense to make them smaller than 4 KB. (It is possible to bypass the page table cache using O_DIRECT, at least for some filesystems, but that has its own size and alignment restrictions; typically you can only read 512 byte blocks.)
- cogman10 3y ago
  This is true even when talking about reading a byte from memory. It's way faster to fill a cache line with 64bytes in one go than it is to make 64 1 byte requests to main memory.
  - sltkr 3y ago
    Yes, good point!
9735194 3y ago
We learnt that at school back in 1994. Don’t they teach kids about how computers work anymore?
- cogman10 3y ago
  Surprisingly, no. These foundational things are pretty frequently skipped or glossed over. There's a lot of stuff above and beyond that needs to be covered. For example, in 1994, you probably didn't spend a whole lot of time talking about concurrency or parallelism. Most software didn't need to think about those things. But beyond that, kids these days have a very different relationship with computers than we did. They very VERY rarely interact with the filesystem (in my day, that was everything). Instead, it's a slick UX with apps to tell you everything to do. Back in my day, even fairly novice computer users were familiar with the idea of mounting/unmounting stuff because they'd pop in a 3 1/4 floppy write stuff there, and pop it out. Formatting a floppy was a fairly common occurrence (for some reason). Things are just different. On the one hand information about this stuff is WAY more available than it ever was. On the other, there's a lot more information and things are constantly changing. You can see that in old games like starcraft. There, the authors had things like their own custom linked list implementation. You'd pretty much never see something like that in modern software. You'd get funny looks with people saying "Why aren't you using a library for that? There's `ac`"
stefantalpalaru 3y ago
[dead]
knorker 3y ago
> This is called non-blocking IO No, that's not quite right. It happened not to block this time, but the write was not "non-blocking". It could have blocked.
ivolimmen 3y ago
As interesting as the article is I am really distracted by the fact that the headers have no spacing before the next paragraph begins.
- dmazin 3y ago
  Thanks for pointing this out. I'll fix it!
jchw 3y ago
It's certainly good to understand that there is caching, journaling, etc. though I think that most people realize it by now. It's very easy to notice at least part of this by removing a USB stick without unmounting it on Linux :) However, if you're one of today's lucky 10,000, welcome to the curse of knowledge. On that note, I can only imagine the war stories that PostgreSQL developers could tell about disk syncing. > How slow? An SSD is 1,000 times slower than memory. A spinning hard drive is one MILLION times slower. Disks are multiple orders of magnitude slower than memory! Latency? Throughput? I don't think SSDs are 1,000 times slower than RAM in either category, assuming fairly typical consumer machines, but I could be wrong.
- dmazin 3y ago
  Thank you for engaging. Also, thank you for finding that imprecise language. An original draft had stated that I'm talking about I/O duration, but I lost that language. I will re-add it. My numbers are for "main memory access", "SSD I/O" and "rotational disk I/O" from Gregg System Performance (chapter 2). In retrospect, with all respect to Gregg, that wording is itself pretty imprecise. I think what really matters here is I/O duration rather than throughput. That is, the reason we use the cache so much is because a single I/O is really slow against disks...
  - jchw 3y ago
    Yeah, that's true. Raw speeds don't really mean a whole lot without more context, and in practice writing to disk is going to be a lot slower even if the peak throughputs are not quite that far off. The performance characteristics are quite different in practice. I assume at one point in history though, these numbers were closer to true even for raw throughput... PCIe and NVMe have probably helped close the gap a bit, I would guess.
- marginalia_nu 3y ago
  > Latency? Throughput? I, don't think SSDs are 1,000 times slower than RAM in either category, assuming fairly typical consumer machines, but I could be wrong. Everything in this business is like a great big onion of caching and buffering layers. Disks have them too. This is why you often see consumer SSDs have very fast writes only as long as you don't write continuously. Continuous writes causes their internal buffers to fill up and the real write performance to be revealed. There's also factors like write amplification and overprovisioning that enter into this set of equations. But if there's any takeaway from this, it's that I/O benchmarking is hell.
seeknotfind 3y ago
This article had an extremely positive tone, compared to what I expected from the URL. It's refreshing to hear the author's excitement about filesystems, and I learned a couple of things. Thanks.
bpoyner 3y ago
And specifically when you tell the shell to redirect stdout to a file, the shell is opening a file descriptor to handle the request. A file descriptor also sits in the middle for piping stdout for one command to stdin of another. It seems like it's doing magic, but it's not.
- passion__desire 3y ago
  Are we continuously told in our courses to flush() the file so that it gets written to disk?
  - bcrl 3y ago
    If you're talking about fflush() from the C standard libray, fflush() doesn't write to disk. It pushes writes from the FILE's buffer to the kernel and drops any read data that has been cached. The C standard library doesn't really provide you with all the tools needed to commit writes to disk fully.
  - 95014_refugee 3y ago
    Hopefully not, because that murders performance. The caching exists for a reason, let it do its thing. If you care about data integrity, you don’t trust the disk anyway, so flushing to the disk isn’t enough.
    - Filligree 3y ago
      There's fsync. Unfortunately, if you want atomicity but not durability you have ~zero options on common filesystems. (On ZFS and bcachefs you can get that by disabling syncs completely; they never reorder writes in a visible manner.)
scarygliders 3y ago
In ye olde unixe days, I learned a habit. That habit was; # sync # sync # sync # halt When powering down ye olde unixe maschine. It was so far back I just vaguely remember I'd read that somewhere; possibly in a ye olde unixe manual, but I can't remember exactly. I think back then it made sense, due to older spinning rust technology and there might have been some possibility of the OS not flushing any waiting cache to disk before powering down. Even though it's probably not needed today, I still do throw a sync command in when powering down, say, my NAS with spinning rust drives, and my PCs/laptop with SSDs. There is one thing where I 100% perform a couple of sync commands, and that's right before unmounting a USB memory stick - I usually witness linux Very Quickly Copying a file from PC to USB stick. "Aha! No, you're just caching that operation you sly dog you!" - the sync command on a CLI tells me when linux has finally actually copied the file onto the stick.
- fullstop 3y ago
  I still do this. There was another command which was used to park the heads, too.
  - sroussey 3y ago
    Ah yes! And park the heads before opening up the server or moving it. Absolute must.
    - JdeBP 3y ago
      FXPARK.EXE! I'm getting flashbacks of Tandon DOS! Aaargh!
- dpifke 3y ago
  A coworker once asked me for shell tips. I gave them a few that came to mind, and also invited them to peruse my .bash_history, because I figured there might be other tricks hiding in there. They later asked why I sync all the time, almost always in threes. "Huh?" Looking through history, I see that I do. Apparently it's been ingrained into my subconscious, like a sort of stutter, and I type it sometimes out of habit when I'm deep in thought. Old habits really do die hard.
mega_dingus 3y ago
This isn't specific to Linux Raymond Chen has a tangentially amusing story that I just found while trying to google the win95 screen: https://devblogs.microsoft.com/oldnewthing/20160419-00/?p=93315 https://devblogs.microsoft.com/oldnewthing/20160419-00/?p=93...
taeric 3y ago
It helps me to expand the concept of what is happening. Instead of "asking the computer to write a file," it can be seen as "asking the computer to coordinate some information from keyboard, to memory, to disk." You can then expand that to constituent parts. "From memory, to the disk controller, to disk." In this way "disk controller" can easily expand to S3 or some such. And you can begin to enumerate all of the things that have to happen for that to work. (And yes, you could even expand that some, as there is a memory controller and many other things you may want to think about.) And buffers for efficiency aren't new to computers. Consider returning a book to a library. You drop it off in the return box. Eventually, a worker moves it from returns to processing, where they will then put it on a shelving queue. Where it will eventually be carted around and reshelved. Odds are high that you have some form of buffering before it gets to the return box, even.
- pjc50 3y ago
  > In this way "disk controller" can easily expand to S3 or some such. Disk semantics are subtly different from S3-style blob semantics in important ways. Crucially the immutability of blobs makes life _much_ easier for the blob storage than the ability to write into the middle of a disk file. Given the latency problem, I think it would be interesting and useful for there to be devices which expose a "blob" API to a bunch of Flash on a PCIe bus, and for there to be a suitable OS API for this. Flash itself is only weakly mutable! You can only write in one direction (usually 1->0) to a Flash cell. To go 0->1 you have to do a block erase on a whole bunch of bits, which takes much longer.
  - wtallis 3y ago
    > I think it would be interesting and useful for there to be devices which expose a "blob" API to a bunch of Flash on a PCIe bus, and for there to be a suitable OS API for this. We're partly there. NVMe has standardized a key-value interface for SSDs, but as with any compatibility-breaking new storage interface it's pretty much only used by and available to large cloud computing companies that can afford to rewrite their entire storage stack to take advantage of it.
  - taeric 3y ago
charcircuit 3y ago
This behavior is up to the file system or virtual file system implementation. Not all writes on Linux use a page cache. It's misleading to tell people that there is always a page cache involved.
- dmazin 3y ago
  Originally, I was going to explain that I was discussing ext4 specifically but, honestly, can you name a common file system where the page cache isn't involved by default?
  - charcircuit 3y ago
    NFS
SteveNuts 3y ago
I don’t remember the last time I actually lost any data when a machine loses power. Am I just lucky or is there more to the story for this comment: > What happens if you unplug your computer before the operating system writes the data to disk? Well, you will lose the data. It’s as simple as that.
- scbrg 3y ago
  It absolutely happens. My guess is that your machines just don't actually lose power very often. Any normal desk- or laptop won't. When you hit the power button, it doesn't just cut the power; the system is told to shut down so that it can neatly terminate processes and unmount filesystems before halting. Takes a few seconds. At work I'm writing software for industrial machines. It's essentially a Linux system in kiosk mode running on semi custom hardware. We actually do have a power button that simply shuts off power (much to my dismay), and we have data loss all the time. We just make sure that whenever there's actually important data that needs to be stored (very seldom) we make it blatantly obvious to the user that now is a very bad time to push the power button, and sync before those warnings are removed. But we've had boxes that were rendered unbootable because someone decided to push the power button the moment an apt-get upgrade completed and a lot of updated files were still living in memory only.
- sltkr 3y ago
  There are several reasons why it's actually really unlikely to lose important data when you lose power on modern systems. Firstly, modern filesystems are designed to be robust to random power loss, so it's unlikely that you will corrupt your filesystem in a way that cannot be recovered from. It does mean that some unflushed data would not be written, but it will only affect files that were opened for writing at or shortly before your system lost power. Some filesystems (including ext4) can offer quite strong consistency guarantees if you configure them correctly, though usually stronger consistency means lower performance in some scenarios. Secondly, the Linux regularly flushes dirty pages to disk anyway (15 seconds on my system, see `cat /proc/sys/vm/dirty_expire_centisecs` for yours) so you cannot lose more than a few seconds of data in the worst case. This is particularly important for the case of "oh, I forgot to unmount the USB stick before pulling it out of my laptop". Thirdly, most applications guard against data loss by using some form of write-ahead logging and explicit syncing. For example, if you edit a file in vim and then exit with `wq!` it will call fsync() before exiting, which causes the contents of the file to be flushed to disk immediately. In theory it's possibly that you lose power in the middle of the fsync() call, but it's extremely unlikely. (Of course it would be just as likely that you lose power right before issuing the write-and-quit command.) The sort of data that is not explicitly synced is where it's not really important to keep the latest information. Think about log files that are appended to continuously, or for example downloads in browsers. Do you really care if you have to restart an incomplete download after a sudden power loss? Probably not.
amelius 3y ago
All this talk about why writing files is lazy. But why isn't booting lazy? Why do I have to wait for the network interface to boot even if I don't really intend to use it right now?
- wmf 3y ago
  systemd does this with socket-activated units.
  - amelius 3y ago
    It doesn't work very well, though. Booting into a tty login prompt should be possible in well under 1 second. On a modern Linux system that uses systemd, however, it still takes on the order of a minute or more.
    - JdeBP 3y ago
      There's a lot of seemingly purely local infrastructure that nowadays depends from networking. login calls into a PAM module, which does database lookups, which goes through nsswitch and under the covers tries to consult nscd or systemd-resolved or some such, which either have network timeouts to tick down or aren't started until boot network interfaces are configured .... Mind you, there was the same issue with NIS.
      - amelius 3y ago
        Yes, I guess the problem is that laziness shouldn't stop at service boundaries.

150 points by dmazin 3y ago

As a corollary, it's not possible to read a single byte from a file (with regular usermode I/O). When you call e.g. `read(fd, buf, 1)` the Linux kernel will read an entire 4096 page into the page table cache, and then copy 1 byte from that into your buffer. This might sound inefficient but in practice consecutive reads are so much more efficient than random reads that the difference between reading 1 byte and 4096 bytes is negligible when the data is stored consecutively. Consequently, when designing on-disk data structures that are accessed randomly (like Btree pages in a database, for example), it usually doesn't make sense to make them smaller than 4 KB. (It is possible to bypass the page table cache using O_DIRECT, at least for some filesystems, but that has its own size and alignment restrictions; typically you can only read 512 byte blocks.)

cogman10 3y ago
This is true even when talking about reading a byte from memory. It's way faster to fill a cache line with 64bytes in one go than it is to make 64 1 byte requests to main memory.
- sltkr 3y ago
  Yes, good point!

9735194 3y ago

We learnt that at school back in 1994. Don’t they teach kids about how computers work anymore?

cogman10 3y ago
Surprisingly, no. These foundational things are pretty frequently skipped or glossed over. There's a lot of stuff above and beyond that needs to be covered. For example, in 1994, you probably didn't spend a whole lot of time talking about concurrency or parallelism. Most software didn't need to think about those things. But beyond that, kids these days have a very different relationship with computers than we did. They very VERY rarely interact with the filesystem (in my day, that was everything). Instead, it's a slick UX with apps to tell you everything to do. Back in my day, even fairly novice computer users were familiar with the idea of mounting/unmounting stuff because they'd pop in a 3 1/4 floppy write stuff there, and pop it out. Formatting a floppy was a fairly common occurrence (for some reason). Things are just different. On the one hand information about this stuff is WAY more available than it ever was. On the other, there's a lot more information and things are constantly changing. You can see that in old games like starcraft. There, the authors had things like their own custom linked list implementation. You'd pretty much never see something like that in modern software. You'd get funny looks with people saying "Why aren't you using a library for that? There's `ac`"

stefantalpalaru 3y ago

[dead]

knorker 3y ago

> This is called non-blocking IO No, that's not quite right. It happened not to block this time, but the write was not "non-blocking". It could have blocked.

ivolimmen 3y ago

As interesting as the article is I am really distracted by the fact that the headers have no spacing before the next paragraph begins.

dmazin 3y ago
Thanks for pointing this out. I'll fix it!

jchw 3y ago

It's certainly good to understand that there is caching, journaling, etc. though I think that most people realize it by now. It's very easy to notice at least part of this by removing a USB stick without unmounting it on Linux :) However, if you're one of today's lucky 10,000, welcome to the curse of knowledge. On that note, I can only imagine the war stories that PostgreSQL developers could tell about disk syncing. > How slow? An SSD is 1,000 times slower than memory. A spinning hard drive is one MILLION times slower. Disks are multiple orders of magnitude slower than memory! Latency? Throughput? I don't think SSDs are 1,000 times slower than RAM in either category, assuming fairly typical consumer machines, but I could be wrong.

dmazin 3y ago
Thank you for engaging. Also, thank you for finding that imprecise language. An original draft had stated that I'm talking about I/O duration, but I lost that language. I will re-add it. My numbers are for "main memory access", "SSD I/O" and "rotational disk I/O" from Gregg System Performance (chapter 2). In retrospect, with all respect to Gregg, that wording is itself pretty imprecise. I think what really matters here is I/O duration rather than throughput. That is, the reason we use the cache so much is because a single I/O is really slow against disks...
- jchw 3y ago
  Yeah, that's true. Raw speeds don't really mean a whole lot without more context, and in practice writing to disk is going to be a lot slower even if the peak throughputs are not quite that far off. The performance characteristics are quite different in practice. I assume at one point in history though, these numbers were closer to true even for raw throughput... PCIe and NVMe have probably helped close the gap a bit, I would guess.
marginalia_nu 3y ago
> Latency? Throughput? I, don't think SSDs are 1,000 times slower than RAM in either category, assuming fairly typical consumer machines, but I could be wrong. Everything in this business is like a great big onion of caching and buffering layers. Disks have them too. This is why you often see consumer SSDs have very fast writes only as long as you don't write continuously. Continuous writes causes their internal buffers to fill up and the real write performance to be revealed. There's also factors like write amplification and overprovisioning that enter into this set of equations. But if there's any takeaway from this, it's that I/O benchmarking is hell.

seeknotfind 3y ago

This article had an extremely positive tone, compared to what I expected from the URL. It's refreshing to hear the author's excitement about filesystems, and I learned a couple of things. Thanks.

bpoyner 3y ago

And specifically when you tell the shell to redirect stdout to a file, the shell is opening a file descriptor to handle the request. A file descriptor also sits in the middle for piping stdout for one command to stdin of another. It seems like it's doing magic, but it's not.

passion__desire 3y ago
Are we continuously told in our courses to flush() the file so that it gets written to disk?
- bcrl 3y ago
  If you're talking about fflush() from the C standard libray, fflush() doesn't write to disk. It pushes writes from the FILE's buffer to the kernel and drops any read data that has been cached. The C standard library doesn't really provide you with all the tools needed to commit writes to disk fully.
- 95014_refugee 3y ago
  Hopefully not, because that murders performance. The caching exists for a reason, let it do its thing. If you care about data integrity, you don’t trust the disk anyway, so flushing to the disk isn’t enough.
  - Filligree 3y ago
    There's fsync. Unfortunately, if you want atomicity but not durability you have ~zero options on common filesystems. (On ZFS and bcachefs you can get that by disabling syncs completely; they never reorder writes in a visible manner.)

scarygliders 3y ago

In ye olde unixe days, I learned a habit. That habit was; # sync # sync # sync # halt When powering down ye olde unixe maschine. It was so far back I just vaguely remember I'd read that somewhere; possibly in a ye olde unixe manual, but I can't remember exactly. I think back then it made sense, due to older spinning rust technology and there might have been some possibility of the OS not flushing any waiting cache to disk before powering down. Even though it's probably not needed today, I still do throw a sync command in when powering down, say, my NAS with spinning rust drives, and my PCs/laptop with SSDs. There is one thing where I 100% perform a couple of sync commands, and that's right before unmounting a USB memory stick - I usually witness linux Very Quickly Copying a file from PC to USB stick. "Aha! No, you're just caching that operation you sly dog you!" - the sync command on a CLI tells me when linux has finally actually copied the file onto the stick.

fullstop 3y ago
I still do this. There was another command which was used to park the heads, too.
- sroussey 3y ago
  Ah yes! And park the heads before opening up the server or moving it. Absolute must.
  - JdeBP 3y ago
    FXPARK.EXE! I'm getting flashbacks of Tandon DOS! Aaargh!
dpifke 3y ago
A coworker once asked me for shell tips. I gave them a few that came to mind, and also invited them to peruse my .bash_history, because I figured there might be other tricks hiding in there. They later asked why I sync all the time, almost always in threes. "Huh?" Looking through history, I see that I do. Apparently it's been ingrained into my subconscious, like a sort of stutter, and I type it sometimes out of habit when I'm deep in thought. Old habits really do die hard.

mega_dingus 3y ago

This isn't specific to Linux Raymond Chen has a tangentially amusing story that I just found while trying to google the win95 screen: https://devblogs.microsoft.com/oldnewthing/20160419-00/?p=93315 https://devblogs.microsoft.com/oldnewthing/20160419-00/?p=93...

taeric 3y ago

It helps me to expand the concept of what is happening. Instead of "asking the computer to write a file," it can be seen as "asking the computer to coordinate some information from keyboard, to memory, to disk." You can then expand that to constituent parts. "From memory, to the disk controller, to disk." In this way "disk controller" can easily expand to S3 or some such. And you can begin to enumerate all of the things that have to happen for that to work. (And yes, you could even expand that some, as there is a memory controller and many other things you may want to think about.) And buffers for efficiency aren't new to computers. Consider returning a book to a library. You drop it off in the return box. Eventually, a worker moves it from returns to processing, where they will then put it on a shelving queue. Where it will eventually be carted around and reshelved. Odds are high that you have some form of buffering before it gets to the return box, even.

pjc50 3y ago
> In this way "disk controller" can easily expand to S3 or some such. Disk semantics are subtly different from S3-style blob semantics in important ways. Crucially the immutability of blobs makes life _much_ easier for the blob storage than the ability to write into the middle of a disk file. Given the latency problem, I think it would be interesting and useful for there to be devices which expose a "blob" API to a bunch of Flash on a PCIe bus, and for there to be a suitable OS API for this. Flash itself is only weakly mutable! You can only write in one direction (usually 1->0) to a Flash cell. To go 0->1 you have to do a block erase on a whole bunch of bits, which takes much longer.
- wtallis 3y ago
  > I think it would be interesting and useful for there to be devices which expose a "blob" API to a bunch of Flash on a PCIe bus, and for there to be a suitable OS API for this. We're partly there. NVMe has standardized a key-value interface for SSDs, but as with any compatibility-breaking new storage interface it's pretty much only used by and available to large cloud computing companies that can afford to rewrite their entire storage stack to take advantage of it.
- taeric 3y ago

charcircuit 3y ago

This behavior is up to the file system or virtual file system implementation. Not all writes on Linux use a page cache. It's misleading to tell people that there is always a page cache involved.

dmazin 3y ago
Originally, I was going to explain that I was discussing ext4 specifically but, honestly, can you name a common file system where the page cache isn't involved by default?
- charcircuit 3y ago
  NFS

SteveNuts 3y ago

I don’t remember the last time I actually lost any data when a machine loses power. Am I just lucky or is there more to the story for this comment: > What happens if you unplug your computer before the operating system writes the data to disk? Well, you will lose the data. It’s as simple as that.

scbrg 3y ago
It absolutely happens. My guess is that your machines just don't actually lose power very often. Any normal desk- or laptop won't. When you hit the power button, it doesn't just cut the power; the system is told to shut down so that it can neatly terminate processes and unmount filesystems before halting. Takes a few seconds. At work I'm writing software for industrial machines. It's essentially a Linux system in kiosk mode running on semi custom hardware. We actually do have a power button that simply shuts off power (much to my dismay), and we have data loss all the time. We just make sure that whenever there's actually important data that needs to be stored (very seldom) we make it blatantly obvious to the user that now is a very bad time to push the power button, and sync before those warnings are removed. But we've had boxes that were rendered unbootable because someone decided to push the power button the moment an apt-get upgrade completed and a lot of updated files were still living in memory only.
sltkr 3y ago
There are several reasons why it's actually really unlikely to lose important data when you lose power on modern systems. Firstly, modern filesystems are designed to be robust to random power loss, so it's unlikely that you will corrupt your filesystem in a way that cannot be recovered from. It does mean that some unflushed data would not be written, but it will only affect files that were opened for writing at or shortly before your system lost power. Some filesystems (including ext4) can offer quite strong consistency guarantees if you configure them correctly, though usually stronger consistency means lower performance in some scenarios. Secondly, the Linux regularly flushes dirty pages to disk anyway (15 seconds on my system, see `cat /proc/sys/vm/dirty_expire_centisecs` for yours) so you cannot lose more than a few seconds of data in the worst case. This is particularly important for the case of "oh, I forgot to unmount the USB stick before pulling it out of my laptop". Thirdly, most applications guard against data loss by using some form of write-ahead logging and explicit syncing. For example, if you edit a file in vim and then exit with `wq!` it will call fsync() before exiting, which causes the contents of the file to be flushed to disk immediately. In theory it's possibly that you lose power in the middle of the fsync() call, but it's extremely unlikely. (Of course it would be just as likely that you lose power right before issuing the write-and-quit command.) The sort of data that is not explicitly synced is where it's not really important to keep the latest information. Think about log files that are appended to continuously, or for example downloads in browsers. Do you really care if you have to restart an incomplete download after a sudden power loss? Probably not.

amelius 3y ago

All this talk about why writing files is lazy. But why isn't booting lazy? Why do I have to wait for the network interface to boot even if I don't really intend to use it right now?

wmf 3y ago
systemd does this with socket-activated units.
- amelius 3y ago
  It doesn't work very well, though. Booting into a tty login prompt should be possible in well under 1 second. On a modern Linux system that uses systemd, however, it still takes on the order of a minute or more.
  - JdeBP 3y ago
    There's a lot of seemingly purely local infrastructure that nowadays depends from networking. login calls into a PAM module, which does database lookups, which goes through nsswitch and under the covers tries to consult nscd or systemd-resolved or some such, which either have network timeouts to tick down or aren't started until boot network interfaces are configured .... Mind you, there was the same issue with NIS.
    - amelius 3y ago
      Yes, I guess the problem is that laziness shouldn't stop at service boundaries.