9 ms·

Linux's fsync() woes are getting some attention

135 points by r4um 13y ago

deleted 13y ago
[deleted]
bodyfour 13y ago
I think part of the problem is that fsync() is that it's an insufficient interface. Most of the time, you want two things: * write ordering ("write B must hit disk after write A") * notification ("let me know when write A is on disk") In particular, you often don't want to actually force I/O to happen immediately, since for performance reasons it's better for the kernel to buffer as much as it wants. In other words, what you want should be nearly free, but instead you have to do a very expensive operation. For an example for notification: suppose I have a temporary file with data that is being journaled into a data store. The operation I want to do is: 1. Apply changes to the store 2. Wait until all of those writes hit disk 3. Delete the temporary file I don't care if step 2 takes 5 minutes, nor do I want the kernel to schedule my writes in any particular way. If you implement step 2 as a fsync() (or fdatasync()) you're having a potentially huge impact on I/O throughput. I've seen these frequent fsync()s cause 50x performance drops!
- Qantourisc 13y ago
  Not a kernel dev but to my knowledge "write B must hit disk after write A" would actually be rather easy to implement: if the storage device supports it: insert a write barrier. For the "let me know when write A is on disk" adding a callback to a write-barrier would do, not sure what the kernel queue looks like though.
  - Filligree 13y ago
    In principle, yes. I believe most modern filesystems use (kernel-level) write barriers to ensure filesystem consistency, and those do depend on having hardware barriers as well. The kernel is not set up to export this interface, however, and there doesn't appear to be any serious work being done to fix that. EDIT: http://pl.atyp.us/2013-11-fixing-fsync.html http://pl.atyp.us/2013-11-fixing-fsync.html has more details.
  - deathanatos 13y ago
    While a "notify me when this write completes" (but don't feel the need to rush) might be useful, I think it'd be more useful to be able to push both write A and write B to the kernel, along with the condition of "A before B". You not only get the write ordering, but the kernel gets full knowledge of what's going on. Perhaps it could use this to write A and B together (but journaled at the FS level) to guarantee the order, whereas a simple write-complete notification implies at least two writes, and the latency of doing such.
    - bodyfour 13y ago
      The two-write operation you propose is simpler, but you're losing a lot of generality. Specifically, think about: * I want to notify a network client that its write is done * When this write is done, I want to delete a temporary file etc
      - deathanatos 13y ago
        Good points — there's definitely a place for a write barrier. It may be that a simple write barrier, while not as efficient for this use case, might be generally more useful. For your second point, if you take my initial "write B depends on the completion of write A", and extend it allow arbitrary kernel commands to depend on others, then you could still do that, (delete B depends on write A), however, things are probably too complex at this point, and a simple write barrier is better.
- mnarayan01 13y ago
  Is that not the purpose of [aio_fsync](http://pubs.opengroup.org/onlinepubs/009696899/functions/aio_fsync.html http://pubs.opengroup.org/onlinepubs/009696899/functions/aio...)? Disclaimer: I've never used it, I just filed it in the back of my mind for if I wanted to do basically what you just described.
  - justincormack 13y ago
    Doesn't exist on Linux as a kernel interface, although the API is there; if it works it is either a thread doing fsync or its synchronous.
- jcampbell1 13y ago
  I don't understand, but I am not very knowledgable about this any of this. My understanding is that fsync is an instruction to command the physical disk to flush its write buffer to the non-volatile (platters/flash) portion of storage, so it is safe to yank the cord from the wall. In your scenario, it seems like you could have a situation where the temp file and the data are sitting on RAM inside the HD. I honestly don't see how multiple processes that share a single physical disk can't get trashed by a processes that instructs the disk to constantly flush the buffer. It seems like asking for a network interface that doesn't slow down the network due other processes.
  - nknighthb 13y ago
    There are multiple levels of caching. The cache in a disk is just one. Filesystem cache is another, which happens in RAM. There may be a layer of caching between filesystem and disk. Applications routinely cache data in their own memory allocations before writing to the filesystem. fsync lives between applications and the physical disk. Its (intended) effect is to flush dirty data that an application has written to the filesystem all the way to the physical disk. For maximum performance and minimum fragmentation, you want to wait as long as possible before data is flushed from the filesystem's cache. In a typical storage system, the filesystem is what decides the approximate ordering of writes and the allocation of disk space. Disks have no idea what's going on, they just respond to commands to write certain data to certain locations; their discretion is minimal, at most they will re-order a small batch of writes for optimum head movement. Delaying a flush as long as possible allows the filesystem to coalesce multiple writes to the same location into one write, allocate disk space in the largest possible chunks in the best locations to avoid fragmentation and excessive seeking, order the writes for rough optimization and for data and metadata consistency. Caching data in RAM is a good thing. fsync is what you use when you've got important data that needs to be preserved immediately. Most data on a typical PC or server is just not that important. You're better off letting the filesystem do its job. And there's a whole continuum of data consistency in between, where you might want certain operations to happen in a certain order, or data to be written in a certain order if it's written, even if you don't necessarily need them to end up on the disk right now.
- dap 13y ago
  > In particular, you often don't want to actually force I/O to happen immediately, since for performance reasons it's better for the kernel to buffer as much as it wants. If you don't issue an fsync(), the kernel never has to write anything. If you had a different function that did what you suggest, returning when the kernel had decided to write the data (without instructing the kernel to do so soonish), you could literally be waiting forever. If the function ever returned, it would only be because something else on the system asked for a sync (and it was easier to write everything), or because you're operating on a filesystem that chooses to write data sooner than required. Both of these are basically working by accident (i.e., may well not work on other POSIX systems). I think what you really want is to instruct the kernel that this data should be written out, and you want to block until that happens. That's exactly what fsync() is supposed to do. The fact that some kernels hammer the I/O subsystem while doing so is a bug in those kernels, not the fsync() interface.
  - ryandrake 13y ago
    Coming from a graphics background, this reminds me of a similar problem in graphics. In the OpenGL API, you have two functions (hard core graphics pedants, please forgive my simplification): There's glFinish() which instructs the graphics system to process all graphics commands previously sent and to block until everything is done. Then there's glFlush() which returns immediately, and basically says, make sure all commands previously sent will finish "in a finite amount of time" (as opposed to "maybe never"). From the discussion here, it seems that fsync() is like glFinish(). Maybe what's needed would be something similar to glFlush()? Something that says, "kernel, now would be a good time to start writing out data, but I'm not going to wait."
    - dap 13y ago
      That's potentially interesting, but the question is: if you're not going to do anything that relies on the data being on stable storage, why bother asking the system to write it out? Most of the use cases I think of involve a server of some kind servicing client requests. Most of the time, the only sane semantics are that if the request completes successfully, then the change will survive a system crash. In that case, you have to fsync() (or equivalent). Conversely, if the client doesn't need that guarantee (e.g., this is a cache that can be reconstructed from elsewhere, as in the case of a CDN), then there's no reason to make sure it's on stable storage at all. You're basically using the filesystem as a large, slow extension of DRAM, and if everything you ever write fits in DRAM, you wouldn't care if the kernel ever wrote it out. Is there a use case you had in mind? I can't think of a middle ground.
- sanxiyn 13y ago
  Featherstitch http://lwn.net/Articles/354861/ http://lwn.net/Articles/354861/ is an interface (and an implementation) for write ordering without blocking. As far as I can tell, it went nowhere.
rwmj 13y ago
Interesting couple of related articles / rants by Jeff Darcy: http://pl.atyp.us/2013-08-local-filesystems-suck.html http://pl.atyp.us/2013-08-local-filesystems-suck.html http://pl.atyp.us/2013-11-fixing-fsync.html http://pl.atyp.us/2013-11-fixing-fsync.html
mrottenkolber 13y ago
I think this is a very important area of improvements for linux. While we call it "multitasking" there are a lot of situations where one might doubt it deserves that title. I have been experimenting with very low cost computing setups that optimize for robustness and that led me to pretty slow disk I/O. While thats not a typical scenario for desktop computing, it can and should be possible with the limited but sane resources I ended up with. In practice however, certain loads freeze the whole system until a single usually non-urgent write finishes. Basically the whole throuput is used for a big write and then X (and others) freeze because they are waiting for the filesystem (probably just a stat and similar). There are differences between applications. Some "behave" worse than others. Some even manage to choke themselves (ever seen GIMP take over an hour to write 4MB to an NFS RAID with 128kb/s throughput?). I guess this is a hard problem, but I would wish for an OS to never stall on load. It is even better to slow down exponentially than to halt other tasks. Ideally the sytem would be smart and deprioritize long-running tasks so that small, presumably urgent, tasks are impacted as little as possible.
haberman 13y ago
It always amazes me that after all these years, Linux still hasn't fixed this. In my experience, any program that overloads I/O will make the system grind to a halt on Linux. Any notion of graceful degradation is gone and your system just thrashes for a while. My theory about this has always been that any I/O related to page faults is starved, which means that every process spends its time slice just trying to swap in its program pages (and evicting other programs from the cache, ensuring that the thrashing will continue). I've never gotten hard data to prove this, and part of me laments that SSDs are "fast enough" that this may never actually get fixed. Can anyone who knows more about this comment? It seems like a good rule inside Linux would be never to evict pages that are mapped executable if you can help it. Has anyone experimented with ionice or iotop? http://www.electricmonk.nl/log/2012/07/30/setting-io-priorities-on-linux/ http://www.electricmonk.nl/log/2012/07/30/setting-io-priorit...
- FigBug 13y ago
  Happens to Windows and Mac OS too. Every time I boot Dropbox thrashes my disk for 10 minutes while the system is almost completely unresponsive.
  - PhantomGremlin 13y ago
    Yes, OS X isn't very advanced. In its heyday, Solaris was outstanding in terms of being responsive while simultaneously doing large amounts of I/O. (Or at least that's my perhaps clouded recollection, I haven't used Solaris in over 5 years).
    - ksk 13y ago
      Well, you'll have to be more specific than 'being responsive' and 'large amounts of I/O' for anyone to make sense out of your statement :)
  - ksk 13y ago
    >Every time I boot Dropbox thrashes my disk for 10 minutes while the system is almost completely unresponsive. I seriously doubt that any usermode program could overwhelm the OS scheduler like that. What are your use case parameters?
est 13y ago
Hmm. maybe that's why they had so many issues with single instance Redis & Mongodb? As soon as fsync() the whole db became unresponsive.
- fiatmoney 13y ago
  MongoDB's storage engine is more or less a mmap'd linked list of documents. It has a lot of issues once you actually start doing reads or writes, whether it's because your working set exceeds RAM or you actually want durability. It's a nice term-paper DB implementation but there's a good reason why most of the big single-instance RDBMS's use their own I/O algos instead of delegating to the kernel. Fundamentally the kernel doesn't know your optimal access pattern or your desired tradeoffs. Redis turned away mmap'd storage some time ago; you can snapshot the DB to disk, but that's done via a fork() and the writes happen in that other process.
cratermoon 13y ago
O_PONIES ride again?
peapicker 13y ago
So, what does Oracle Enterprise Linux do on an Exadata box running Linux?
- caf 13y ago
  ora uses O_DIRECT.
zurn 13y ago
Good to see this summit involving the kernel developers, since their past situation sounds rather bleak interaction-wise: using kernel version from 2009 and haven't tested the improvements in the (2012) 3.2 kernel. BTW, Linux provides the direct I/O O_DIRECT interface that allows apps to bypass the kernel caching business altogether. This is also discussed in Mel Gorman's message that this blog borrows from.
zvrba 13y ago
Ditch Linux and port PgSQL to run on top of raw Xen interfaces. You get to control your own buffering, worker thread scheduling, and talk directly to the (virtual) disk driver. I believe it'd be a win.
ibotty 13y ago
here is a mail from mel gorman with many more details. http://mid.gmane.org/%3C20140310101537.GC10663%40suse.de%3E http://mid.gmane.org/%3C20140310101537.GC10663%40suse.de%3E
dschiptsov 13y ago
Oh yes, that annoying problem (especially fro MongoDB) that data should eventually be committed to a disk.) Informix (and PostgreSQL) allows DBA to choice "checkpoint/vacuum intervals". The rule of thumb, unless you are a Mongo fan, is that checkpoints should be performed often enough to not take too long, which depends only on the actual insert/update data flow. But any real DBA could tell the same - sync quickly, sync often, so server will run smoothly, but not "at web scale" and the pain of recovery will be less severe.)
zvrba 13y ago
Re Mel Gorman's details in http://article.gmane.org/gmane.linux.kernel/1663694 http://article.gmane.org/gmane.linux.kernel/1663694 I don't understand why PostgreSQL people don't want to write their own IO scheduler and buffer management. It's not that hard to implement (even a MT IO+BM is not really complicated), and there are major advantages: - you become truly platform-independent instead of relying on particulars of some kernel [the only thing you need from the OS is some form of O_DIRECT; it exists also on Win32] - you have total control over buffer memory allocation and IO scheduling - whatever scheduling and buffer management policy you're using, you can more easily adapt it to SSDs and other storage types, which are still in their infancy (e.g., memristors) [thus not depending on the kernel developers' goodwill] I mean, really: these pepole have implemented a RDBMS with a bunch of extensions to standard SQL, and IO+buffer management layer is suddenly complicated, or [quote from the link]: "While some database vendors have this option, the Postgres community do not have the resources to implement something of this magnitude." This smells more like politics than a technical issue.

135 points by r4um 13y ago

deleted 13y ago

[deleted]

I think part of the problem is that fsync() is that it's an insufficient interface. Most of the time, you want two things: * write ordering ("write B must hit disk after write A") * notification ("let me know when write A is on disk") In particular, you often don't want to actually force I/O to happen immediately, since for performance reasons it's better for the kernel to buffer as much as it wants. In other words, what you want should be nearly free, but instead you have to do a very expensive operation. For an example for notification: suppose I have a temporary file with data that is being journaled into a data store. The operation I want to do is: 1. Apply changes to the store 2. Wait until all of those writes hit disk 3. Delete the temporary file I don't care if step 2 takes 5 minutes, nor do I want the kernel to schedule my writes in any particular way. If you implement step 2 as a fsync() (or fdatasync()) you're having a potentially huge impact on I/O throughput. I've seen these frequent fsync()s cause 50x performance drops!

Qantourisc 13y ago
Not a kernel dev but to my knowledge "write B must hit disk after write A" would actually be rather easy to implement: if the storage device supports it: insert a write barrier. For the "let me know when write A is on disk" adding a callback to a write-barrier would do, not sure what the kernel queue looks like though.
- Filligree 13y ago
  In principle, yes. I believe most modern filesystems use (kernel-level) write barriers to ensure filesystem consistency, and those do depend on having hardware barriers as well. The kernel is not set up to export this interface, however, and there doesn't appear to be any serious work being done to fix that. EDIT: http://pl.atyp.us/2013-11-fixing-fsync.html http://pl.atyp.us/2013-11-fixing-fsync.html has more details.
- deathanatos 13y ago
  While a "notify me when this write completes" (but don't feel the need to rush) might be useful, I think it'd be more useful to be able to push both write A and write B to the kernel, along with the condition of "A before B". You not only get the write ordering, but the kernel gets full knowledge of what's going on. Perhaps it could use this to write A and B together (but journaled at the FS level) to guarantee the order, whereas a simple write-complete notification implies at least two writes, and the latency of doing such.
  - bodyfour 13y ago
    The two-write operation you propose is simpler, but you're losing a lot of generality. Specifically, think about: * I want to notify a network client that its write is done * When this write is done, I want to delete a temporary file etc
    - deathanatos 13y ago
      Good points — there's definitely a place for a write barrier. It may be that a simple write barrier, while not as efficient for this use case, might be generally more useful. For your second point, if you take my initial "write B depends on the completion of write A", and extend it allow arbitrary kernel commands to depend on others, then you could still do that, (delete B depends on write A), however, things are probably too complex at this point, and a simple write barrier is better.
mnarayan01 13y ago
Is that not the purpose of [aio_fsync](http://pubs.opengroup.org/onlinepubs/009696899/functions/aio_fsync.html http://pubs.opengroup.org/onlinepubs/009696899/functions/aio...)? Disclaimer: I've never used it, I just filed it in the back of my mind for if I wanted to do basically what you just described.
- justincormack 13y ago
  Doesn't exist on Linux as a kernel interface, although the API is there; if it works it is either a thread doing fsync or its synchronous.
jcampbell1 13y ago
I don't understand, but I am not very knowledgable about this any of this. My understanding is that fsync is an instruction to command the physical disk to flush its write buffer to the non-volatile (platters/flash) portion of storage, so it is safe to yank the cord from the wall. In your scenario, it seems like you could have a situation where the temp file and the data are sitting on RAM inside the HD. I honestly don't see how multiple processes that share a single physical disk can't get trashed by a processes that instructs the disk to constantly flush the buffer. It seems like asking for a network interface that doesn't slow down the network due other processes.
- nknighthb 13y ago
  There are multiple levels of caching. The cache in a disk is just one. Filesystem cache is another, which happens in RAM. There may be a layer of caching between filesystem and disk. Applications routinely cache data in their own memory allocations before writing to the filesystem. fsync lives between applications and the physical disk. Its (intended) effect is to flush dirty data that an application has written to the filesystem all the way to the physical disk. For maximum performance and minimum fragmentation, you want to wait as long as possible before data is flushed from the filesystem's cache. In a typical storage system, the filesystem is what decides the approximate ordering of writes and the allocation of disk space. Disks have no idea what's going on, they just respond to commands to write certain data to certain locations; their discretion is minimal, at most they will re-order a small batch of writes for optimum head movement. Delaying a flush as long as possible allows the filesystem to coalesce multiple writes to the same location into one write, allocate disk space in the largest possible chunks in the best locations to avoid fragmentation and excessive seeking, order the writes for rough optimization and for data and metadata consistency. Caching data in RAM is a good thing. fsync is what you use when you've got important data that needs to be preserved immediately. Most data on a typical PC or server is just not that important. You're better off letting the filesystem do its job. And there's a whole continuum of data consistency in between, where you might want certain operations to happen in a certain order, or data to be written in a certain order if it's written, even if you don't necessarily need them to end up on the disk right now.
dap 13y ago
> In particular, you often don't want to actually force I/O to happen immediately, since for performance reasons it's better for the kernel to buffer as much as it wants. If you don't issue an fsync(), the kernel never has to write anything. If you had a different function that did what you suggest, returning when the kernel had decided to write the data (without instructing the kernel to do so soonish), you could literally be waiting forever. If the function ever returned, it would only be because something else on the system asked for a sync (and it was easier to write everything), or because you're operating on a filesystem that chooses to write data sooner than required. Both of these are basically working by accident (i.e., may well not work on other POSIX systems). I think what you really want is to instruct the kernel that this data should be written out, and you want to block until that happens. That's exactly what fsync() is supposed to do. The fact that some kernels hammer the I/O subsystem while doing so is a bug in those kernels, not the fsync() interface.
- ryandrake 13y ago
  Coming from a graphics background, this reminds me of a similar problem in graphics. In the OpenGL API, you have two functions (hard core graphics pedants, please forgive my simplification): There's glFinish() which instructs the graphics system to process all graphics commands previously sent and to block until everything is done. Then there's glFlush() which returns immediately, and basically says, make sure all commands previously sent will finish "in a finite amount of time" (as opposed to "maybe never"). From the discussion here, it seems that fsync() is like glFinish(). Maybe what's needed would be something similar to glFlush()? Something that says, "kernel, now would be a good time to start writing out data, but I'm not going to wait."
  - dap 13y ago
    That's potentially interesting, but the question is: if you're not going to do anything that relies on the data being on stable storage, why bother asking the system to write it out? Most of the use cases I think of involve a server of some kind servicing client requests. Most of the time, the only sane semantics are that if the request completes successfully, then the change will survive a system crash. In that case, you have to fsync() (or equivalent). Conversely, if the client doesn't need that guarantee (e.g., this is a cache that can be reconstructed from elsewhere, as in the case of a CDN), then there's no reason to make sure it's on stable storage at all. You're basically using the filesystem as a large, slow extension of DRAM, and if everything you ever write fits in DRAM, you wouldn't care if the kernel ever wrote it out. Is there a use case you had in mind? I can't think of a middle ground.
sanxiyn 13y ago
Featherstitch http://lwn.net/Articles/354861/ http://lwn.net/Articles/354861/ is an interface (and an implementation) for write ordering without blocking. As far as I can tell, it went nowhere.

rwmj 13y ago

Interesting couple of related articles / rants by Jeff Darcy: http://pl.atyp.us/2013-08-local-filesystems-suck.html http://pl.atyp.us/2013-08-local-filesystems-suck.html http://pl.atyp.us/2013-11-fixing-fsync.html http://pl.atyp.us/2013-11-fixing-fsync.html

mrottenkolber 13y ago

I think this is a very important area of improvements for linux. While we call it "multitasking" there are a lot of situations where one might doubt it deserves that title. I have been experimenting with very low cost computing setups that optimize for robustness and that led me to pretty slow disk I/O. While thats not a typical scenario for desktop computing, it can and should be possible with the limited but sane resources I ended up with. In practice however, certain loads freeze the whole system until a single usually non-urgent write finishes. Basically the whole throuput is used for a big write and then X (and others) freeze because they are waiting for the filesystem (probably just a stat and similar). There are differences between applications. Some "behave" worse than others. Some even manage to choke themselves (ever seen GIMP take over an hour to write 4MB to an NFS RAID with 128kb/s throughput?). I guess this is a hard problem, but I would wish for an OS to never stall on load. It is even better to slow down exponentially than to halt other tasks. Ideally the sytem would be smart and deprioritize long-running tasks so that small, presumably urgent, tasks are impacted as little as possible.

haberman 13y ago

It always amazes me that after all these years, Linux still hasn't fixed this. In my experience, any program that overloads I/O will make the system grind to a halt on Linux. Any notion of graceful degradation is gone and your system just thrashes for a while. My theory about this has always been that any I/O related to page faults is starved, which means that every process spends its time slice just trying to swap in its program pages (and evicting other programs from the cache, ensuring that the thrashing will continue). I've never gotten hard data to prove this, and part of me laments that SSDs are "fast enough" that this may never actually get fixed. Can anyone who knows more about this comment? It seems like a good rule inside Linux would be never to evict pages that are mapped executable if you can help it. Has anyone experimented with ionice or iotop? http://www.electricmonk.nl/log/2012/07/30/setting-io-priorities-on-linux/ http://www.electricmonk.nl/log/2012/07/30/setting-io-priorit...

FigBug 13y ago
Happens to Windows and Mac OS too. Every time I boot Dropbox thrashes my disk for 10 minutes while the system is almost completely unresponsive.
- PhantomGremlin 13y ago
  Yes, OS X isn't very advanced. In its heyday, Solaris was outstanding in terms of being responsive while simultaneously doing large amounts of I/O. (Or at least that's my perhaps clouded recollection, I haven't used Solaris in over 5 years).
  - ksk 13y ago
    Well, you'll have to be more specific than 'being responsive' and 'large amounts of I/O' for anyone to make sense out of your statement :)
- ksk 13y ago
  >Every time I boot Dropbox thrashes my disk for 10 minutes while the system is almost completely unresponsive. I seriously doubt that any usermode program could overwhelm the OS scheduler like that. What are your use case parameters?

est 13y ago

Hmm. maybe that's why they had so many issues with single instance Redis & Mongodb? As soon as fsync() the whole db became unresponsive.

fiatmoney 13y ago
MongoDB's storage engine is more or less a mmap'd linked list of documents. It has a lot of issues once you actually start doing reads or writes, whether it's because your working set exceeds RAM or you actually want durability. It's a nice term-paper DB implementation but there's a good reason why most of the big single-instance RDBMS's use their own I/O algos instead of delegating to the kernel. Fundamentally the kernel doesn't know your optimal access pattern or your desired tradeoffs. Redis turned away mmap'd storage some time ago; you can snapshot the DB to disk, but that's done via a fork() and the writes happen in that other process.

cratermoon 13y ago

O_PONIES ride again?

peapicker 13y ago

So, what does Oracle Enterprise Linux do on an Exadata box running Linux?

caf 13y ago
ora uses O_DIRECT.

zurn 13y ago

Good to see this summit involving the kernel developers, since their past situation sounds rather bleak interaction-wise: using kernel version from 2009 and haven't tested the improvements in the (2012) 3.2 kernel. BTW, Linux provides the direct I/O O_DIRECT interface that allows apps to bypass the kernel caching business altogether. This is also discussed in Mel Gorman's message that this blog borrows from.

zvrba 13y ago

Ditch Linux and port PgSQL to run on top of raw Xen interfaces. You get to control your own buffering, worker thread scheduling, and talk directly to the (virtual) disk driver. I believe it'd be a win.

ibotty 13y ago

here is a mail from mel gorman with many more details. http://mid.gmane.org/%3C20140310101537.GC10663%40suse.de%3E http://mid.gmane.org/%3C20140310101537.GC10663%40suse.de%3E

dschiptsov 13y ago

Oh yes, that annoying problem (especially fro MongoDB) that data should eventually be committed to a disk.) Informix (and PostgreSQL) allows DBA to choice "checkpoint/vacuum intervals". The rule of thumb, unless you are a Mongo fan, is that checkpoints should be performed often enough to not take too long, which depends only on the actual insert/update data flow. But any real DBA could tell the same - sync quickly, sync often, so server will run smoothly, but not "at web scale" and the pain of recovery will be less severe.)

zvrba 13y ago

Re Mel Gorman's details in http://article.gmane.org/gmane.linux.kernel/1663694 http://article.gmane.org/gmane.linux.kernel/1663694 I don't understand why PostgreSQL people don't want to write their own IO scheduler and buffer management. It's not that hard to implement (even a MT IO+BM is not really complicated), and there are major advantages: - you become truly platform-independent instead of relying on particulars of some kernel [the only thing you need from the OS is some form of O_DIRECT; it exists also on Win32] - you have total control over buffer memory allocation and IO scheduling - whatever scheduling and buffer management policy you're using, you can more easily adapt it to SSDs and other storage types, which are still in their infancy (e.g., memristors) [thus not depending on the kernel developers' goodwill] I mean, really: these pepole have implemented a RDBMS with a bunch of extensions to standard SQL, and IO+buffer management layer is suddenly complicated, or [quote from the link]: "While some database vendors have this option, the Postgres community do not have the resources to implement something of this magnitude." This smells more like politics than a technical issue.