9 ms·

Why does C have the best file API

• FrankWilhoit 7mo ago
  A file API is not the same thing as a filesystem API. The holy grail is still a universal but high(-enough)-level filesystem API.
• seba_dos1 7mo ago
  mmap is not a C feature, but POSIX. There are C platforms that don't provide mmap, and on those that do you can use mmap from other languages (there's mmap module in the Python's standard library, for example).
  • deleted 7mo ago
    [deleted]
  • ajross 7mo ago
    I think this is sort of missing the point, though. Yes, mmap() is in POSIX[1] in the sense of "where is it specified". But mmap() was implemented in C because C is the natural language for exposing Unix system calls and mmap() is a syscall provided by the OS. And this is true up and down the stack. Best language for integrating with low level kernel networking (sockopts, routing, etc...)? C. Best language for async I/O primitives? C. Best language for SIMD integration? C. And it goes on and on. Obviously you can do this stuff (including mmap()) in all sorts of runtimes. But it always appears first in C and gets ported elsewhere. Because no matter how much you think your language is better, if you have to go into the kernel to plumb out hooks for your new feature, you're going to integrated and test it using a C rig before you get the other ports. [1] Given that the pedantry bottle was opened already, it's worth pointing out that you'd have gotten more points by noting that it appeared in 4.2BSD.
    • nickelpro 7mo ago
      If we're going to be pedantic, mmap is a syscall. It happens that the C version is standardized by POSIX. The underlying syscall doesn't use the C ABI, you need to wrap it to use it from C in the same way you need to wrap it to use it from any language, which is exactly what glibc and friends do. Moral of the story is mmap belongs to the platform, not the language.
      • a-dub 7mo ago
        it also appears in operating systems that aren't written in c. i see it as an operating system feature, categorically.
      • deleted 7mo ago
        [deleted]
      • ajross 7mo ago
        No, that's too far down the pedantry rabbit hole. "mmap()" is quite literally a C function in the 4.2BSD libc. It happens to wrap a system call of the same name, but to claim that they are different when they arrived in the same software and were written by the same author at the same time is straining the argument past the breaking point. You now have a "C Erasure Polemic" and not a clarifying comment. If you take a kernel written in C and implement a VM system for it in C and expose a new API for it to be used by userspace processes written in C, it doesn't magically become "not C" just because there's a hardware trap in the middle somewhere. mmap() is a C API. I mean, duh.
        • nagaiaida 7mo ago
          and if i directly do an mmap syscall on linux from a freestanding forth that doesn't go through libc for anything? sure, c unfortunately defines how i have say, pass a string, but that's effectively an arbitrary calling convention at that point; there's no c runtime on the calling side so it's not particularly useful to contend that what i'm using is a c api. or perhaps mmap is incontrovertibly a c function on platforms where libc wrappers are the sole stable interface to the kernel but something else entirely on linux?
          • ajross 7mo ago
            > and if i directly do an mmap syscall on linux from a freestanding forth ... mmap() remains a system call to a C kernel designed for use from the C library in C programs, and you're running what amounts to an emulator. The fact that you can imagine[1] an environment where that might not be the case doesn't mean that it isn't the case in the real world. Your argument appears to be one of Personal Liberty: de facto truths don't matter because you can just make your own. This is sort of a software variant of a Sovereign Citizen, I think. [1] Can you even link a "freestanding forth" with an mmap() binding on any Unix that doesn't live above the libc implementation? I mean, absent everything else it would have to open code all the flag constants, whose values change between systems. This appears to be a completely fictitious runtime you've invented, which if anything sits as evidence in my favor and not yours.
    • projektfu 7mo ago
      Why does Ada have the best file API? https://github.com/AdaCore/florist/blob/master/libsrc/posix-memory_mapping.ads https://github.com/AdaCore/florist/blob/master/libsrc/posix-...
    • deleted 7mo ago
      [deleted]
    • qalmakka 7mo ago
      > C is the natural language for exposing Unix system calls No, C is the language _designed_ to write UNIX. Unix is older than C, C was designed to write it and that's why all UNIX APIs follow C conventions. It's obvious that when you design something for a system it will have its best APIs in the language the system is written in. C has also multiple weird and quirky APIs that suck, especially in the ISO C libc.
      • benj111 7mo ago
        If I write an OS in Basic, surely the 'natural' language for exposing the system calls is Basic? Yes Unix predates C. But at this point in time 50+ years down the road, where the majority on nix users don't use anything that ever contained that code, and the minority use a nix that has been thoroughly ship of Theseused, Unix is to all intents and purposes a C operating system.
        • qalmakka 7mo ago
          > If I write an OS in Basic, surely the 'natural' language for exposing the system calls is Basic? For that specific OS, that would probably be the case? I think every API is bound to reflect the specific constraints of the language it has been written in. What I was trying to clarify was that UNIX and C are intertwined in an especially deep way, more than basically other OS that doesn't have a UNIX API, because both were born and written alongside each other, so some Unix APIs rely on C-specific behaviour and quirks and some C features were born and designed around the same historical context UNIX was born
      • ajross 7mo ago
        >> C is the natural language for exposing Unix system calls > No, C is the language _designed_ to write UNIX. [...] This is one of those hilarious situations where internet discussion goes off the rails. Everything you wrote, to the last word, would carry the same meaning and the same benefit to the discussion had you written "Yes" instead of "No" as the first word. Literally you're agreeing with me, but phrasing it as a disagreement only because you feel I left something out.
      • 7mo ago
    • t-3 7mo ago
      >> Best language for SIMD integration? C Uh, no. C intrinsics are so much worse than just writing assembly that it's not even comparable.
      • ajross 7mo ago
        Agree to disagree there. For casual "I need to vectorize this code" tasks, modern compilers are almost magic. I mean, have you looked at the generated code for array-based numerics processing? It's like, you start the process of "vectorizing" the algorithm and realize the compiler already did 80% of it for you.
  • flohofwoe 7mo ago
    And it's not just mmap(), all the functions in the code snippet except printf() are not actually C stdlib functions.
• mmastrac 7mo ago
  "best file API" and the man page for the O_ flags disagree.
• srean 7mo ago
  > However, in other most languages, you have to read() in tiny chunks, parse, process, serialize and finally write() back to the disk. This works, but is verbose and needlessly limited C has those too and am glad that they do. This is what allows one to do other things while the buffer gets filled, without the need for multithreading. Yes easier standardized portable async interfaces would have been nice, not sure how well supported they are.
  • general_reveal 7mo ago
    Wouldn’t we need to implement all of that extra stuff if we really wanted to work with text from files? I have a use case where I do need extra fast text input/output from files. If anyone has thoughts on this, I’d love it.
    • srean 7mo ago
      The standard way is to use libraries like libevent, libuv that wraps system calls such as epoll, kqueue etc. The other palatable way is to register consumer coroutines on a system provided event-loop. In C one does so with macro magic, or using stack switching with the help of tiny bit of insight inline assembly. Take a look at Simon Tatham's page on coroutines in C. To get really fast you may need to bypass the kernel. Or have more control on the event loop / scheduler. Database implementations would be the place to look.
• Const-me 7mo ago
  I think C# standard library is better. You can do same unsafe code as in C, SafeBuffer.AcquirePointer method then directly access the memory. Or you can do safer and slightly slower by calling Read or Write methods of MemoryMappedViewAccessor. All these methods are in the standard library, i.e. they work on all platforms. The C code is specific to POSIX; Windows supports memory mapped files too but the APIs are quite different.
  • SvenL 7mo ago
    I think you don’t need to be unsafe, they have normal API for it. https://learn.microsoft.com/en-us/dotnet/standard/io/memory-mapped-files https://learn.microsoft.com/en-us/dotnet/standard/io/memory-...
    • Const-me 7mo ago
      Indeed, but these normal APIs have runtime costs for bounds checking. For some use cases, unsafe can be better. For instance, last time I used a memory-mapped file was for a large immutable Bloom filter. I knew the file should be exactly 4GB, validated that in the constructor, then when testing 12 bits from random location of the mapped file on each query, I opted for unsafe codes.
      • pjmlp 7mo ago
        It is a matter of the deployment scenario, in the days people ship Electron, and deploy production code in CPython, those bounds checking don't hurt at all. When they do, thankfully there is unsafe if needed.
• koakuma-chan 7mo ago
  How do you handle read/write errors with mmap?
  • Falcondor 7mo ago
    mmap on file io errors would manifest in Signals (For example SIGBUS or SIGSEGV). So if you wanted to handle file read/write errors you would need to implement signal handlers. https://stackoverflow.com/questions/6791415/how-do-memory-mapped-files-handle-i-o-errors https://stackoverflow.com/questions/6791415/how-do-memory-ma...
    • koakuma-chan 7mo ago
      ... which is not great for an API.
  • icedchai 7mo ago
    In my experience, having worked with a large system that used almost exclusively mmap for I/O, you don’t. The process segfaults and is restarted. In practice it almost never happened.
• castral 7mo ago
  I think OP and I have very divergent opinions on what makes a file API "best". This may have been the best 30 years ago. The world has moved on.
• Dwedit 7mo ago
  Using mmap means that you need to be able to handle memory access exceptions when a disk read or write fails. Examples of disk access that fails includes reading from a file on a Wifi network drive, a USB device with a cable that suddenly loses its connection when the cable is jiggled, or even a removable USB drive where all disk reads fail after it sees one bad sector. If you're not prepared to handle a memory access exception when you access the mapped file, don't use mmap.
  • justmedep 7mo ago
    You can even mmap a socket on some systems (iOS and macOS via GCD). But doing that is super fragile. Socket errors are swallowed. My interpretation always was the mmap should only be used for immutable and local files. You may still run into issues with those type of files but it’s very unlikely.
    • kelnos 7mo ago
      mmap is also good for passing shared memory around. (You still need to be careful, of course.)
      • usefulcat 7mo ago
        It’s also great for when you have a lot of data on local storage, and a lot of different processes that need to access the same subset of that data concurrently. Without mmap, every process ends up caching its own private copy of that data in memory (think fopen, fread, etc). With mmap, every process accesses the same cached copy of that data directly from the FS cache. Granted this is a rather specific use case, but for this case it makes a huge difference.
  • phoronixrly 7mo ago
    Ah, reminds me of 'Are You Sure You Want to Use MMAP in Your Database Management System? (2022)' https://db.cs.cmu.edu/mmap-cidr2022/ https://db.cs.cmu.edu/mmap-cidr2022/
    • 7mo ago
• zahlman 7mo ago
  Aside from what https://news.ycombinator.com/item?id=47210893 https://news.ycombinator.com/item?id=47210893 said, mmap() is a low-level design that makes it easier to work with files that don't fit in memory and fundamentally represent a single homogeneous array of some structure. But it turns out that files commonly do fit in memory (nowadays you commonly have on the order of ~100x as much disk as memory, but millions of files); and you very often want to read them in order, because that's the easiest way to make sense of them (and tape is not at all the only storage medium historically that had a much easier time with linear access than random access); and you need to parse them because they don't represent any such array. When I was first taught C formally, they definitely walked us through all the standard FILE* manipulators and didn't mention mmap() at all. And when I first heard about mmap() I couldn't imagine personally having a reason to use it.
  • nickelpro 7mo ago
    mmap is also relatively slow (compared to modern solutions, io_uring and friends), and immensely painful for error handling. It's simple, I'll give it that.
    • nixon_why69 7mo ago
      Page faults are slower than being deliberate about your I/O but mapped memory is no faster or slower than "normal" memory, its the same mechanism.
      • namibj 7mo ago
        Nah, usually can't have huge pages. Almost certainly can't have giant pages. Can't even fit all L3$ capacity into the L2 TLB if done via 4k pages...
        • nixon_why69 7mo ago
          I hadn't thought of that but apparently Linux at least has had support for a while, according to manpage? https://man7.org/linux/man-pages/man2/mmap.2.html https://man7.org/linux/man-pages/man2/mmap.2.html
• nickelpro 7mo ago
  > Why does C have the best file API > Look inside > Platform APIs Ok. I agree platform APIs are better than most generic language APIs at least. I disagree on mmap being the "best".
• charcircuit 7mo ago
  C's API does not include mmap, nor does it contain any API to deal with file paths, nor does it contain any support for opening up a file picker. This paired with C's bad string support results in one of it being one of the worst file APIs. Also using mmap is not as simple as the article lays out. For example what happens when another process modifies the file and now your processes' mapped memory consists of parts of 2 different versions of the file at the same time. You also need to build a way to know how to grow the mapping if you run out room. You also want to be able to handle failures to read or write. This means you pretty much will need to reimplement a fread and fwrite going back to the approach the author didn't like: "This works, but is verbose and needlessly limited to sequential access." So it turns out "It ends up being just a nicer way to call read() and write()" is only true if you ignore the edge cases.
• userbinator 7mo ago
  It still works if the file doesn't fit in RAM No it doesn't. If you have a file that's 2^36 bytes and your address space is only 2^32, it won't work. On a related digression, I've seen so many cases of programs that could've handled infinitely long input in constant space instead implemented as some form of "read the whole input into memory", which unnecessarily puts a limit on the input length.
  • actionfromafar 7mo ago
    You can mmap with offset, for that case. Just FYI in anyone thought it was a hard limit.
  • dekhn 7mo ago
    Address space size and RAM are two different things.
    • dataflow 7mo ago
      What they said is correct regardless of that though?
      • conradludgate 7mo ago
        The point the article makes is that a 32GB file can be mmapped even if you only have 8GB of memory available - it wasn't talking about the address space. So the response is irrelevant even if technically correct
      • lelanthran 7mo ago
        > What they said is correct regardless of that though? I don't think so. Their post is basically: >> It still works if the file doesn't fit in RAM > No it doesn't. Which is incorrect: it actually does work for files that don't fit in RAM. It doesn't work only for files that don't fit in the address space, which is not what the author claimed.
      • dekhn 7mo ago
        I'm pretty sure the parent post to mine was updated from "RAM" to "address space", although I'm not 100% sure.
• alanfranz 7mo ago
  Well... I'm not sure what the author really wants to say. mmap is available in many languages (e.g. Python) on Linux (and many other *nix I suppose). C provides you with raw memory access, so using mmap is sort-of-convenient for this use case. But if you use Python then, yes, you'll need a bytearray, because Python doesn't give you raw access to such memory - and I'm not sure you'd want to mmap a PyObject anyway? Then, writing and reading this kind of raw memory can be kind of dangerous and non-portable - I'm not really sure that the pickle analogy even makes sense. I very much suppose (I've never tried) that if you mmap-read malicious data in C, a vulnerability would be _quite_ easy to exploit.
  • bvrmn 7mo ago
    Actually in Python you could recast (zerocopy) bytearray as other primitive C type or even any other structure using ctypes module.
  • okanat 7mo ago
    Creating memory mapped files is a very common OS feature since 90s. Many high level languages have it as OS agnostic POSIX or not.
    • racingmars 7mo ago
      > very common OS feature since 90s And if you want to go farther back, even if it wasn't called "mmap" or a specific function you had to invoke -- there were operating systems that used a "single-level store" (notably MULTICS and IBM's AS/400..err OS/400... err i5 OS... err today IBM i [seriously, IBM, pick a name and stick with it]) where the interface to disk storage on the platform is that the entire disk storage/filesystem is always mapped into the same address space as the rest of your process's memory. Memory-mapped files were basically the only interface there was, and the operating system "magically" persisted certain areas of your memory to permanent storage.
    • alanfranz 7mo ago
      And? Did I claim something different? I just didn’t use that feature on other OSes.
• chuckadams 7mo ago
  It has the best API for the author, that's for sure. One size does not fit all: believe it or not, different files have different uses. One does not mmap a pipe or /dev/urandom.
• ibejoeb 7mo ago
  The article only touches on `open` and `close` and doesn't deal with any of the realities of file access. Not a particularly compelling write-up.
• andersmurphy 7mo ago
  mmap is nice. But, I find sqlite is a better filesystem API [1]. If you are going to use mmap why not take it further and use LMDB? Both have bindings for most languages. [1] - https://sqlite.org/fasterthanfs.html https://sqlite.org/fasterthanfs.html
• okanat 7mo ago
  I guess the author didn't use that many other programming languages or OSes. You can do the same even in garbage collected languages like Java and C# and on Windows too. https://docs.oracle.com/javase/8/docs/api/java/nio/MappedByteBuffer.html https://docs.oracle.com/javase/8/docs/api/java/nio/MappedByt... https://learn.microsoft.com/en-us/dotnet/api/system.io.memorymappedfiles.memorymappedfile?view=net-10.0 https://learn.microsoft.com/en-us/dotnet/api/system.io.memor... https://learn.microsoft.com/en-us/windows/win32/memory/creating-a-file-mapping-object https://learn.microsoft.com/en-us/windows/win32/memory/creat... Memory mapping is very common.
  • dekhn 7mo ago
    mmap is a built-in module on python! Also true for perl.
  • paulddraper 7mo ago
    And since Java 4 no less.
  • deleted 7mo ago
    [deleted]
  • neonsunset 7mo ago
    [dead]
  • nitwit005 7mo ago
    I'd be careful though, as they all have quirks due to how tricky it is handling mmap faults. The Java API mentions both unique garbage collection behavior and throwing unspecified exceptions at unspecified times.
• jccx70 7mo ago
  This is like: I discovered the wheel and want to let you know!
• teunispeters 7mo ago
  technically yes, because there's a failure path for every single failure that an OS knows about. And most others aren't so resilient. However, mmap bypasses a lot of that....
• nice_byte 7mo ago
  mmap is not a language feature. it is also full of its own pitfalls that you need to be aware of. recommended reading: https://db.cs.cmu.edu/mmap-cidr2022/ https://db.cs.cmu.edu/mmap-cidr2022/
• TZubiri 7mo ago
  Here's a negative signal I'm seeing often: When a developer that usually consumes the language starts critiquing the language. I could go on as to why it's a bad signal, psychologically, but let's just say that empirically it usually doesn't come from a good place, it's more like a developer raising the stakes of their application failing and blaming the language. Sure one out of a thousand devs might be the next Linus Torvalds and develop the next Rust. But the odds aren't great.
• kelnos 7mo ago
  mmap() is useful for some narrow use-cases, I think, but error-handling is a huge pain. I don't want to have to deal with SIGBUS in my application. I agree that the model of mmap() is amazing, though: being able to treat a file as just a range of bytes in memory, while letting the OS handle the fussy details, is incredibly useful. (It's just that the OS doesn't handle all of the fussy details, and that's a pain.)
• amluto 7mo ago
  I can’t entirely tell what the article’s point is. It seems to be trying to say that many languages can mmap bytes, but: > (as far as I'm aware) C is the only language that lets you specify a binary format and just use it. I assume they mean: struct foo { fields; }; foo *data = mmap(…); And yes, C is one of relatively few languages that let you do this without complaint, because it’s a terrible idea. And C doesn’t even let you specify a binary format — it lets you write a struct that will correspond to a binary format in accordance with the C ABI on your particular system. If you want to access a file containing a bunch of records using mmap, and you want a well defined format and good performance, then use something actually intended for the purpose. Cap’n Proto and FlatBuffers are fast but often produce rather large output; protobuf and its ilk are more space efficient and very widely supported; Parquet and Feather can have excellent performance and space efficiency if you use them for their intended purposes. And everything needs to deal with the fact that, if you carelessly access mmapped data that is modified while you read it in any C-like language, you get UB.
  • dvt 7mo ago
    Had the same thought. Also confused at the backhanded compliment that pickle got: > Just look at Python's pickle: it's a completely insecure serialization format. Loading a file can cause code execution even if you just wanted some numbers... but still very widely used because it fits the mix-code-and-data model of python. Like, are they saying it's bad? Are they saying it's good? I don't even get it. While I was reading the post, I was thinking about pickle the whole time (and how terrible that idea is, too).
    • zadikian 7mo ago
      The article is saying it's good, or at least good enough. I don't necessarily agree with the rest of the article.
    • benj111 7mo ago
      A thing can be good and bad. Everything is a tradeoff. The reason why C is 'good' in this instance is the lack of safety, and everything else that makes C, C (see?) but that is also what makes C bad.
• Perenti 7mo ago
  It may have a tidy mmap api, but Smalltalk has a much better file api through its Streams hierarchy IMHO. You can create a stream on a diskfile, you can create a stream on a byteArray, you can create a stream on standard Unix streams, you can create a stream on anything where "next" makes sense.
• deleted 7mo ago
  [deleted]
• deleted 7mo ago
  [deleted]
• teo_zero 7mo ago
  What a bizarre conclusion to draw! It's like saying that cars are the best means of transportation because you can travel to the Grand Canyon in them and the Grand Canyon is the best landscape in the world, and yes you could use other means to get there, but cars are what everybody's using. If the real goal of TFA was to praise C's ability to reinterpret a chunk of memory (possibly mapped to a file) as another datatype, it would have been more effective to do so using C functions and not OS-specific system calls. For example: FILE *f = fopen(...); uint32_t *numbers; fread(numbers, ..., f); access numbers[...] frwite(numbers, ..., f); fclose(f);
  • m-schuetz 7mo ago
    This is way more cumbersome than mmap if you need to out-of-core process the file in non-sequential patterns. Way way more cumbersome, since you need to deal with intermediate staging buffers, and reuse them if you actually want to be fast. mmap, on the other hand, is absolutely trivial to use, like any regular buffer pointer. And at least on windows, the mmap counterpart can be faster when processing the file with multiple threads, compared to fread. But I agree that it's a bizarre article since mmap is not a C standard, and relies on platform-dependend Operating System APIs.
• pmontra 7mo ago
  I think that I open files in very few cases in my job. I read and write PDF, xlsx, csv, yaml and I write docx. Those have their own formats and we use them to communicate with other apps or with users. Everything else goes in a PostgreSQL database or in sqlite3 because of many reasons and among them because of interoperability with other apps and ease of human inspection. A custom file format could be OK for data that only that app must use or for performance reasons, if you know how to make data retrieval performant.
• raincole 7mo ago
  At first glance, it's a quite weird article. But at the bottom: > This simply isn't true on memory constrained systems — and with 100 GB files — every system is memory constrained. I suppose the author might have a point in the context of making apps that constantly need to process 100GB files? I personally never have to deal with 100GB files so I am no one to judge if the rest of the article makes sense.
  • jim33442 7mo ago
    Yeah, the article makes more sense when you assume it's for that use case, but still not really sure when this case comes up. I've dealt with plenty of 100GB files, but they were coming from outside as CSVs or sqlite DBs or something. This one is, your C program is going to generate a 100GB file to use later, and you also don't need a DB.
• jim33442 7mo ago
  You can't do dynamic memory with that, right? Not without a custom malloc implementation. So it's not all that comparable to pickle.
  • bregma 7mo ago
    Most modern C runtime libraries on POSIX OSes implement malloc() and friends using the mmap() system call.
    • jim33442 7mo ago
      I mean in your C code that's using mmap, I know you can do this with a custom malloc impl, but idk if there's a built-in way: int len = 1000; int file = open("numbers.void", O_RDWR | O_CREAT, 0600); ftruncate(file, len); void\* buf = mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_SHARED, file, 0); // Treat mmapped buffer as a heap initialize_heap(buf); // Manage dynamically-sized array on disk int* my_array = malloc(sizeof(int) * 8, buf); my_array = realloc(my_array, sizeof(int) * 16, buf); free(my_array, buf); Protobuf, Python pickle, etc can all handle dynamic memory that gets flattened when you want to serialize.
• swaminarayan 7mo ago
  If mmap-style file access is this powerful, why do most higher-level languages avoid exposing typed, struct-level mappings directly instead of just byte buffers?
• qalmakka 7mo ago
  > C has the best API POSIX has the best API. C has `fopen` which, while not terrible, isn't what I'd call "great"
• Surac 7mo ago
  After reading the comments here it boils down to: But my language is better then yours. mmap is not a feature of C. Some more modern languages try to prevent people form shooting in there feet and only allow byte wise access to such mmaped regions. The have a point doing this, but on the other hand also the C-Users have a valid point. Safety and Speed are 2 Factors you have to consider using the tools you use. From a Hardware point of view C might be more direct but it also enables you to make "stupid" errors fast. More Modern languages prevent you from the "stupid" errors but make you copy or transform the data more. Scotty from the Enterprise sayed once: Allways use the fitting tool
• karel-3d 7mo ago
  In go, you can do mmap with some help of external library :) you can mmap a file - https://github.com/edsrzf/mmap-go https://github.com/edsrzf/mmap-go - and then unsafe-retype that to slice of objects, and then read/write to it. It's very handy sometimes! It's unsafe though. You also need to be careful to not have any pointers in the struct (so also no slices, maps); or if you have pointers, the must be nil. Once you start unsafe-retyping random bytes to pointers, thing explode very quickly. So maybe this article has a point.
• groundzeros2015 7mo ago
  nobody mentioning the "file system as a noSQL database comment". I found most of the friction when using bash/unix style tools when I tried to put everything in structured files that needed parsing. Once you see folder/files as the structure these tools work great. It's also interesting to me that many noSQL starts from the assumption that relations are too complex, and that trees are preferred.
  • zadikian 7mo ago
    NoSQL was more of a convenience thing during that trend. You might have some bag of nested attributes that maps perfectly fine onto relations, but it's cumbersome for someone who just wants to load it all into an object, edit, then resave. People used to use ORMs to get around that, then NoSQL became popular, and now you can just use jsonb in SQL while still maintaining relations for other things. This doesn't say much about the horizontal scaling that NoSQL systems were really designed for, but most people getting on that train didn't need that kind of scale.
    • groundzeros2015 7mo ago
      Yes, and I’m suggesting to try the file system if you think a tree is convenient.
• panzi 7mo ago
  This is not the C file API, this is the POSIX file API. Doesn't standard C only define fopen() & co? In any case under Win32 this is all different.
• zephen 7mo ago
  I'm surprised nobody has mentioned zig. Surely its "bring your own allocator" paradigm also allows this.