12 ms·
There are many ways to fail to read a file in a C program
- ale42 4y agoThere's no intro about it, but this looks related to IBM mainframes...
- jgtrosh 4y agoYeah, it's tagged under “Z/OS”. Incomprehensible without context.
- nemetroid 4y agoThis seems to be about the z/OS C API.
- nuc1e0n 4y agoMan, IBM mainframe stuff is pretty messed up. Attribute splitting much?
- aaaaaaaaaaab 4y agoEw… I guess this is what 60 years of backwards compatibility looks like?
- Joker_vD 4y agoThis is what trying to heave 50 years-old, backward compatible API on top of a completely differently designed, 60 years-old, backward compatible API looks like.
- lmz 4y agoAnother example of putting C on top of a non-UNIX base would probably be VMS e.g. all the file options here (found using a search) http://odl.sysworks.biz/disk$axpdocdec971/progtool/deccv56/5763profile0111.html#creat_routine http://odl.sysworks.biz/disk$axpdocdec971/progtool/deccv56/5...
- mek6800d2 4y agoBut it works well! Those file options are syntactically optional. I worked on VAX/VMS with Fortran for 5 years and then helped develop generic spacecraft control-center software for NASA in C under Unix. In 1992-1993, we ported the system to VAX/VMS for a European Space Agency project. It went very smoothly and quickly, thanks to DEC's largely complete implementation of the C library (including BSD networking), leaving us with plenty of time to develop the project-specific software. I ported over Sun's rpcgen and GNU's flex and cccp; plus bison or an actual yacc. As you showed, some of the C calls have optional parameters; e.g., strerror() could take an additional argument, a VMS status code, that would provide a more specific error message than just the ERRNO-based messages. However, in almost all cases, the normal Unix call signatures worked as expected. (I did have to come up with a work-around for our one case of a network server using fork().) Thumbs up on VMS and C!
- aaaaaaaaaaab 4y agoThis sentence says it all: >“Normal files” have data in EBCDIC
- AlexanderDhoore 4y agoC has nothing to do with this. You'll have the same problems reading from Java.
- bregma 4y agoThe one single standard way to read a file in C. There are (potentially infinite) many non-standard ways using third-party vendored libraries or vendor-specific extensions to the standard C library. This article discusses a small subset of the latter, specific to a single vendor.
- anonymoushn 4y agoWhat is the standard way?
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- oogali 4y agoI'd wager they are referring to the function named "read".
- anonymoushn 4y agoThat doesn't seem to be part of the C standard library.
- deleted 4y ago[deleted]
- phao 4y agoUsing FILE*, fread, fgets, fscanf, ...
- anonymoushn 4y ago
- nuc1e0n 4y agoSo what is the best practice for reading files from z/OS and OMVS? Should you: Look to see whether the file is binary or text, if it's text what encoding is it in and what is the record length (if there is one). Then open the file as binary and write wrappers for fread and fgets to do the necessary conversions yourself to utf-8 and unix newlines? BTW, fixed length records? Way old school and yuck. Just use an index into the file with lengths already. What about all the wasted space from records that aren't full?
- Joker_vD 4y agoWell, what about all the wasted space from disk clusters that aren't full? IIRC, z/OS actually packs files densely, unlike cluster-based FSs.
- planede 4y agoThis is the first time I saw "keyword arguments" in fopen's mode argument. I'm not a fan. https://www.ibm.com/docs/en/zos/2.4.0?topic=functions-fopen-open-file https://www.ibm.com/docs/en/zos/2.4.0?topic=functions-fopen-...
- mananaysiempre 4y agoGlibc uses those as well[1], ccs=ENCODING opens a file in ENCODING for use with wchar_t functions (“CCS” for “coded character set” being prehistoric standardese for what I colloquially called an encoding here). I’m not a fan, either, FWIW, but those are the most frequent answer I’ve seen as to why fopen() accepts a string and not a set of flags like open(). Might be a post hoc rationalization, though,—I’m willing to believe it was originally just a hack for conciseness, for example. [1] https://www.gnu.org/software/libc/manual/html_node/Opening-Streams.html https://www.gnu.org/software/libc/manual/html_node/Opening-S...
- deleted 4y ago[deleted]
- atoav 4y agoAs someone who uses mainly python, rust and js, as well as C on embedded. I recently came around reading The C Programming Language (2nd edition) and was surprised how many of the language decisions I found extremely horrible. I mean all the examples in the first chapter teach you how to do stuff with strings that will fail the moment you throw Unicode into the mix, and no programmer should use ASCII for any but the lowest level stuff today. I see this as a relic from older, nobler times and the language is interesting to learn about especially since it is the base of a lot of things, but if C was a sport it would be free climbing, or maybe something even more dangerous that requires a lot of skill that I can't think of now. In Rust there are many string types (e.g. OsString, CString, String, PathBuf, ..) because the truth is that you need to know the rules that the string your program reads or creates must adhere to, if there is no type system that will enforce those rules to you. The way a properly written program has to deal with strings in the different parts of systems could be explored in an entire programming career. Similarily Rust tends to make you handle all the errors that could occure with file I/O. This can feel complicated, but it could also serve as a reminder on how many potential errors we don't handle in other programming languages (or at least as a big questionmark how these other languages handle or don't handle these errors). Surely you could also ignore those error cases in Rust and just have your program crash, but then it was your active decision and not something that hit you out of nowhere like a bag of bricks, with the only realistic option of ignoring it and hoping it will not happen again. If anything something like Rust gave me a much better understanding why actual good C programs are akin to art. Freeclimbing in a minefield and all that.
- rmind 4y agoIt is very easy to say, retrospectively, in year 2022, that the decisions were horrible. You are talking about the language and decisions made in 1970s. The knowledge was different. The computers were different, their capability was different. Try running your Python on PDP-7, an 18-bit system! ASCII vs EBCDIC is a computer architecture issue and it's unfair to blame the language that it doesn't have automatic/transparent support for EBCDIC (stuff from 1960s, by the way!). Unicode simply didn't exist at that time. And so on and so forth. On the contrary, I would say C aged really well for a language which was created to support an entire zoo of computers and operating systems. It is worth pointing out that the language has progressed a lot since then and you don't have to deal with many old headaches if you write C on a modern CPU architecture.
- edflsafoiewq 4y agoBy far the biggest problem with reading a file in C is Microsoft's ill-conceived wide-char functions, _wfopen, etc. that produced decades of "ensure the path has no unicode characters" problems. Basically every C/C++ project has a wrapper to fix this. The good news is the bad days may be over soon, thanks to MS moving towards the Unix solution of using UTF8, as well as modern languages like Rust moving this stuff into the stdlib where you can't mess it up.
- zokier 4y ago> The good news is the bad days may be over soon, thanks to MS moving towards the Unix solution of using UTF8, as well as modern languages like Rust moving this stuff into the stdlib where you can't mess it up. It's not all roses on unix or rust side either. In unix filenames are not utf-8, which leads rust having fun things like OsString.
- jcranmer 4y agoOn Unix, filenames are probably UTF-8. The kernel doesn't require them to be UTF-8, but if you're stuck trying to display a filename, you have to figure out what charset the filename is in, and UTF-8 is almost certainly the answer.
- jefftk 4y agoAPIs don't do well with "probably". For example, say you're working with a language (Python, Rust, etc) that distinguishes between utf-8 strings and byte strings. You're making an API for listing a directory: it gives you an array of strings but what kind should they be? (Python's approach is that os.listdir("/") gives you a List[str] (silently omitting undecodable entries), while os.listdir(b"/") gives you a List[bytes]. That is, if you give the path as a utf-8 string it returns utf-8 strings, otherwise it returns bytes.)
- jcranmer 4y agoHonestly? Convince operating systems to have a switch that enforces UTF-8 path names, and then convince distros to flip that switch by default for new installs. That is to say, we need to move the world from a probably to a definitely state. The reality is that file names are "stringy" in nature--people expect to be able to do display them--and that means you need to have some up-front agreement on how to interpret those strings. In practice, on Unix systems, everyone has generally agreed that this is UTF-8, to the point that trying to not be UTF-8 generally causes interesting breaks in the system. It would be great if we could actually get the operating system to help enforce these rules, rather than placing the blame on other software for not correctly handling situations where the correct solution is itself incredibly ambiguous.
- h2odragon 4y agoAlternate title: "I've chose to use a C stdlib which sucks on this OS" I dunno Z/OS, perhaps everything sucks that bad there. I strongly suspect that there's alternate interfaces available that hide this complexity from those afraid to trip over it.
- Joker_vD 4y agoI'd say it's just vastly different, and some rather basic assumptions C makes about the environment (basically that it's sorta kinda UNIX-y if you squint hard enough) simply don't hold. You can still see the UNIX-centric point of view in stdlibs of other languages: I am particularly amused by Golang's "os" package. It's kinda-sorta supposed to be portable and OS- and platform-independent, but it's designed for POSIX-likes first which is why one has to pass 0666 or whatever as permissions when trying to open a file on Windows (even though it is completely ignored).
- salawat 4y ago>it's designed for POSIX-likes first which is why one has to pass 0666 or whatever as permissions when trying to open a file on Windows <Snicker> Well. That's amusing. So everytime one does file access through Go on a windows machine, one invokes the number of the Beast, eh? Apropos af if true. Also hilarious if it just by chance worked out that way.
- Joker_vD 4y agoWell, as I said, you can pass pretty much whatever, none of those bits (except the owner-writeable bit) do anything because Windows uses a fundamentally different model of access control. And no, Golang's os.Create() passes 0666 to os.OpenFile [0], just as fopen(3) passes 0666 to creat(2) [1] on Linux as well. [0] https://cs.opensource.google/go/go/+/refs/tags/go1.19:src/os/file.go;drc=b72a6a7b868deb5b671020c08fbf8d61ad8803d4;l=326 https://cs.opensource.google/go/go/+/refs/tags/go1.19:src/os... [1] https://linux.die.net/man/3/fopen https://linux.die.net/man/3/fopen
- deleted 4y ago[deleted]
- carapace 4y agoI've come around to the view that filesystems are idea whose time has passed, a relic or holdover or atavism from the era of small, slow machines. What you would like, I think, is something like git (or IPFS) where data is stored as content-addresses blobs and metadata (including filenames and directory structures) are also just blobs in the object store.
- salawat 4y agoFilesystems are literally what it says on the tin. It is a filing system. Look in library and secretarial annals for the earliest foundational thinking from which computing's idea of filesystems were born. A systemization of behaviors and abstractions that facilitate the organization, addressing, and access of data. Go to any library, or talk to any long time/old school secretary or warden of archived paperwork, and I assure you, they will be happy to extoll the virtues of simple or reckonable information storage. A hierarchical data store comes baked in with an opportunity of implementing topical locality for the end user, which allows you to utilize pathfinding logic baked into your brain to navigate the corpus of information in question. Content addressable stores, require praying that the layers of cryptography work, or you have enough understanding of the implementation details and tooling around the store to find what you need. In short, find | grep being strictly necessary, rather than a fallback, means you've failed at organizing things so your user can understand where the hell something even is, and why it is there. I assure you, more harm is done by forgetting the fundamental human way of life that computing tries to plaster over, as we inflict impedance mismatch on Users by forcing them to search in a way that makes sense only to the machine, rather than to them. Sometimes a little less ideal computational performance pays dividends in ease of picking up.
- carapace 4y agoI'm old enough to be familiar with non-computerized filing systems myself. But I don't think there's a close match between computer files and directories and old-school hardcopy filing systems. But that's not really what I'm getting at. I'm more thinking like POSIX API vs. Git plumbing API. > A hierarchical data store comes baked in with an opportunity of implementing topical locality for the end user, which allows you to utilize pathfinding logic baked into your brain to navigate the corpus of information in question. Most documents naturally fall into more than one hierarchy, and some "flat" patterns as well (e.g. alphabetical by author). One of the downsides of computer FS is that they encourage a single name-based hierarchy (although using symlinks or hardlinks you can reference files from several directories.) The hierarchy can and should be separate from the object store. Then you can also use e.g. Jef Raskin's Zooming UI to organize topical locality, in addition to more traditional UIs like directory trees.