7 ms·
The result you get with this trick is signed, while the result you get with sizeof is unsigned. Edit: Just to clarify, what you get is ptrdiff_t instead of siz
by ycmbntrthrwaway 10y ago
The result you get with this trick is signed, while the result you get with sizeof is unsigned.
Edit: Just to clarify, what you get is ptrdiff_t instead of size_t. So if array size is greater than PTRDIFF_MAX, you get undefined behavior [1].
[1] http://en.cppreference.com/w/c/types/ptrdiff_t http://en.cppreference.com/w/c/types/ptrdiff_t
- flamedoge 10y agoHow likely do you run into array bigger than 2gb?
- minipci1321 10y agoProbably not very likely, but keep in mind that this method could also be used without actually allocating the array -- akin to the 'offsetof()' macro. (Which is undefined behavior.)
- dragandj 10y agoToday, with ML, big data and similar applications, that might be often.
- quotemstr 10y agoThe "how likely is it, really?" response to questions of technical correctness has always bothered me. It takes a mindset completely alien to mine to say "Here's a race condition. Sure, it's undefined behavior, but the race is narrow, so it's rare" or to say "Sure, memory allocation can theoretically fail, but in practice almost never does" or to say "fsync is too slow and most computers have batteries these days". Software is unreliable enough as it is due to problems beneath our notice. It seems reckless to avoid fixing problems that we do notice. Sure, you could argue that rare problems are rare and that users probably won't notice them --- this attitude is penny-wise and pound-foolish, because you can't meaningfully reason about a system that's only probably correct.
- mjburgess 10y agoThe problem you're latching on to I think is how the context for caculating a probability can vary. If it were really as likely as, say, the sun exploding that X happened then it would be of no use to expend time on X. BUT very often people speaking about the probability of events given suspicious constraints. While a memory allocation might not fail in most situations it will fail often in some situations. And a one-in-a-million chance is almost guaranteed when there are millions of uses.
- spdionis 10y agoThat's why it's called one in a million...
- tholmes 10y agoOne in a million happens quite often if you're processing something like ~100k requests a second.
- jschwartzi 10y agoBut it's extremely unlikely when you're processing 10 requests a week, such as might be the case for the web server in a consumer-grade router.
- paulddraper 10y ago...by hundreds of thousands of customers.
- quotemstr 10y agoIt's amazing how skilled blackhats are at converting "rare bug that doesn't affect the UI" into "massive DDoS cannon".
- jschwartzi 10y ago
- vram22 10y agoNot likely, but possible. This reminds me of the bug that was found in the binary search algorithm a few years ago, IIRC, in Java. The interesting thing is that binary search is probably one of the earliest-invented algorithms. Yet, in the book Writing Efficient Programs by Jon Bentley (which I mentioned in a recent HN comment), he says that in a class he taught to several industrial programmers with many years of experience, some had bugs in their implementations of binary search that he set them as an exercise. Not sure but I think I remember reading in the article about the Java binary search issue, that even his algorithm had the bug that was found in the Java version. Why it was not found earlier is (maybe) because it only occurred with an extremely large array, IIRC. Don't have a link right now, but it can probably be found by searching for the right phrase.
- vram22 10y agoJust did a google search, and it even partially auto-completed this search for me: bug in java binary search and showed a related search in the drop-down, 'programming pearls ...', a book by Jon Bentley, which seems to confirm what I said above (though I saw it in his other book, "Efficient Programs", IIRC - he might have mentioned the same issue in the Programming Pearls book too). Edit: and the Wikipedia article confirms it too: https://en.wikipedia.org/wiki/Binary_search_algorithm#Implementation_issues https://en.wikipedia.org/wiki/Binary_search_algorithm#Implem...
- pigz 10y agohttps://research.googleblog.com/2006/06/extra-extra-read-all-about-it-nearly.html https://research.googleblog.com/2006/06/extra-extra-read-all...
- vram22 10y agoYes, thanks, that closely describes the assignment I read that he gave.
- jbn 10y agoI think this is it: http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6412541 http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6412541
- cperciva 10y agoA few months ago I was doing FFTs on arrays larger than 4GB. Amusingly, this uncovered a bug in the LLVM optimizer: It was looking at stride lengths to figure out if accesses were independent, and truncated a 4GB stride down to 0.
- ksml 10y agoI would be extremely interested to hear how you found this bug. Sounds like a difficult bug to track down, and I always learn from good debugging stories.
- cperciva 10y agoIt was pretty easy to track down: clang38 was exiting with Assertion failed: (Distance > 0 && "The distance must be non-zero"), function areStridedAccessesIndependent, file /wrkdirs/usr/ports/devel/ llvm38/work/llvm-3.8.0.src/lib/Analysis/LoopAccessAnalysis.cpp, line 1004. Looking at the file it was easy to see what was being asserted, and to see that the type was a 32-bit integer; since I knew I was dealing with huge FFTs, the problem was obvious. Let this be a lesson: Asserting that impossible things don't happen makes debugging much easier when they do happen!
- flamedoge 10y agoWell good thing it was ReleaseWithAsserts build!
- icedchai 10y agoOn a 64-bit platform (anything modern), ptrdiff_t is going to be 64-bit so this will not be an issue (ok, 63-bit... but you get my point.)
- mannykannot 10y agoIn this context, the difference between 63 bits and 64 bits is not trivial.
- yoklov 10y agoBoth are bigger than the x64 address space...
- kabdib 10y agoOften enough; "pack" files in video games are often many GB. Memory-map one of those and there you are . . .
- bonzini 10y agoThe parent is taking about >2gb on a 32-bit machine.
- icebraining 10y agoSo what? You can have up to 64GB of RAM on a 32-bit machine: https://en.wikipedia.org/wiki/Physical_Address_Extension https://en.wikipedia.org/wiki/Physical_Address_Extension
- bonzini 10y agoBut those have to be mapped a slice at a time into virtual addresses.
- loeg 10y agoIt's basically bogus to have a single object bigger or equal to half of address space (represented by size_t) in C. 32-bit platforms should detect and abort in such conditions (compiler/linker for static objects, malloc() implementation for dynamic allocations).
- icebraining 10y agoWhy? If you're running a system with PAE, half of a 32-bit address space is a small fraction of the whole addressable memory.
- loeg 10y agoIt's either addressable or it isn't. My understanding of typical PAE systems is that userspace is still limited to 32 bits of address space per process. Any system where userspace is not limited to 32 bits should have a larger than 32-bit size_t. (PAE systems are not true 32-bit platforms.)
- sfifs 10y agoOh surprisingly easily. Say you're handling a few billion cookies in RAM or manipulating DNA data.
- hsivonen 10y agoIt's quite easy to serve over 2 GB of spaces over the network. (gzip, brotli)
- tedunangst 10y agoAs far as I know, every compiler is badly broken with arrays greater than SIZE_MAX / 2, so this would be the least of your troubles.
- pascal_cuoq 10y agoSince I have it at hand, here is a list of examples of how compilers are broken if an array is larger than SIZE_MAX / 2 (which is called PTRDIFF_MAX in the post): http://trust-in-soft.com/objects-larger-than-ptrdiff_max-bytes/ http://trust-in-soft.com/objects-larger-than-ptrdiff_max-byt...
- mannykannot 10y agoIs there a circle in hell reserved for C standards committee members who add to the number of cases where 'undefined behavior' occurs in the standards?
- michaelmior 10y agoI hope not. One of the reasons for allowing undefined behaviour is to allow room for compilers to perform certain optimizations that may not be possible if a specific result were required.
- microtherion 10y agoJust how would you make this case defined? The alternatives to undefined behavior tend to be (1) being silent about the issue and letting users and implementers find out themselves (2) defining the behavior in a way that makes it difficult to implement on some architectures or (3) defining the behavior in a way that imposes costs on all architectures. None of these is particularly attractive.
- mannykannot 10y agoI would go with option 1 because it does not suggest the problem is solved. As I write this, I realize that there is a counter-argument in that introducing ptrdiff_t gives you a place to issue a warning.
- kbart 10y agoRead "undefined behavior" as "depending on architecture and compiler". It's not like anything can happen, but it's simply not to describe every architecture and every compiler into a standard. Sure, somebody is free to write an implementation where a nuke is launched every time "undefined behavior" is encountered, and they would be right according to C standard, but in real world, you pretty much know what to expect on a given system.
- Asooka 10y agoI would really love if undefined behaviour meant defined by architecture, but current compiler writers are firmly in the launch nukes camp.