9 ms·
Swift: Madness of Generic Integer
- Filligree 12y agoFor amusement value, the Haskell equivalent is: import Data.Bits import Data.List(unfoldr) f :: (Num a, Bits a) => a -> [a] f = unfoldr $ \case 0 -> Nothing n -> Just (n .&. 0xff, n `shift` 8)
- panic 12y agoThat's the inverse of the function the author wants, which ought to have type '[Word8] -> Maybe a' (where 'Nothing' is returned when the bytes exceed the range of the type in question).
- tel 12y agoYou could use the Bits instance of Integer and then convert from Integer to your final type. All types instantiating Num support `fromInteger :: Num a => Integer -> a`. Unfortunately, the design of this interface was for syntactic convenience instead of semantic strength so it is silent on what happens when you project an infinite type like Integer into a finite one like Word8. Implementers might wrap the conversion or fail with some kind of exception. A better (if less convenient) implementation would end in a Maybe type.
- Filligree 12y agoMy bad. Then I'll leave it as an exercise for the reader; the signature of the correct function is the same.
- poikniok 12y agoThis is so typical in the Haskell world, comment with a short snippet of code that does not produce the intended result, and when this is pointed out dismiss it as a trivial implementation problem.
- bkirwi 12y agoHere you go, then: -- little-endian sequence of bytes to arbitrary integral type integerWithBytes :: (Bits a, Integral a) => [Word8] -> a integerWithBytes = foldr (\byte acc -> (acc `shiftL` 8) + fromIntegral byte) 0 At least, I'm pretty sure that's what the article was trying to do... I don't think this is some amazing showcase of Haskell; after all, the article includes a (working?) implementation in C++. But it does serve as a nice counterexample to the idea -- expressed elsewhere in the thread -- that being generic is necessarily a hugely painful or complicated thing.
- codygman 12y agoYep this works: λ> import Data.Bits λ> let integerWithBytes = foldr (\byte acc -> (acc `shiftL` 8) + fromIntegral byte) 0 λ> integerWithBytes [0xFF, 0xFF, 0xFF, 0xFF] 4294967295
- codygman 12y agoI feel like you are overstating both how much this happens in Haskell and how exclusive it is to the Haskell community. Also see bkirwi's response for the working version, then my reply to him for a sample run.
- grimlck 12y agoHow does [0xFF, 0xFF, 0xFF, 0xFF], interpreted as a UInt32 turn into 16777215? I would have guessed 4294967295
- Hannan 12y agoI assume it's just a typo/incomplete edit; his graphic directly below has the bit representation 00000000111111111111111111111111 indicating [0x00, 0xFF, 0xFF, 0xFF].
- shurcooL 12y agoIt is 4294967295. http://play.golang.org/p/r8LHo8mfc9 http://play.golang.org/p/r8LHo8mfc9
- codygman 12y agoYep (credit to bkirwi): λ> import Data.Bits λ> let integerWithBytes = foldr (\byte acc -> (acc `shiftL` 8) + fromIntegral byte) 0 λ> integerWithBytes [0xFF, 0xFF, 0xFF, 0xFF] 4294967295
- shurcooL 12y agoGeneric code is like nerd sniping. I look at this and think "why would you want to write generic code for all those ints?" The integer types may look similar but they're different in more ways than they're similar. They have different bit sizes, different signedness. The CPU literally has to do different things depending if its `uint8` or `int64`. So why do you want or expect one piece of code that does it all? It's just so much easier and faster to do it like Go, have non-generic functions that do exactly what you want and as a result, get meaningful work done. It's faster to write (because you don't need to figure out how to do in a generic way), faster and easier to read, and possible to make changes to one func but not others.
- Chinjut 12y agoWhy should I as the programmer have to do different things just because the CPU has to do different things? If the logic of what I want to do is the same in multiple cases, then I only want to write it once and let the compiler figure out what to do each time I call it. (It's the whole reason I write and call functions in the first place, instead of tediously manually inlining the corresponding code at each instance!)
- alricb 12y agoBut in this case, it isn't. Two's complement signed fixed-size integers are completely different from their unsigned brethren, and confusing them is an endless source of bugs.
- pcwalton 12y ago> But in this case, it isn't. Two's complement signed fixed-size integers are completely different from their unsigned brethren, and confusing them is an endless source of bugs. They both support a meaningful left-bitshift operation, which is what the author wanted to abstract over.
- tacos 12y agoBut they do not support a common right-bitshift operation, nor sign extend. Which is what both the compiler and the poster above you are trying to make clear.
- deleted 12y ago[deleted]
- gilgoomesh 12y agoTo summarize, the author has two problems: 1. Bit shift is not part of the IntegerType protocol, when it should be (although the author could avoid the issue by accumulating the bytes in a UIntMax instead of the generic type). 2. Construction from (and conversion to) a UIntMax bit pattern is not part of the IntegerType protocol, when it should be (done correctly, this addresses the author's sign and construction complaints) The author incorrectly claims/implies that these are problems with generics or protocols or the diversity of integer types in Swift. They're really a problem of omissions in the standard library protocols that are forcing some very cumbersome workarounds. The necessary functionality exists, it just isn't part of the relevant protocols. Submit these as bugs to Apple. Edit: As a follow up, here's a version that gets around the standard library limitations using an unsafe pointer... func integerWithBytes<T: IntegerType>(bytes:[UInt8]) -> T? { if (bytes.count < sizeof(T)) { return nil } var i:UIntMax = 0 for (var j = 0; j < sizeof(T); j++) { i = i | (UIntMax(bytes[j]) << UIntMax(j * 8)) } return withUnsafePointer(&i) { ip -> T in return UnsafePointer<T>(ip).memory } } Of course, at that point, why not simply reinterpret the array buffer directly... func integerWithBytes<T: IntegerType>(bytes:[UInt8]) -> T? { if (bytes.count < sizeof(T)) { return nil } return bytes.withUnsafeBufferPointer() { bp -> T in return UnsafePointer<T>(bp.baseAddress).memory } }
- st3fan 12y agoThis is a great pragmatic example. Swift is a fantastic language with many features that let you write safe and strict code. But do not forget, Swift sits on top of a foundation of C and Objective-C. In my opinion a great Swift programmer knows when to take advantage of that. And, if written correctly, like the last example above, it is totally possible to write elegant and safe code. In my opinion there is nothing wrong with an approach that uses something like withUnsafeBufferPointer().
- barrkel 12y agoThe more protocols that are added, the more concepts there are to scare people away from what they think of as relatively simple primitives. 26 is already a scary number of concepts to tie to simple whole numbers. To be clear, the complexity is inherent in using numbers programmatically. The only real way around it would be to reduce flexibility around overloading operators, forcing people to implement bundles of related operators that are all associated with a concept (protocol). This would decrease the utility of overloading for implementing some algebras.
- ChuckMcM 12y agoWow that takes me back. Back to a conference room where we were talking about Integers in Java. If you made it a class, an Integer could carry along all this other stuff about how big it was, what the operators were, etc. But generating code for it was painful because your code had to do all of these checks when 99% of the time you probably just wanted the native integer implementation of the CPU. And Boolean's were they their own type or just a 1 bit Integer? And did that make an enum {foo, bar, baz, bletch, blech, barf, bingo} just a 3 bit integer? Integers as types can compile quickly, but then you need multiple types to handle the multiple cases. Essentially you have pre-decoded the size by making into a type. At one point you had class Number, subclasses Real, Cardinal, and Complex, and within those a constructor which defined their precision. But I think everyone agreed it wasn't going to replace Fortran. The scripting languages get pretty close to making this a non-visible thing, at the cost of some execution speed. Swift took it to an extreme, which I understand, but I probably wouldn't have gone there myself. The old char, short, long types seem so quaint now.
- viraptor 12y agoJava's initial approach/implementation is just one way to do that. If you actually use objects to represent your integers, then of course it will have to do extra checks. But Java is not the best example here, because with a better language you can drop all those checks at the compile time and get the "native integer implementation" in the code where it's used. Even C can do a very basic compile-time generics since C11. There's no runtime overhead at all in that case. (http://www.robertgamble.net/2012/01/c11-generic-selections.html http://www.robertgamble.net/2012/01/c11-generic-selections.h...)
- barrkel 12y agoCompile-time polymorphism has no need to incur the kinds of costs you're hinting at here. Consider e.g. Ada attributes, range types, 'First, 'Last etc.: these things are all resolvable at compile time.
- pjmlp 12y agoJust because they look like objects to the programmer they don't need to be at implementation level, like the Pascal family of languages.
- thought_alarm 12y agoThe Swift equivalent of his first `NSData` example is essentially this: func integerWithBytes<T:IntegerType>(bytes:[UInt8]) -> T { let valueFromArrayPointer = { (arrayPointer: UnsafePointer<UInt8>) in return unsafeBitCast(arrayPointer, UnsafePointer<T>.self).memory } return valueFromArrayPointer(bytes) } let bytes:[UInt8] = [0x00, 0x01, 0x00, 0x00] let result: UInt32 = integerWithBytes(bytes) assert(result == 256)
- lectrick 12y agoI guess this sort of gets at the crux of the issue: Do you want it to be more like a scripting language (which would basically give you the mathematical equivalent of "integer" including unlimited size) at the cost of speed, or do you want it to be closer to the implementation in the CPU, which entails dealing with 8/16/32/64 bit limits and sign bits? Why not have a way to do both? You can get an easy-to-use Int when speed is less of a concern, and can deal with Int16's, Int32's, UInt32's and whatnot when the job demands it.
- iopq 12y agoIf you have tagged integers you can have 31 bit ints that are super fast (one shift away from the actual number). The performance cost (allocation) is only when it overflows.
- frozenport 12y ago"All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections." ~David John Wheeler
- cjensen 12y ago> More or less, what I want to archive can be done with old world NSData: data.getBytes(&i, length: sizeofValue(i)) That doesn't work in C/C++ if you are using a modern optimizer. C does not have the other Swift issues the author mentions, so shifting into the largest int and casting from there does work.
- IgorPartola 12y agoSo no macros? Seriously, isn't that the way to solve this?
- Sephiroth87 12y agoNo macros in swift
- Animats 12y agoI once, many years ago, wrote something titled "Type Integer Considered Harmful". (This was way back during the 16-32 bit transition). My position was that the user should declare integer types with ranges (as in Pascal and Ada), and it was the compiler's job to insure that intermediate variables must not overflow unless the user-defined ranges would also be violated. Overflowing a user range would be an error. The goal was to get the same answer on all platforms regardless of the underlying architecture. The main implication is that expressions with more than one operator tend to need larger intermediate temporary variables. (For the following examples, assume all the variables are the same integer type.) For "a = b * c", the expression "b * c" is limited by the size of "a", so you don't need a larger intermediate. But "a = (b * c)/d" requires an temporary big enough to handle "b * c", which may be bigger than "a". Compilers could impose some limit on how big an intermediate they supported. This hides the underlying machine architecture and makes arithmetic behave consistently. Either you get the right answer numerically, or you get an overflow exception. Because integer ranges weren't in C/C++, this approach didn't get much traction. Dealing with word size differences became less of an issue when the 24-bit, 36-bit, 48-bit and 60-bit machines died off in favor of the 32/64 bit standard. So this never became necessary. It's still a good way to think about integer arithmetic.
- deathanatos 12y agoEspecially in higher-level languages, I've wished language designers would move toward using variable-size/bignum integers instead of fixed size integers (Python does this, for example). It eliminates overflow, and the need to analyze each int to see if it will overflow the type you're sticking it into. I wouldn't mind being able to have a "RangedInt<min, max>" type either, in addition. If the bounds are tight enough, the compiler could just use the next-bigger machine integer type (and do bounds-checking, please!). I think a integral type that was always modulo the max would be useful in many applications as well (i.e., unsigned, and overflow is well-defined to wrap, but you explicitly opt-in to this behavior.) You can imagine, int: signed, bounded only by memory ranged_int<min, max>: integer type capable of holding anything in [min, max]. Over/underflow is an error (exception? panic? [1]) modulo_int<min, max>: unsigned, overflow wraps. (Mathematicians probably have a better name here… "ring"?) "usize" or "size_t": capable of holding any memory address, so useful for indexes. native::uint8, native::uint16, etc: whatever your hardware gives you, if you really need it. The default type a new coder would grab for (int) won't overflow on them, although there are questions about what does some_array[int_index] do, esp. if it overflows the index type. [1] Rust has some interesting thoughts here, and I thought they did a good job of detailing the consequences and their reasoning; see https://github.com/rust-lang/rfcs/blob/master/text/0560-integer-overflow.md https://github.com/rust-lang/rfcs/blob/master/text/0560-inte....
- chvid 12y agoI think you all are being too nice to Apple. I had a similar experience as the blog post author. I spent many hours battling generics and the huge forest of (undocumented) protocols to do something seemingly trivial. I just gave up rather than try to pin down exactly what was wrong in a long and detailed blog post. The prevailing answer to everything seem to be: Write a bug report to Apple and use Objective-C (or Swift's UnsafePointer and related). This ignores what I think really is the issue here: Swift has an overly complex type system. This picture: http://swiftdoc.org/type/Int/hierarchy/ http://swiftdoc.org/type/Int/hierarchy/ Tells a lot. And this is from unofficial documentation that has been generated from Swift libraries. When you read the documentation Apple provides there is little explanation of this huge protocol hierarchy and the rationale behind it. It seems to me that: Swift has been released in a rush with bugs even in the core language and compiler. Lacking documentation. And of course even a larger number of bugs in the IDE support, debugging etc. Secondly: Swift battles the problem of easy-to-understand typesafe generics like so many other languages only it has it much worse: It carries a lot of stuff from Objective-C and it has to support easy interoperability. Plus it has ideas like not allowing implicit conversion of number types (requiring an integer added to a double to be explicitly converted to a double) causing the big type system to show it's messy head again and again. I really want to love Swift but it will take years for Swift to be as clean and productive as Objective-C. I my opinion what Apple should have done was to create the "CoffeeScript" of Objective-C. A language that essentially was Objective-C in terms of language features but with a concise syntax.
- rismay 12y agoI'm glad I'm not the only one having this issue.
- bluehex 12y agoI wouldn't expect a much better design from a language developed behind closed doors with no community input. Now all we can do is file bugs against Apple, and hope they improve it; they who chose to release a new language with but three months of public beta. They obviously didn't care much to have their designs tested or incorporate feedback then.