5 ms·

Nope! (TIL) http://stackoverflow.com/questions/5516044/system-where-1-byte-8-bit http://stackoverflow.com/questions/5516044/system-where-1-by... I think in the

Nope! (TIL) http://stackoverflow.com/questions/5516044/system-where-1-byte-8-bit http://stackoverflow.com/questions/5516044/system-where-1-by... I think in the context of the original answer there's also the problem that char types aren't necessarily 8 bits wide, e.g. wchar etc.

pdonis 10y ago
Ok, I should have been a bit more specific: aren't all bytes on machines that Python can realistically run on 8 bits wide?
- lisper 10y ago
  No. All modern machines can address 8-bit-wide bytes for historical reasons, but they can also address wider bytes. 32 and 64 bit bytes are common. We call them "words" instead of "bytes", but they're really the same thing: a chunk of data that can be operated on as a single unit by the hardware). And having that capability exposed as a software abstraction can be extremely useful, particularly in applications like cryptography. EDIT: Downvotes? Seriously?
  - pdonis 10y ago
    > a chunk of data that can be operated on as a single unit by the hardware I didn't realize that this concept was referred to by the term "byte". But this clarifies how you are using that term. Given that usage, I'm not sure why 8-bit bytes are an issue at all for a language like Python that is supposed to be hardware-agnostic; the whole point is that programs should not care about how the data is physically represented in the hardware. Python 3 got rid of the distinction between int and long for the same reason. The only reason the "bytes" object exists at all is to represent binary data, for example data that gets read from/written to files, network sockets, and other I/O, and AFAIK bytes in such data are always defined to be 8-bit chunks of data. (If you really insist on having Python representations of hardware data chunks, there's the ctypes module.)
    - lisper 10y ago
      > I didn't realize that this concept was referred to by the term "byte". It is common to see "byte" used to mean 8-bit-byte, and also to see the term used for wider units of binary data. If you want to be unambiguous, the word "octet" is commonly employed to mean 8-bit byte. > a language like Python that is supposed to be hardware-agnostic That is exactly why you want wide bytes. If you want to represent an integer >255 as fixed-width binary data and you don't have wide bytes, then you have to choose an endianness convention. If you have wide bytes then you can be endianness-agnostic. > AFAIK bytes in such data are always defined to be 8-bit chunks of data This is usually true, but not always. For example, UTF-16 and UTF-32 data is specified as 16-bit and 32-bit bytes. This is why you need a BOM. > there's the ctypes module Yes, this is why I said it's not a serious problem.
      - pdonis 10y ago
        > If you want to represent an integer >255 as fixed-width binary data But Python 3 doesn't represent integers that way, because Python 3 doesn't put any size limit on integers. There's no way to represent integers of potentially unbounded size as fixed-width binary data. This is one of the things I meant by "hardware agnostic"--I didn't mean "able to represent the different register sizes of different hardware"; I meant "having a representation that doesn't have any concept of register size at all". (Python 2 does have a separate int type, which IIRC is either 32 or 64 bits wide depending on the platform. AFAIK this is just a wrapper around the underlying C int, so it isn't affected by any 8-bit byte restriction.) > UTF-16 and UTF-32 data is specified as 16-bit and 32-bit bytes. This is why you need a BOM. Um, no. UTF-16 and UTF-32 data is specified as 16-bit and 32-bit code points. You need a Byte Order Mark (BOM) to specify in which order the 8-bit bytes that make up a 16-bit or 32-bit code point appear. From the Unicode FAQ[0]: "Q: What is a UTF? "A: A Unicode transformation format (UTF) is an algorithmic mapping from every Unicode code point (except surrogate code points) to a unique byte sequence. The ISO/IEC 10646 standard uses the term “UCS transformation format” for UTF; the two terms are merely synonyms for the same concept." Further down, under the table listing the UTFs and their properties: "In the table <BOM> indicates that the byte order is determined by a byte order mark" And in the answer to the next question after that: "UTF-16 and UTF-32 use code units that are two and four bytes long respectively." [0]: http://unicode.org/faq/utf_bom.html http://unicode.org/faq/utf_bom.html
  - pdonis 10y ago
    > Downvotes? Seriously? I upvoted you to compensate. :-)

pdonis 10y ago

Ok, I should have been a bit more specific: aren't all bytes on machines that Python can realistically run on 8 bits wide?

lisper 10y ago
No. All modern machines can address 8-bit-wide bytes for historical reasons, but they can also address wider bytes. 32 and 64 bit bytes are common. We call them "words" instead of "bytes", but they're really the same thing: a chunk of data that can be operated on as a single unit by the hardware). And having that capability exposed as a software abstraction can be extremely useful, particularly in applications like cryptography. EDIT: Downvotes? Seriously?
- pdonis 10y ago
  > a chunk of data that can be operated on as a single unit by the hardware I didn't realize that this concept was referred to by the term "byte". But this clarifies how you are using that term. Given that usage, I'm not sure why 8-bit bytes are an issue at all for a language like Python that is supposed to be hardware-agnostic; the whole point is that programs should not care about how the data is physically represented in the hardware. Python 3 got rid of the distinction between int and long for the same reason. The only reason the "bytes" object exists at all is to represent binary data, for example data that gets read from/written to files, network sockets, and other I/O, and AFAIK bytes in such data are always defined to be 8-bit chunks of data. (If you really insist on having Python representations of hardware data chunks, there's the ctypes module.)
  - lisper 10y ago
    > I didn't realize that this concept was referred to by the term "byte". It is common to see "byte" used to mean 8-bit-byte, and also to see the term used for wider units of binary data. If you want to be unambiguous, the word "octet" is commonly employed to mean 8-bit byte. > a language like Python that is supposed to be hardware-agnostic That is exactly why you want wide bytes. If you want to represent an integer >255 as fixed-width binary data and you don't have wide bytes, then you have to choose an endianness convention. If you have wide bytes then you can be endianness-agnostic. > AFAIK bytes in such data are always defined to be 8-bit chunks of data This is usually true, but not always. For example, UTF-16 and UTF-32 data is specified as 16-bit and 32-bit bytes. This is why you need a BOM. > there's the ctypes module Yes, this is why I said it's not a serious problem.
    - pdonis 10y ago
      > If you want to represent an integer >255 as fixed-width binary data But Python 3 doesn't represent integers that way, because Python 3 doesn't put any size limit on integers. There's no way to represent integers of potentially unbounded size as fixed-width binary data. This is one of the things I meant by "hardware agnostic"--I didn't mean "able to represent the different register sizes of different hardware"; I meant "having a representation that doesn't have any concept of register size at all". (Python 2 does have a separate int type, which IIRC is either 32 or 64 bits wide depending on the platform. AFAIK this is just a wrapper around the underlying C int, so it isn't affected by any 8-bit byte restriction.) > UTF-16 and UTF-32 data is specified as 16-bit and 32-bit bytes. This is why you need a BOM. Um, no. UTF-16 and UTF-32 data is specified as 16-bit and 32-bit code points. You need a Byte Order Mark (BOM) to specify in which order the 8-bit bytes that make up a 16-bit or 32-bit code point appear. From the Unicode FAQ[0]: "Q: What is a UTF? "A: A Unicode transformation format (UTF) is an algorithmic mapping from every Unicode code point (except surrogate code points) to a unique byte sequence. The ISO/IEC 10646 standard uses the term “UCS transformation format” for UTF; the two terms are merely synonyms for the same concept." Further down, under the table listing the UTFs and their properties: "In the table <BOM> indicates that the byte order is determined by a byte order mark" And in the answer to the next question after that: "UTF-16 and UTF-32 use code units that are two and four bytes long respectively." [0]: http://unicode.org/faq/utf_bom.html http://unicode.org/faq/utf_bom.html
- pdonis 10y ago
  > Downvotes? Seriously? I upvoted you to compensate. :-)