14 ms·

A false midnight

82 points by lifthrasiir 13y ago

davvid 13y ago
Guido's got the right idea. "just do it already"
- jessaustin 13y ago
  I agree not changing it until 3.6 is misguided, although ISTM this isn't a good candidate for backporting. I guess deprecation makes sense, but I don't see why adding a DeprecationWarning is some giant change that couldn't have gone into 3.4 at the last minute, or for that matter into 3.4.1 whenever that comes.
TrainedMonkey 13y ago
This clearly violates rule of least astonishment, however as Linus says: "we do not break userspace".
- jeffdavis 13y ago
  It would be interesting to compare how many cases would be silently fixed by such a change.
- nathancahill 13y ago
  In this case, I think the userspace is already broken, but people haven't realized their code is broken yet.
- einhverfr 13y ago
  What would happen if an API turned out to be based on a clearly incorrect and useless reading of a POSIX standard? I would think that even Linus might fix something in that case.
  - caf 13y ago
    Right, Linus's actual guideline seems to be "If no-one notices we broke backwards-compatibility, we didn't break backwards compatibility", along the lines of the tree-falls-in-the-forest saw.
- general_failure 13y ago
  Every syscall fix in kernel breaks user space in some subtle way.
whoopdedo 13y ago
I've become a advocate for the idea that the only values which should evaluate in boolean context are actual booleans. If you need to test for the number zero or a null value, you explicitly write 'value == 0' or 'value is null'. Is it more verbose? Yes. But it avoids any ambiguity about what you intend to evaluate. Typing speed should not be a determinant in the language you use.
- eru 13y ago
  In Haskell, you can have your cake and eat it too. Normally, the language follows your comments philosophy, but it's really easy to add arbitrary truthiness and falsiness definitions: class ToBool a where toBool :: a -> Bool if' :: ToBool a => a -> b -> b -> b if' cond th el = if toBool cond then th else el And add instances like these: instance ToBool Int where toBool 0 = False ToBool _ = True instance ToBool String where toBool "" = False toBool _ = True Or even: instance ToBool TimeStamp where toBool = not . isMidnight Lisp, of course, can do similar things via Macros.
  - mercurial 13y ago
    On the other hand, any attempt at writing something like this in production code warrants the firing squad.
    - eru 13y ago
      Why? It seems unnecessary, but not overly hard to read. Some re-education should be enough.
      - gleenn 13y ago
        It's bad because it beckons bad behavior. Instead of doing the simple thing and changing your if-statement which puts the idiosyncrasy in the relevant piece of code, you are tucking away the behavior out of sight when something like this almost definitely belongs directly in front of your eye-balls. This is a bit of a strawman, but being able to do class Foo(object): def __bool__(self): return random.random() > 0.5 shows how crazy you could be. If you really want something weird, put it in the if-statement
einhverfr 13y ago
First, I can see some benefits to a false midnight. The case of truncated timestamps discussed on the Python list is actually useful. However, the problem with the Python truth value of a date here goes way deeper than what folks have generally discussed and make the behavior here unsafe for any use. When designing a module, contracts are important. You don't want to ever have to break a contract if you can avoid it. Sometimes it will be unavoidable, but you hope not. What this means is that your contracts need to be clear, precise, and meaningful, and the code needs to implement the contract. I good contract would have read something like: A time value is false if it represents a midnight, either when unqualified by timezone or when midnight GMT. This gives you a specification. It gives you something you can use and count on. Maybe it is a killer feature for someone. However the implementation instead was: A time value is false if and only if it evaluates to zero after subtracting the timezone offset if it has one. Think about what this means when you have a negative vs a positive offset. It means that negative timezone offsets allow a date to always be true, but positive ones allow it to be false at UTC midnight. The real takeaway from this really should be the fact that API documentation should be declarative in nature and not procedural. Don't tell me how you get an answer. Tell me what the answer means. if you do the latter, then nothing can ever be fixed but probably cannot be safely used either. The problem then is first and foremost a problem of API design and a design which is fundamentally flawed. This is why absent where the algorithm is critical to the contract it should not be mentioned. Explain what is expected and what it means, not how you get at the answer.
- jessaustin 13y ago
  ...negative timezone offsets allow a date to always be true, but positive ones allow it to be false at UTC midnight. I haven't had my tea yet, but is this true? ISTM that if a positive offset can be false in the AM, a negative one might be false in the PM.
  - plorkyeran 13y ago
    The actual check is for if time_in_minutes == tz_offset_in_minutes. The problem is that times are always positive, so this check doesn't work if the offset is negative. The correct implementation would be time_in_minutes == (tz_offset_in_minutes + minutes_in_day) % minutes_in_day.
buro9 13y ago
How does time exist without date? And how does time have a timezone without a date? It's the small things in Python that surprise me. I understand the desire to have truthiness to determine whether it's been set, but a minimum value is a better test than a zero value. Testing for a "seconds since 1970" would be less bad, and perhaps if you're going to detach time from a concept of date any negative value could be considered the unset/falsey value (which is probably what people testing the value think they're testing).
- cowsandmilk 13y ago
  > How does time exist without date? How does an alarm clock app store the idea that it is supposed to go off at 6 am every day?
  - e12e 13y ago
    6 am where? Where the app (and presumably you) are, or where you were when you set the alarm? Or better yet, if the alarm is set on a server somewhere: did you mean 6 am where your phone is, where the server is, or where you where when you set the alarm (possibly on a different machine with a different locale). Arguably "in N hours from now" is an unambiguous "time". Possibly so is "in N hours from now, and then every 24 hours" is too, but most places that'll be wrong come daylight saving. An alarm clock app (vs a regular, physical alarm clock) is possibly a bad example for "simple time". Because most apps are connected to a clock that automagically updates somehow, and will then need to "decide" if 6 am is 6 am or some other time. Maybe you just crossed one timezone and don't want to change the (subjective) time you get up, but still need to be able to tell the local time without having to calculate what the local time actually is...
rplnt 13y ago
This problem is iterated over and over. It's mentioned in every discussion about why python is bad. Even here on HN I've read it a few times, most recently about two weeks ago I think. I don't know if it's annoying or is it good that more people know about this behavior.
- meowface 13y ago
  >This problem is iterated over and over. It's mentioned in every discussion about why python is bad. What? It's quite a stretch to say "Python is bad" because objects have implicit truthiness and that in this one rare occasion it's implemented poorly in the standard library.
  - rplnt 13y ago
    Yes it is. One could say that the fact this silly problem is brought up every time proves that python is not that bad.
gleenn 13y ago
I firmly think languages like Ruby and Clojure get it right, nil and false are falsy, everything else is true, end of story. This is one of my biggest pet-peeves with some languages. PHP is horrifically bad, there is a huge chart to consult if things get even remotely weird.
- einhverfr 13y ago
  I like PostgreSQL's SQL dialect. You can't treat non-bools as bools, but you can create casts yourself if you want to. Therefore if I want to cast a varchar as bool, I could decide what I mean by it and build the logic to do this. It would be preferable to do this with a domain though instead of a base type so as to avoid compatibility issues down the road.
- simias 13y ago
  I think that's inherited from C which itself inherits it from various CPU architectures (although I guess nowadays many CPU architectures are tailored for machine code generated by C compilers...). I agree that for higher level languages capable of offering a true "nil" (and not a macro for 0 like NULL in C) it makes sense to make all other values "truthy". I'm not a python coder but it surprises me it doesn't force comparisons explicitly like Java. After all "explicit is better than implicit". That being said, what's the point of having nil and false? Seems a bit redundant to me.
  - leoc 13y ago
    > That being said, what's the point of having nil and false? Seems a bit redundant to me. D: Here's a good reason to begin with: https://en.wikipedia.org/wiki/Open_world_assumption https://en.wikipedia.org/wiki/Open_world_assumption (In case wiki doeesn't make it clear enough, the CWA is squeaking mad in any situation where you cannot, in fact, determine the truth value of every proposition. Just because SQL does it doesn't make it rational.)
  - ufo 13y ago
    C treats 0 as falsy so I would say that its more likely that Pythons behaviour is based on it - the Ruby and Clojure way has more to do with Scheme > That being said, what's the point of having nil and false? Seems a bit redundant to me. In some languages, like Common Lisp, nil and false are the same thing. However, it can be useful to separate the "this is a false value" from "this value does not exist", specially in a dynamically typed language. For example, in Lua you get a "nil" if you access a table field that does not exist and you delete a table field if you store null to it. Before they added a separate boolean type, it was really annoying to work with boolean arrays because storing nil in the array would make it full of holes and would confuse most library functions.
logicallee 13y ago
This is interesting because possibly no one has ever used it in code logic, for anything, ever. (Except to work around the faulty falsy definition.) The way this would be apparent is to simply look through a statistical sampling of uses. If 80% of the time the code is used in a boolean context, the code is actually broken, meanwhile the falsiness is used, correctly, in 0.0% of the code, then you can silently fix 80% of errors without breaking anything. Moreover, if falsiness is not explicitly relied on, but just caught kind of as an exception, then removing the false sentinel value may only cause errors by creating "exception-handling" that never runs. If this is the only "cost", while you can subtlely fix 80% of code that uses the false value (wrongly), then do it. It would be interesting to know if anyone was using the sentinel value (correctly) and relying on it.
- pekk 13y ago
  Tons of people use 'is None' correctly as a sentinel value
  - logicallee 13y ago
    That wasn't what I was discussing.
- maxerickson 13y ago
  It's stated in the article that a couple of people had used the false value in code that was not distributed.
wodenokoto 13y ago
So, when is this smart? The only remotely meaningful use-case for midnight being false I can think about is if you need way to check if a time is midnight? But it still seems easier to just check if time == 0.0
thanatropism 13y ago
Duck typing is borderline psychotic. Is it a number? Is it not a number? What's the logarithm of an hour?
- anon4 13y ago
  > What is the logarithm of an hour? Well, depends. If you try to pass a datetime.time object to the math.log function, you get TypeError: a float is required On the other hand, if you store your hour as a number, then it works just fine. It's still not meaningful, mind.
- yxhuvud 13y ago
  It is not duck typing that is psychotic, but automatic conversion between different types.
otikik 13y ago
> Interpreted, "duck typing" languages often have some idiosyncrasies in their definitions of "truth" As do most other, non-duck-typing languages.
pdeuchler 13y ago
I'm going to kind of echo Paul Moore's argument here, and say this entire discussion is missing the forest for the trees. Duck typing exists so you can code freely and with expression, but the obvious converse of that is it allows you to shoot yourself in the foot. From my POV it seems like people here are complaining that when they point the gun at their foot it seems to randomly go off when testing date values, and the simple answer to that would be "Well, then don't point the gun at your foot!" IM(not so)HO the Zen of Python operates as an excellent counterbalance to the kind of pitfalls Python can lead you to, the "false midnight" being a wonderful example of said pitfalls that is completely removed when following PEP 20. >> "Explicit is better than implicit." Dates IRL cannot be false, so checking for falsiness is ambiguous. Perhaps you meant to check if it was "not None". And look! The wonderful expressiveness of Python allows such a thing! `if timeobj:` is a snippet that has some heavy assumptions behind it, and if I'm reading your code this is very much not explicit to me what you're trying to do. >> "Simple is better than complex." Dealing with dates/time will always be nasty business. Let's not add to that, and make things as simple as possible. If you are explicitly checking for midnight, perhaps a constant value set to `datetime.time(0,0,0)` would be useful for a conditional. >> "There should be one-- and preferably only one --obvious way to do it." There are Pythonic ways to check for `None` and there are ways to compare date time objects. The key word for me here is "obvious", since, again, the idea of a date having a boolean value is entirely foreign to the real world, and more or less a shortcut found only in computing. Don't create an edge case that you don't have to. Now granted, the stated behavior can definitely be seen to also violate PEP 20, but simply avoiding this entire issue renders that point moot. tl;dr: People complaining about unexpected results when writing bad Python code are barking up the wrong tree. Perhaps you should try a different language, or learn to code in a more Pythonic style? Both of these avenues would be much more profitable not only in the short term, but also the long term.
- ufo 13y ago
  I agree that maybe you should write in a different, more strongly typed, language but saying that people encountering this bug are not writing code in a Pythonic style is too much like victim blaming, IMO. Its a bit like those times when people say that if only you were a good programmer following "proper" C++ idioms you will never ever encounter memory leaks or segmentation faults. The default behaviour for objects in Python is being truthy and testing their existence with `if x` instead of `if x is not None` is definitely encouraged by the language (otherwise, why would it even let you use objects in a boolean context?). Having something that behaves like the default except once in a blue moon is just a trap and its unfair to blame it on the programmer.

A false midnight

82 points by lifthrasiir 13y ago

davvid 13y ago

Guido's got the right idea. "just do it already"

jessaustin 13y ago
I agree not changing it until 3.6 is misguided, although ISTM this isn't a good candidate for backporting. I guess deprecation makes sense, but I don't see why adding a DeprecationWarning is some giant change that couldn't have gone into 3.4 at the last minute, or for that matter into 3.4.1 whenever that comes.

TrainedMonkey 13y ago

This clearly violates rule of least astonishment, however as Linus says: "we do not break userspace".

jeffdavis 13y ago
It would be interesting to compare how many cases would be silently fixed by such a change.
nathancahill 13y ago
In this case, I think the userspace is already broken, but people haven't realized their code is broken yet.
einhverfr 13y ago
What would happen if an API turned out to be based on a clearly incorrect and useless reading of a POSIX standard? I would think that even Linus might fix something in that case.
- caf 13y ago
  Right, Linus's actual guideline seems to be "If no-one notices we broke backwards-compatibility, we didn't break backwards compatibility", along the lines of the tree-falls-in-the-forest saw.
general_failure 13y ago
Every syscall fix in kernel breaks user space in some subtle way.

whoopdedo 13y ago

I've become a advocate for the idea that the only values which should evaluate in boolean context are actual booleans. If you need to test for the number zero or a null value, you explicitly write 'value == 0' or 'value is null'. Is it more verbose? Yes. But it avoids any ambiguity about what you intend to evaluate. Typing speed should not be a determinant in the language you use.

eru 13y ago
In Haskell, you can have your cake and eat it too. Normally, the language follows your comments philosophy, but it's really easy to add arbitrary truthiness and falsiness definitions: class ToBool a where toBool :: a -> Bool if' :: ToBool a => a -> b -> b -> b if' cond th el = if toBool cond then th else el And add instances like these: instance ToBool Int where toBool 0 = False ToBool _ = True instance ToBool String where toBool "" = False toBool _ = True Or even: instance ToBool TimeStamp where toBool = not . isMidnight Lisp, of course, can do similar things via Macros.
- mercurial 13y ago
  On the other hand, any attempt at writing something like this in production code warrants the firing squad.
  - eru 13y ago
    Why? It seems unnecessary, but not overly hard to read. Some re-education should be enough.
    - gleenn 13y ago
      It's bad because it beckons bad behavior. Instead of doing the simple thing and changing your if-statement which puts the idiosyncrasy in the relevant piece of code, you are tucking away the behavior out of sight when something like this almost definitely belongs directly in front of your eye-balls. This is a bit of a strawman, but being able to do class Foo(object): def __bool__(self): return random.random() > 0.5 shows how crazy you could be. If you really want something weird, put it in the if-statement

einhverfr 13y ago

First, I can see some benefits to a false midnight. The case of truncated timestamps discussed on the Python list is actually useful. However, the problem with the Python truth value of a date here goes way deeper than what folks have generally discussed and make the behavior here unsafe for any use. When designing a module, contracts are important. You don't want to ever have to break a contract if you can avoid it. Sometimes it will be unavoidable, but you hope not. What this means is that your contracts need to be clear, precise, and meaningful, and the code needs to implement the contract. I good contract would have read something like: A time value is false if it represents a midnight, either when unqualified by timezone or when midnight GMT. This gives you a specification. It gives you something you can use and count on. Maybe it is a killer feature for someone. However the implementation instead was: A time value is false if and only if it evaluates to zero after subtracting the timezone offset if it has one. Think about what this means when you have a negative vs a positive offset. It means that negative timezone offsets allow a date to always be true, but positive ones allow it to be false at UTC midnight. The real takeaway from this really should be the fact that API documentation should be declarative in nature and not procedural. Don't tell me how you get an answer. Tell me what the answer means. if you do the latter, then nothing can ever be fixed but probably cannot be safely used either. The problem then is first and foremost a problem of API design and a design which is fundamentally flawed. This is why absent where the algorithm is critical to the contract it should not be mentioned. Explain what is expected and what it means, not how you get at the answer.

jessaustin 13y ago
...negative timezone offsets allow a date to always be true, but positive ones allow it to be false at UTC midnight. I haven't had my tea yet, but is this true? ISTM that if a positive offset can be false in the AM, a negative one might be false in the PM.
- plorkyeran 13y ago
  The actual check is for if time_in_minutes == tz_offset_in_minutes. The problem is that times are always positive, so this check doesn't work if the offset is negative. The correct implementation would be time_in_minutes == (tz_offset_in_minutes + minutes_in_day) % minutes_in_day.

buro9 13y ago

How does time exist without date? And how does time have a timezone without a date? It's the small things in Python that surprise me. I understand the desire to have truthiness to determine whether it's been set, but a minimum value is a better test than a zero value. Testing for a "seconds since 1970" would be less bad, and perhaps if you're going to detach time from a concept of date any negative value could be considered the unset/falsey value (which is probably what people testing the value think they're testing).

cowsandmilk 13y ago
> How does time exist without date? How does an alarm clock app store the idea that it is supposed to go off at 6 am every day?
- e12e 13y ago
  6 am where? Where the app (and presumably you) are, or where you were when you set the alarm? Or better yet, if the alarm is set on a server somewhere: did you mean 6 am where your phone is, where the server is, or where you where when you set the alarm (possibly on a different machine with a different locale). Arguably "in N hours from now" is an unambiguous "time". Possibly so is "in N hours from now, and then every 24 hours" is too, but most places that'll be wrong come daylight saving. An alarm clock app (vs a regular, physical alarm clock) is possibly a bad example for "simple time". Because most apps are connected to a clock that automagically updates somehow, and will then need to "decide" if 6 am is 6 am or some other time. Maybe you just crossed one timezone and don't want to change the (subjective) time you get up, but still need to be able to tell the local time without having to calculate what the local time actually is...

rplnt 13y ago

This problem is iterated over and over. It's mentioned in every discussion about why python is bad. Even here on HN I've read it a few times, most recently about two weeks ago I think. I don't know if it's annoying or is it good that more people know about this behavior.

meowface 13y ago
>This problem is iterated over and over. It's mentioned in every discussion about why python is bad. What? It's quite a stretch to say "Python is bad" because objects have implicit truthiness and that in this one rare occasion it's implemented poorly in the standard library.
- rplnt 13y ago
  Yes it is. One could say that the fact this silly problem is brought up every time proves that python is not that bad.

gleenn 13y ago

I firmly think languages like Ruby and Clojure get it right, nil and false are falsy, everything else is true, end of story. This is one of my biggest pet-peeves with some languages. PHP is horrifically bad, there is a huge chart to consult if things get even remotely weird.

einhverfr 13y ago
I like PostgreSQL's SQL dialect. You can't treat non-bools as bools, but you can create casts yourself if you want to. Therefore if I want to cast a varchar as bool, I could decide what I mean by it and build the logic to do this. It would be preferable to do this with a domain though instead of a base type so as to avoid compatibility issues down the road.
simias 13y ago
I think that's inherited from C which itself inherits it from various CPU architectures (although I guess nowadays many CPU architectures are tailored for machine code generated by C compilers...). I agree that for higher level languages capable of offering a true "nil" (and not a macro for 0 like NULL in C) it makes sense to make all other values "truthy". I'm not a python coder but it surprises me it doesn't force comparisons explicitly like Java. After all "explicit is better than implicit". That being said, what's the point of having nil and false? Seems a bit redundant to me.
- leoc 13y ago
  > That being said, what's the point of having nil and false? Seems a bit redundant to me. D: Here's a good reason to begin with: https://en.wikipedia.org/wiki/Open_world_assumption https://en.wikipedia.org/wiki/Open_world_assumption (In case wiki doeesn't make it clear enough, the CWA is squeaking mad in any situation where you cannot, in fact, determine the truth value of every proposition. Just because SQL does it doesn't make it rational.)
- ufo 13y ago
  C treats 0 as falsy so I would say that its more likely that Pythons behaviour is based on it - the Ruby and Clojure way has more to do with Scheme > That being said, what's the point of having nil and false? Seems a bit redundant to me. In some languages, like Common Lisp, nil and false are the same thing. However, it can be useful to separate the "this is a false value" from "this value does not exist", specially in a dynamically typed language. For example, in Lua you get a "nil" if you access a table field that does not exist and you delete a table field if you store null to it. Before they added a separate boolean type, it was really annoying to work with boolean arrays because storing nil in the array would make it full of holes and would confuse most library functions.

logicallee 13y ago

This is interesting because possibly no one has ever used it in code logic, for anything, ever. (Except to work around the faulty falsy definition.) The way this would be apparent is to simply look through a statistical sampling of uses. If 80% of the time the code is used in a boolean context, the code is actually broken, meanwhile the falsiness is used, correctly, in 0.0% of the code, then you can silently fix 80% of errors without breaking anything. Moreover, if falsiness is not explicitly relied on, but just caught kind of as an exception, then removing the false sentinel value may only cause errors by creating "exception-handling" that never runs. If this is the only "cost", while you can subtlely fix 80% of code that uses the false value (wrongly), then do it. It would be interesting to know if anyone was using the sentinel value (correctly) and relying on it.

pekk 13y ago
Tons of people use 'is None' correctly as a sentinel value
- logicallee 13y ago
  That wasn't what I was discussing.
maxerickson 13y ago
It's stated in the article that a couple of people had used the false value in code that was not distributed.

wodenokoto 13y ago

So, when is this smart? The only remotely meaningful use-case for midnight being false I can think about is if you need way to check if a time is midnight? But it still seems easier to just check if time == 0.0

thanatropism 13y ago

Duck typing is borderline psychotic. Is it a number? Is it not a number? What's the logarithm of an hour?

anon4 13y ago
> What is the logarithm of an hour? Well, depends. If you try to pass a datetime.time object to the math.log function, you get TypeError: a float is required On the other hand, if you store your hour as a number, then it works just fine. It's still not meaningful, mind.
yxhuvud 13y ago
It is not duck typing that is psychotic, but automatic conversion between different types.

otikik 13y ago

> Interpreted, "duck typing" languages often have some idiosyncrasies in their definitions of "truth" As do most other, non-duck-typing languages.

pdeuchler 13y ago

I'm going to kind of echo Paul Moore's argument here, and say this entire discussion is missing the forest for the trees. Duck typing exists so you can code freely and with expression, but the obvious converse of that is it allows you to shoot yourself in the foot. From my POV it seems like people here are complaining that when they point the gun at their foot it seems to randomly go off when testing date values, and the simple answer to that would be "Well, then don't point the gun at your foot!" IM(not so)HO the Zen of Python operates as an excellent counterbalance to the kind of pitfalls Python can lead you to, the "false midnight" being a wonderful example of said pitfalls that is completely removed when following PEP 20. >> "Explicit is better than implicit." Dates IRL cannot be false, so checking for falsiness is ambiguous. Perhaps you meant to check if it was "not None". And look! The wonderful expressiveness of Python allows such a thing! `if timeobj:` is a snippet that has some heavy assumptions behind it, and if I'm reading your code this is very much not explicit to me what you're trying to do. >> "Simple is better than complex." Dealing with dates/time will always be nasty business. Let's not add to that, and make things as simple as possible. If you are explicitly checking for midnight, perhaps a constant value set to `datetime.time(0,0,0)` would be useful for a conditional. >> "There should be one-- and preferably only one --obvious way to do it." There are Pythonic ways to check for `None` and there are ways to compare date time objects. The key word for me here is "obvious", since, again, the idea of a date having a boolean value is entirely foreign to the real world, and more or less a shortcut found only in computing. Don't create an edge case that you don't have to. Now granted, the stated behavior can definitely be seen to also violate PEP 20, but simply avoiding this entire issue renders that point moot. tl;dr: People complaining about unexpected results when writing bad Python code are barking up the wrong tree. Perhaps you should try a different language, or learn to code in a more Pythonic style? Both of these avenues would be much more profitable not only in the short term, but also the long term.

ufo 13y ago
I agree that maybe you should write in a different, more strongly typed, language but saying that people encountering this bug are not writing code in a Pythonic style is too much like victim blaming, IMO. Its a bit like those times when people say that if only you were a good programmer following "proper" C++ idioms you will never ever encounter memory leaks or segmentation faults. The default behaviour for objects in Python is being truthy and testing their existence with `if x` instead of `if x is not None` is definitely encouraged by the language (otherwise, why would it even let you use objects in a boolean context?). Having something that behaves like the default except once in a blue moon is just a trap and its unfair to blame it on the programmer.