6 ms·

htmlspecialchars does not throw E_WARNING on multibyte problems

30 points by rustc 14y ago

geofft 14y ago
In fairness, display_errors is a terrible idea -- do you actually want errors thrown by your program to be interspersed with the HTML output of your program? This sort of stuff should be going to the error log. And it is, so "throws error" and "error reporting" in the title are both inaccurate here. It does throw an error; it merely refuses to send that error to the HTML output of the page when display_errors is on. If normal, sane error reporting to the error log is on, the error does show up there.
- krapp 14y ago
  I was actually wondering not long ago if there might be some kind of cross-site scripting vulnerability implicit in rendering exceptions as html. Sure, you shouldn't have error reporting on to begin with but who even checks the source code of the exception pages themselves? You're paying close attention to the error itself after all... Turns out yes maybe... https://nealpoole.com/blog/2011/08/cross-site-scripting-via-error-reporting-notices-in-php/ https://nealpoole.com/blog/2011/08/cross-site-scripting-via-...
- nwh 14y ago
  It only really makes sense when PHP is used in a shell. If nothing else, its strange that a lot of PHP implementations have them turned on by default.
  - Firehed 14y ago
    Not really. People that turn off display_errors and set up proper logging typically know what they're doing; it's a pretty sensible default for "my first website" level of projects. Remember, those folks are just uploading PHP files to their shared host - they don't have a local development environment, let alone a custom-configured production setup. Is it unfortunate that a lot of production sites run with stupid settings because they aren't run by full-time software engineers or sysadmins? Sure. But the accessibility of making changes is great, and most people would consider it a good thing that people are taking an interest in learning how to do this stuff themselves rather than outsourcing all of their technology problems. After all, isn't that how most of us got started?
- munger 14y ago
  display_errors can be convenient on non-production sites. I think a lot of beginners are trying to hack stuff on shared hosting, and transfer their php files in via FTP and don't understand how to access logs and do appreciate this feature. It's just too bad they are hacking on their live site, and/or not ever turning it off after they don't need it. Don't get me wrong, FTP/live editing a site on production / shared hosting are all crappy ways of doing things for serious development but I can see how it might be useful for beginners. We've all been there, yeah? even if it was back when php 3 and 4 were common.
- Firehed 14y ago
  Most beginners are not going to be looking at their error log, or even know where to look for it. In a development environment, I think display_errors is a very reasonable behavior, and is in line with PHP's goals: be easy to get up and running, even (especially?) for novice programmers. Even people that have been doing this for a decade may forget to tail their logs, so having execution-time warnings makes a lot of sense, and is really the only practical approach for an interpreted language. I've made vague suggestions (though not written up a full spec/RFC... one of these days) about improving the overall error handling system in PHP since I think there are a limited set of common behaviors that involve a lot of repeated code to get, which basically amount to "use strict". That way people doing serious projects that need a flexible language can get decent and consistent error handling, and newbies hacking together their first dynamic website can continue to work without hitting a brick wall of a learning curve. When I was first learning to code, something like display_errors would have been great. As it was I just had a few compiler warnings, but (especially as a newbie) who cares as long as it compiles and runs? Getting random warning text printed out on your web page is a real incentive to fix things, but without being so harsh that you just give up. Of course that doesn't scale to larger projects, but it doesn't need to. In PHP's case, that's what set_error_handler() is for.
- maratd 14y ago
  > In fairness, display_errors is a terrible idea You're absolutely right! In both development and production, errors should go to syslog. I have all my machines on rsyslog and they transfer all logs to a central server where I can see them very easily and clearly through an internal web app. They even have services nowadays that make this stupid simple.
ecaron 14y ago
This "bug" has been resolved since 01-Mar-2012 when 5.4 was released, and there's a very straight-forward explanation of it if people would just RTFM. This is the worst part of the PHP community - all the in-fighting when the core developers don't bend over backwards to accommodate short-sighted programmers who know their conflicting demand is the one that should be heard.
smsm42 14y ago
The problem is that people have conflicting demands. They want all errors - even most unobvious ones in most deep functions, in all outputs - be immediately shown to them with all details, without having to do even minimal amount of work, and then they want the whole system also be secure. It can not work this way. If you want errors on the output, it can not be secure. If you want secure defaults, you can not have server spewing errors to the output. So we have a choice here - do we prefer, convenience or security? Given that inconvenience is much less dangerous than insecurity, the choice for PHP developers was security. In ideal world, where nobody runs their production servers with display_errors=on and everybody understand the consequences of their actions, this hack would not be needed. We live however in a real world, where thousands of production sites are run with dislpay_errors, so we need to account for this reality. It means somebody has to check their error log from time to time. If it's too hard to do - install custom error handler and override the default behavior.
birken 14y ago
If you find an odd situation in PHP due to backwards compatibility or whatever, I highly recommend you just make a wrapper function to do what you want. Another example of a common mistake with default functionality: Did you know json_decode('null') and json_decode('{') both return the same thing, NULL. One is correct, one isn't. The only way to actually tell the difference? Also call json_last_error() to make sure there wasn't an error parsing the json. Pretty inconvenient to do every time (and most people wont). Solution: Wrap it. At Thumbtack we have lots of wrapper functions which take care of situations where a built-in function doesn't have an interface we like. Most of them are purely for convenience though, not situations like this where the default output does something potentially misleading.
- masklinn 14y ago
  > Solution: Wrap it. Conversely, cut your losses and use something saner with better internal and API coherence.
  - maratd 14y ago
    Please take your hipster elitism somewhere else. Every language has warts and learning how to avoid them is always a good idea.
    - msbarnett 14y ago
      False equivalence. The fact that every language has warts does not mean that all languages are equally warty. There rapidly comes a point when you're spending so much time working around PHP's prolific warts that it would be more efficient for you to move to a less warty language.
    - kstrauser 14y ago
      That's not elitism; that's reality. Notice that PHP is pretty much the sole language garnering this reaction? No one actively makes fun of C#, Python, Clojure, Haskell, Perl, Java, etc., even when they dislike them. For example, I don't care for Java at all, but I admit that it's a really good implementation of a language I just don't happen to like very much. Same for C#. My reasons for not using it have nothing to do with the language itself (which is perfectly decent by all accounts). Programmers joke about Perl looking like line noise, but even that's a criticism of how some people use the language and not anything bad about Perl directly. But no one other than PHP programmers seems to have anything good to say about PHP. It's nearly universally disrespected as an insecure, inconsistent, buggy mess. That's very unusual - perhaps unique - among major programming languages. Maybe there's a reason that non-Java programmers have a grudging respect for Java, non-Python programmers can find things they like about Python, but non-PHP programmers almost never have anything good to say about PHP. When a lot of subject matter experts vocally dislike something, maybe that thing is misunderstood. When nearly all of them dislike it, perhaps there's a reason for it.
gedrap 14y ago
I start finding those arguments about PHP close to ones Adroid vs iPhone. Yes, PHP is inconsistent and etc. But on the other hand, it just gets the shit done. And for historical reasons, it has a huge community. When you add those two factors together and stop being a hipster... It's not that bad, is it?
- mikeash 14y ago
  It's terrible. PHP doesn't "get the shit done", although PHP programmers can. There are far better languages for getting things done. Huge community is not terribly attractive, as plenty of other, better languages have sufficiently large communities. I'd survive if I had to program in PHP, but it would definitely be a last resort. It could be worse (Malbolge?) but I don't understand why people feel a need to defend it or say it's not bad.

30 points by rustc 14y ago

In fairness, display_errors is a terrible idea -- do you actually want errors thrown by your program to be interspersed with the HTML output of your program? This sort of stuff should be going to the error log. And it is, so "throws error" and "error reporting" in the title are both inaccurate here. It does throw an error; it merely refuses to send that error to the HTML output of the page when display_errors is on. If normal, sane error reporting to the error log is on, the error does show up there.

krapp 14y ago
I was actually wondering not long ago if there might be some kind of cross-site scripting vulnerability implicit in rendering exceptions as html. Sure, you shouldn't have error reporting on to begin with but who even checks the source code of the exception pages themselves? You're paying close attention to the error itself after all... Turns out yes maybe... https://nealpoole.com/blog/2011/08/cross-site-scripting-via-error-reporting-notices-in-php/ https://nealpoole.com/blog/2011/08/cross-site-scripting-via-...
nwh 14y ago
It only really makes sense when PHP is used in a shell. If nothing else, its strange that a lot of PHP implementations have them turned on by default.
- Firehed 14y ago
  Not really. People that turn off display_errors and set up proper logging typically know what they're doing; it's a pretty sensible default for "my first website" level of projects. Remember, those folks are just uploading PHP files to their shared host - they don't have a local development environment, let alone a custom-configured production setup. Is it unfortunate that a lot of production sites run with stupid settings because they aren't run by full-time software engineers or sysadmins? Sure. But the accessibility of making changes is great, and most people would consider it a good thing that people are taking an interest in learning how to do this stuff themselves rather than outsourcing all of their technology problems. After all, isn't that how most of us got started?
munger 14y ago
display_errors can be convenient on non-production sites. I think a lot of beginners are trying to hack stuff on shared hosting, and transfer their php files in via FTP and don't understand how to access logs and do appreciate this feature. It's just too bad they are hacking on their live site, and/or not ever turning it off after they don't need it. Don't get me wrong, FTP/live editing a site on production / shared hosting are all crappy ways of doing things for serious development but I can see how it might be useful for beginners. We've all been there, yeah? even if it was back when php 3 and 4 were common.
Firehed 14y ago
Most beginners are not going to be looking at their error log, or even know where to look for it. In a development environment, I think display_errors is a very reasonable behavior, and is in line with PHP's goals: be easy to get up and running, even (especially?) for novice programmers. Even people that have been doing this for a decade may forget to tail their logs, so having execution-time warnings makes a lot of sense, and is really the only practical approach for an interpreted language. I've made vague suggestions (though not written up a full spec/RFC... one of these days) about improving the overall error handling system in PHP since I think there are a limited set of common behaviors that involve a lot of repeated code to get, which basically amount to "use strict". That way people doing serious projects that need a flexible language can get decent and consistent error handling, and newbies hacking together their first dynamic website can continue to work without hitting a brick wall of a learning curve. When I was first learning to code, something like display_errors would have been great. As it was I just had a few compiler warnings, but (especially as a newbie) who cares as long as it compiles and runs? Getting random warning text printed out on your web page is a real incentive to fix things, but without being so harsh that you just give up. Of course that doesn't scale to larger projects, but it doesn't need to. In PHP's case, that's what set_error_handler() is for.
maratd 14y ago
> In fairness, display_errors is a terrible idea You're absolutely right! In both development and production, errors should go to syslog. I have all my machines on rsyslog and they transfer all logs to a central server where I can see them very easily and clearly through an internal web app. They even have services nowadays that make this stupid simple.

ecaron 14y ago

This "bug" has been resolved since 01-Mar-2012 when 5.4 was released, and there's a very straight-forward explanation of it if people would just RTFM. This is the worst part of the PHP community - all the in-fighting when the core developers don't bend over backwards to accommodate short-sighted programmers who know their conflicting demand is the one that should be heard.

smsm42 14y ago

The problem is that people have conflicting demands. They want all errors - even most unobvious ones in most deep functions, in all outputs - be immediately shown to them with all details, without having to do even minimal amount of work, and then they want the whole system also be secure. It can not work this way. If you want errors on the output, it can not be secure. If you want secure defaults, you can not have server spewing errors to the output. So we have a choice here - do we prefer, convenience or security? Given that inconvenience is much less dangerous than insecurity, the choice for PHP developers was security. In ideal world, where nobody runs their production servers with display_errors=on and everybody understand the consequences of their actions, this hack would not be needed. We live however in a real world, where thousands of production sites are run with dislpay_errors, so we need to account for this reality. It means somebody has to check their error log from time to time. If it's too hard to do - install custom error handler and override the default behavior.

birken 14y ago

If you find an odd situation in PHP due to backwards compatibility or whatever, I highly recommend you just make a wrapper function to do what you want. Another example of a common mistake with default functionality: Did you know json_decode('null') and json_decode('{') both return the same thing, NULL. One is correct, one isn't. The only way to actually tell the difference? Also call json_last_error() to make sure there wasn't an error parsing the json. Pretty inconvenient to do every time (and most people wont). Solution: Wrap it. At Thumbtack we have lots of wrapper functions which take care of situations where a built-in function doesn't have an interface we like. Most of them are purely for convenience though, not situations like this where the default output does something potentially misleading.

masklinn 14y ago
> Solution: Wrap it. Conversely, cut your losses and use something saner with better internal and API coherence.
- maratd 14y ago
  Please take your hipster elitism somewhere else. Every language has warts and learning how to avoid them is always a good idea.
  - msbarnett 14y ago
    False equivalence. The fact that every language has warts does not mean that all languages are equally warty. There rapidly comes a point when you're spending so much time working around PHP's prolific warts that it would be more efficient for you to move to a less warty language.
  - kstrauser 14y ago
    That's not elitism; that's reality. Notice that PHP is pretty much the sole language garnering this reaction? No one actively makes fun of C#, Python, Clojure, Haskell, Perl, Java, etc., even when they dislike them. For example, I don't care for Java at all, but I admit that it's a really good implementation of a language I just don't happen to like very much. Same for C#. My reasons for not using it have nothing to do with the language itself (which is perfectly decent by all accounts). Programmers joke about Perl looking like line noise, but even that's a criticism of how some people use the language and not anything bad about Perl directly. But no one other than PHP programmers seems to have anything good to say about PHP. It's nearly universally disrespected as an insecure, inconsistent, buggy mess. That's very unusual - perhaps unique - among major programming languages. Maybe there's a reason that non-Java programmers have a grudging respect for Java, non-Python programmers can find things they like about Python, but non-PHP programmers almost never have anything good to say about PHP. When a lot of subject matter experts vocally dislike something, maybe that thing is misunderstood. When nearly all of them dislike it, perhaps there's a reason for it.

gedrap 14y ago

I start finding those arguments about PHP close to ones Adroid vs iPhone. Yes, PHP is inconsistent and etc. But on the other hand, it just gets the shit done. And for historical reasons, it has a huge community. When you add those two factors together and stop being a hipster... It's not that bad, is it?

mikeash 14y ago
It's terrible. PHP doesn't "get the shit done", although PHP programmers can. There are far better languages for getting things done. Huge community is not terribly attractive, as plenty of other, better languages have sufficiently large communities. I'd survive if I had to program in PHP, but it would definitely be a last resort. It could be worse (Malbolge?) but I don't understand why people feel a need to defend it or say it's not bad.