3 ms·

> What was happening instead is that the Prometheus host agent's HTTP server code will give you a HTTP 200 answer (with a generic front page) for any URL except

> What was happening instead is that the Prometheus host agent's HTTP server code will give you a HTTP 200 answer (with a generic front page) for any URL except the special URL for its metrics endpoint. > Neither party is exactly wrong here, but the result is not ideal. No, Prometheus has a bug if this is the case, and cks says it himself further on... > all it would need to do is only give a HTTP 200 response for '/' and then a 404 (with the same HTML) for everything else that it answers with the generic front page

lifeonlars 3y ago
..or the vuln detection could be less stupid and try to decide whether the content it has retrieved is a bash history or not?
- nubinetwork 3y ago
  I would agree in principle, but that would require an extensive parser for files it thinks it finds... it's easier to deal with this on the server side than the client side.
  - lifeonlars 3y ago
    No it's not. Dealing with it on the server side means that all tools and servers in the world have to go along with this decision. Posting on a blog saying that people 'SHOULD' do something and actually making it so that this is a reasonable expectation are two very different things. A vulnerability scanner having to implement hairy heuristics to decide what's a vuln and what's a common false positive is literally its whole job.
    - JoBrad 3y ago
      There is a standard for HTTP status codes that has dictated what web servers “should” do for quite a while, now. Most web servers respond properly with a 404 “out of the box”, when asked to serve up content that doesn’t exist.
      - lifeonlars 3y ago
        Almost any web server can be configured to provide a generic response to a specific request, for example by ignoring some or all of the url path, and in practice I would bet that a majority of actual instances do this for at least some sets of requests. (To confirm my theory in 20 seconds I checked if https://news.ycombinator.com/user?id=nonexistent_user_1620 https://news.ycombinator.com/user?id=nonexistent_user_1620 returns a 404 or a 200 - it's the latter.) It's silly to pretend that the use of a 404 in this type of circumstance is either clearcut in the standards or ubiquitous in practice.
        shkkmo 3y ago
        > It's silly to pretend that the use of a 404 in this type of circumstance is either clearcut in the standards or ubiquitous in practice. The standards seem pretty clear to me. I would point out that technically, the path portion of the HN URI does indeed point to a valid endpoint, it is the query portion of the URI (usually not used by the server to do any routing) that points to a non-existent resource. Still, HN is wrong here and should be returning a 404 status.
      - Dah00n 3y ago
        But should != Must.
      - semi 3y ago
        I think that's still somewhat beside the point though. in the case of Prometheus yes it should return a 404. but what if it was nginx routing all paths to some app? or even just some actual file being served on that path? in either case the vuln scanner says you have an exposed home directory, and that's a false positive.
    - jjav 3y ago
      > Posting on a blog saying that people 'SHOULD' do something To be fair, it's not about some rando on a blog post. The HTTP response codes and their semantics are in the RFCs. And while it's true the RFCs say SHOULD and not MUST, there are also now ~30 decades of experience and expectation that a non-existent resource is more likely a 404 not 200. Sure a server can always do what it likes but can't expect it to play well with the outside world if it goes against both convention and documentation.
matsemann 3y ago
What might happen is that it's a single page app. So whatever URL you access the same frontend is served. And then the SPA shows the correct content based on the path in the URL. Then it's hard for the backend to know the correct status code to send. Of course you might still consider that a bug or wrong, but it's reality for a lot of web pages.
- nubinetwork 3y ago
  I would argue that is still wrong... if the page doesn't exist, you either 404, or 301 to a page that does exist.
  - Timon3 3y ago
    That would require you to duplicate all routing between backend and frontend. As long as it's not an API I don't see the need.
    - shkkmo 3y ago
      > As long as it's not an API I don't see the need. And I don't see the need to use a SPA almost anywhere. If you are gonna use a SPA for stuff that really doesn't require it, I think it behooves you to still atleast try to conform the the HTTP spec
      - Timon3 3y ago
        > And I don't see the need to use a SPA almost anywhere. Did I advocate for that in any way? > If you are gonna use a SPA for stuff that really doesn't require it, I think it behooves you to still atleast try to conform the the HTTP spec Why? Give me a good reason why it's worth the effort for small tools I write.
        shkkmo 3y ago
        > Why? Give me a good reason why it's worth the effort for small tools I write Why are you using SPAs with internal routing for small tools? An SPA only needs to use urls to represent states that should be externally reachable. If you want those states to be externally reachable, then you probably want those states to be indexable by search engines. You also want users to know if a url is mistyped/miscopied and a 404 response page does a good job of that. In the end, given the spec, it is on you to carefully consider amd have a good reason to go against the recommendations.

lifeonlars 3y ago

..or the vuln detection could be less stupid and try to decide whether the content it has retrieved is a bash history or not?

nubinetwork 3y ago
I would agree in principle, but that would require an extensive parser for files it thinks it finds... it's easier to deal with this on the server side than the client side.
- lifeonlars 3y ago
  No it's not. Dealing with it on the server side means that all tools and servers in the world have to go along with this decision. Posting on a blog saying that people 'SHOULD' do something and actually making it so that this is a reasonable expectation are two very different things. A vulnerability scanner having to implement hairy heuristics to decide what's a vuln and what's a common false positive is literally its whole job.
  - JoBrad 3y ago
    There is a standard for HTTP status codes that has dictated what web servers “should” do for quite a while, now. Most web servers respond properly with a 404 “out of the box”, when asked to serve up content that doesn’t exist.
    - lifeonlars 3y ago
      Almost any web server can be configured to provide a generic response to a specific request, for example by ignoring some or all of the url path, and in practice I would bet that a majority of actual instances do this for at least some sets of requests. (To confirm my theory in 20 seconds I checked if https://news.ycombinator.com/user?id=nonexistent_user_1620 https://news.ycombinator.com/user?id=nonexistent_user_1620 returns a 404 or a 200 - it's the latter.) It's silly to pretend that the use of a 404 in this type of circumstance is either clearcut in the standards or ubiquitous in practice.
      - shkkmo 3y ago
        > It's silly to pretend that the use of a 404 in this type of circumstance is either clearcut in the standards or ubiquitous in practice. The standards seem pretty clear to me. I would point out that technically, the path portion of the HN URI does indeed point to a valid endpoint, it is the query portion of the URI (usually not used by the server to do any routing) that points to a non-existent resource. Still, HN is wrong here and should be returning a 404 status.
    - Dah00n 3y ago
      But should != Must.
    - semi 3y ago
      I think that's still somewhat beside the point though. in the case of Prometheus yes it should return a 404. but what if it was nginx routing all paths to some app? or even just some actual file being served on that path? in either case the vuln scanner says you have an exposed home directory, and that's a false positive.
  - jjav 3y ago
    > Posting on a blog saying that people 'SHOULD' do something To be fair, it's not about some rando on a blog post. The HTTP response codes and their semantics are in the RFCs. And while it's true the RFCs say SHOULD and not MUST, there are also now ~30 decades of experience and expectation that a non-existent resource is more likely a 404 not 200. Sure a server can always do what it likes but can't expect it to play well with the outside world if it goes against both convention and documentation.

matsemann 3y ago

What might happen is that it's a single page app. So whatever URL you access the same frontend is served. And then the SPA shows the correct content based on the path in the URL. Then it's hard for the backend to know the correct status code to send. Of course you might still consider that a bug or wrong, but it's reality for a lot of web pages.

nubinetwork 3y ago
I would argue that is still wrong... if the page doesn't exist, you either 404, or 301 to a page that does exist.
- Timon3 3y ago
  That would require you to duplicate all routing between backend and frontend. As long as it's not an API I don't see the need.
  - shkkmo 3y ago
    > As long as it's not an API I don't see the need. And I don't see the need to use a SPA almost anywhere. If you are gonna use a SPA for stuff that really doesn't require it, I think it behooves you to still atleast try to conform the the HTTP spec
    - Timon3 3y ago
      > And I don't see the need to use a SPA almost anywhere. Did I advocate for that in any way? > If you are gonna use a SPA for stuff that really doesn't require it, I think it behooves you to still atleast try to conform the the HTTP spec Why? Give me a good reason why it's worth the effort for small tools I write.
      - shkkmo 3y ago
        > Why? Give me a good reason why it's worth the effort for small tools I write Why are you using SPAs with internal routing for small tools? An SPA only needs to use urls to represent states that should be externally reachable. If you want those states to be externally reachable, then you probably want those states to be indexable by search engines. You also want users to know if a url is mistyped/miscopied and a 404 response page does a good job of that. In the end, given the spec, it is on you to carefully consider amd have a good reason to go against the recommendations.