4 ms·

The only circumstances in which I can see writing tests against HTML directly is if you're doing a move or a migration. I did exactly this for a hosting compan

The only circumstances in which I can see writing tests against HTML directly is if you're doing a move or a migration. I did exactly this for a hosting company back when Windows Server 2003 was getting EOL'd. They setup on 2012, migrated the sites over, and then my script would pull from both versions and see how much variation there was. Even then it was mostly just a diff. Otherwise that's the equivalent of testing implementation details.

gwbas1c 6y ago
Write a test that: 1: Stick an entity in the database 2: Try to get the page for the entity without logging in. Verify the error page 3: Log in as someone who doesn't have the entity. Try to get the page for the entity. Verify the error page 4: Log in as someone who can get the entity. Verify some basics of the page contents Write a test that: 1: Posts the contents of the form without logging in. Verify the error. ... 4: Log in as someone who can post the form. Post the contents of the form. Verify the contents are in the database. Tests like above meet the higher-level post's definition of a "unit test" and are rather easy to write. You don't even have to be in the same language as application. They're simple enough that developers can write them as part of their workflow, and simple enough that developers can run them to ensure no regressions. Trying to do something like that with a thick UI application is much, much harder. Possible, but not easy.
- RandoHolmes 6y ago
  Here's my concern with that approach. It implies to me that the absence of an error is a success. In other words, if you don't find the HTML you're looking for that represents an error, then it's considered a success. The problem is that at some point it's possible for the page to be redesigned (or the entire site) such that the HTML you're looking for that represents an error is no longer there for a legitimate error. IOW, the tests are not future proof. But this is the general problem with looking at HTML, it's fragile. That doesn't mean I don't see the need, I understand the goal, I just don't agree with this approach. I think if it were me I'd do 1 of 2 things. 1. rely on HTTP codes (403 is a reasonable response for what you've described), or 2. rely on HTTP headers. And I would argue it's better to set them on success and consider it an error anytime the header is missing. Obviously redesigns don't happen that often so it's not a super large issue, probably not worth rewriting anything but I do think if I were about to implement that functionality I would try and use HTTP codes or HTTP headers.
  - gwbas1c 6y ago
    > Verify some basics of the page contents Implies that you're looking for the contents that are supposed to be returned. When there's an error, the contents you're looking for won't be there, therefore the test will fail. Furthermore, HTTP testing libraries typically require you to specify required HTTP status codes.
    - RandoHolmes 6y ago
      - "you can, for example, examine HTML in your tests" - "that sounds super fragile, there's only a few circumstances in which I would that" - "Here's the circumstances in which I'm doing that" - "Here are the problems with what you're doing, and here are alternatives that don't involve examining the HTML" - "I'm not examining the HTML!?!? Whatever gave you that impression!" --- Not worth my time.

gwbas1c 6y ago

Write a test that: 1: Stick an entity in the database 2: Try to get the page for the entity without logging in. Verify the error page 3: Log in as someone who doesn't have the entity. Try to get the page for the entity. Verify the error page 4: Log in as someone who can get the entity. Verify some basics of the page contents Write a test that: 1: Posts the contents of the form without logging in. Verify the error. ... 4: Log in as someone who can post the form. Post the contents of the form. Verify the contents are in the database. Tests like above meet the higher-level post's definition of a "unit test" and are rather easy to write. You don't even have to be in the same language as application. They're simple enough that developers can write them as part of their workflow, and simple enough that developers can run them to ensure no regressions. Trying to do something like that with a thick UI application is much, much harder. Possible, but not easy.

RandoHolmes 6y ago
Here's my concern with that approach. It implies to me that the absence of an error is a success. In other words, if you don't find the HTML you're looking for that represents an error, then it's considered a success. The problem is that at some point it's possible for the page to be redesigned (or the entire site) such that the HTML you're looking for that represents an error is no longer there for a legitimate error. IOW, the tests are not future proof. But this is the general problem with looking at HTML, it's fragile. That doesn't mean I don't see the need, I understand the goal, I just don't agree with this approach. I think if it were me I'd do 1 of 2 things. 1. rely on HTTP codes (403 is a reasonable response for what you've described), or 2. rely on HTTP headers. And I would argue it's better to set them on success and consider it an error anytime the header is missing. Obviously redesigns don't happen that often so it's not a super large issue, probably not worth rewriting anything but I do think if I were about to implement that functionality I would try and use HTTP codes or HTTP headers.
- gwbas1c 6y ago
  > Verify some basics of the page contents Implies that you're looking for the contents that are supposed to be returned. When there's an error, the contents you're looking for won't be there, therefore the test will fail. Furthermore, HTTP testing libraries typically require you to specify required HTTP status codes.
  - RandoHolmes 6y ago
    - "you can, for example, examine HTML in your tests" - "that sounds super fragile, there's only a few circumstances in which I would that" - "Here's the circumstances in which I'm doing that" - "Here are the problems with what you're doing, and here are alternatives that don't involve examining the HTML" - "I'm not examining the HTML!?!? Whatever gave you that impression!" --- Not worth my time.