4 ms·

I will always remember this incident as the time when the UK general public were exposed, en masse, to Spring error messages. The confusion caused by ordering

I will always remember this incident as the time when the UK general public were exposed, en masse, to Spring error messages. The confusion caused by ordering a member of the general public not to request a bean from a bean factory in a destroy method implementation still makes me laugh, even now.

ilyt 4y ago
I remember when we were telling devs to stop returning java 503's with stacktraces to the user. Devs fixed it by returning 200's with stack traces. And as page was ESI stitched together on Varnish, when they fucked up there wasn't just a stacktrace, but a bunch of different ones in various parts of the page.
- ywei3410 4y ago
  I really feel this is an infra problem rather than a dev problem though; the reverse-proxy should strip 5xx response bodies before the egress no?
  - ilyt 4y ago
    Eh, depends, for some apps 503 is legit response that should be returned to customer/app, in other cases it's app being badly designed. We did that few times when it made sense but in most it does not. There is no error code for "down for maintenance" so 503 was also used sometimes for that purpose (although we recommended devs to just fail healthchecks so loadbalancer displays its own error page in those cases) The other problem (let's just say devs were not... that great with architecture) was that they were getting exceptions in the logs without attached URL and other metadata so they kinda wanted to get that exception directly on the webpage. I'll paint you a picture of how shoddy some stuff was: they were using a templating language to generate JSONs (and had many bugs in it too) for years, instead of just natively encoding some JSON in Java, coz of party shoddy architecture, partly years-long war between frontend and backed dept. They migrated to Git in... 2018 I think ? No CI/CD of any sort till recently. End result (of them insisting that they will send the exception to frontend to be seen) was actually exceptions being signed and encrypted blob wrapped with a bunch of JS that gathers all the errors (incl. errors that JS on site might've done), adds all the metadata it sees from "browser perspective" and sends it to monitoring endpoint where that is shoved into ES cluster. Sort of retarded version of distributed tracing that is now in vogue... done somewhere in 2013. But it did catch a bunch of bugs that were "only" showing to users in browser
Nextgrid 4y ago
The fact that those messages were visible to external users is a major problem and sign of incompetence.
jw1224 4y ago
“Just want to see my balance and these guys @tsb think I'm robbing a bean factory with a bomb, jesus” Brilliant. Reminisce with a screenshot here: https://twitter.com/thejackthomson_/status/988564354512687104 https://twitter.com/thejackthomson_/status/98856435451268710...
grumpyprole 4y ago
I really hope my bank runs cobol rather than any of this spring bean junk.
chinabot 4y ago
If I cant raise a specific exception or its in a block of code I dont expect an error I generally make the exception memorable or dangerous sounding just so the user is more inclined to report it (hopefully not via a tweet tho)

ilyt 4y ago

I remember when we were telling devs to stop returning java 503's with stacktraces to the user. Devs fixed it by returning 200's with stack traces. And as page was ESI stitched together on Varnish, when they fucked up there wasn't just a stacktrace, but a bunch of different ones in various parts of the page.

ywei3410 4y ago
I really feel this is an infra problem rather than a dev problem though; the reverse-proxy should strip 5xx response bodies before the egress no?
- ilyt 4y ago
  Eh, depends, for some apps 503 is legit response that should be returned to customer/app, in other cases it's app being badly designed. We did that few times when it made sense but in most it does not. There is no error code for "down for maintenance" so 503 was also used sometimes for that purpose (although we recommended devs to just fail healthchecks so loadbalancer displays its own error page in those cases) The other problem (let's just say devs were not... that great with architecture) was that they were getting exceptions in the logs without attached URL and other metadata so they kinda wanted to get that exception directly on the webpage. I'll paint you a picture of how shoddy some stuff was: they were using a templating language to generate JSONs (and had many bugs in it too) for years, instead of just natively encoding some JSON in Java, coz of party shoddy architecture, partly years-long war between frontend and backed dept. They migrated to Git in... 2018 I think ? No CI/CD of any sort till recently. End result (of them insisting that they will send the exception to frontend to be seen) was actually exceptions being signed and encrypted blob wrapped with a bunch of JS that gathers all the errors (incl. errors that JS on site might've done), adds all the metadata it sees from "browser perspective" and sends it to monitoring endpoint where that is shoved into ES cluster. Sort of retarded version of distributed tracing that is now in vogue... done somewhere in 2013. But it did catch a bunch of bugs that were "only" showing to users in browser

Nextgrid 4y ago

The fact that those messages were visible to external users is a major problem and sign of incompetence.

jw1224 4y ago

“Just want to see my balance and these guys @tsb think I'm robbing a bean factory with a bomb, jesus” Brilliant. Reminisce with a screenshot here: https://twitter.com/thejackthomson_/status/988564354512687104 https://twitter.com/thejackthomson_/status/98856435451268710...

grumpyprole 4y ago

I really hope my bank runs cobol rather than any of this spring bean junk.

chinabot 4y ago

If I cant raise a specific exception or its in a block of code I dont expect an error I generally make the exception memorable or dangerous sounding just so the user is more inclined to report it (hopefully not via a tweet tho)