4 ms·

Why would someone "sanitize" OutOfMemoryException out of their logs? That is a silly point to make.

owebmaster 1y ago
An OutOfMemoryException log should not be the same as a search log Error: OutOfMemoryException And Search: OutOfMemoryException Should not be related in any way
- dusanz 1y ago
  Until someone searches for "Error: OutOfMemoryException"
  - PhilipRoman 1y ago
    If structured logging is too much, unique prefixes solve this issue. Basically you need some token that user provided data is unable to output to the log. If you rigorously escape all newlines, you can then use start-of-line and end-of-line as unforgeable tokens. The possibilities are endless and it all comes down to understanding whether the intersection of two grammars is empty.
  - happysadpanda2 1y ago
    I read the gp to mean that error.log (being parsed to look for OOM) would have no associations with userSearches.log, in which an end-user searched for OOM
teraflop 1y ago
The point is not to sanitize known strings like "OutOfMemoryException". The point is to sanitize or (preferably) escape any untrusted data that gets logged, so that it won't be confused for something else.
- swyx 1y ago
  i think GP's point is how would you even sanitize the string "OutOfMemoryException" which presumably comes from a trusted system i guess demanding "Structured logs for everything or bust" is the answer? (i'm not a big o11y guy so pardon me if this is obvious)
  - PhilipRoman 1y ago
    Low tech example: escape all newlines in user supplied strings, then add a known prefix to all user supplied data (let's say a double hashtag ##, but anything else works too). When you want to search logs for strings coming from your system, remove/ignore everything after the marker. It all comes down to understanding whether the intersection of two grammars is empty.
    - jethro_tell 1y ago
      The difficulty here is that in the example above, it's unlikely, given any amount of scale, that the two people were on the same team. They were doing different things with the same data and probably didn't know what the other was doing. Sure you could add a convention to your 'how to log' doc that specifies that all user input should be tagged with double '#' but who reads docs until things break? convention is a shitty way to make things work. There's 100 ways that you could make this work correctly. Only restarting on a much more specific string, i.e. including the app name in the log line etc . . . but that's all just reducing the likely hood that you get burned. I've also written a OOM-Killer.sh myself, I'm not above that, but it's one of those edge cases that's impossible to do correctly, which is why parsing and acting on log data generally considered and anti-pattern.
  - noisem4ker 1y ago
    "o11y" stands for "observability". Numeronyms are evil and we should stop using them.
    - j1elo 1y ago
      Thanks. My mind started running the random string generator given those restrictions, like a puzzle game. But had no idea of what it meant until you wrote it. Who invented that stupid idea and thought it would be a good one?
MortyWaves 1y ago
Absolutely incredible how dense HN can be and that no one has explained. Obviously that isn’t what they are saying, they are saying it’s profoundly stupid to have the server be controlled by a simple string search at all.

An OutOfMemoryException log should not be the same as a search log Error: OutOfMemoryException And Search: OutOfMemoryException Should not be related in any way

dusanz 1y ago
Until someone searches for "Error: OutOfMemoryException"
- PhilipRoman 1y ago
  If structured logging is too much, unique prefixes solve this issue. Basically you need some token that user provided data is unable to output to the log. If you rigorously escape all newlines, you can then use start-of-line and end-of-line as unforgeable tokens. The possibilities are endless and it all comes down to understanding whether the intersection of two grammars is empty.
- happysadpanda2 1y ago
  I read the gp to mean that error.log (being parsed to look for OOM) would have no associations with userSearches.log, in which an end-user searched for OOM

teraflop 1y ago

The point is not to sanitize known strings like "OutOfMemoryException". The point is to sanitize or (preferably) escape any untrusted data that gets logged, so that it won't be confused for something else.

swyx 1y ago
i think GP's point is how would you even sanitize the string "OutOfMemoryException" which presumably comes from a trusted system i guess demanding "Structured logs for everything or bust" is the answer? (i'm not a big o11y guy so pardon me if this is obvious)
- PhilipRoman 1y ago
  Low tech example: escape all newlines in user supplied strings, then add a known prefix to all user supplied data (let's say a double hashtag ##, but anything else works too). When you want to search logs for strings coming from your system, remove/ignore everything after the marker. It all comes down to understanding whether the intersection of two grammars is empty.
  - jethro_tell 1y ago
    The difficulty here is that in the example above, it's unlikely, given any amount of scale, that the two people were on the same team. They were doing different things with the same data and probably didn't know what the other was doing. Sure you could add a convention to your 'how to log' doc that specifies that all user input should be tagged with double '#' but who reads docs until things break? convention is a shitty way to make things work. There's 100 ways that you could make this work correctly. Only restarting on a much more specific string, i.e. including the app name in the log line etc . . . but that's all just reducing the likely hood that you get burned. I've also written a OOM-Killer.sh myself, I'm not above that, but it's one of those edge cases that's impossible to do correctly, which is why parsing and acting on log data generally considered and anti-pattern.
- noisem4ker 1y ago
  "o11y" stands for "observability". Numeronyms are evil and we should stop using them.
  - j1elo 1y ago
    Thanks. My mind started running the random string generator given those restrictions, like a puzzle game. But had no idea of what it meant until you wrote it. Who invented that stupid idea and thought it would be a good one?

MortyWaves 1y ago

Absolutely incredible how dense HN can be and that no one has explained. Obviously that isn’t what they are saying, they are saying it’s profoundly stupid to have the server be controlled by a simple string search at all.