5 ms·

I have been bitten by this many times. Null character in strings, as weird as it sounds like, happens in real life. It just happens. The pattern I have seen th

I have been bitten by this many times. Null character in strings, as weird as it sounds like, happens in real life. It just happens. The pattern I have seen the most is logs. More often than not logs may embed binary chunks, which may contain the infamous null character. However, the whole log itself is a string. For example, this happens on web server logs (binary data tends to be attack attempts). Because (obviously) this doesn't happen on PostgreSQL databases, I have experienced this while migrating from either Oracle or MongoDB to PostgreSQL. Nulls in both. One on logs, the other one as part of a user name (!!!). But still there. PostgreSQL must support this. It is long due and required, specially to help database migrations. The author is clearly on point. I actually raised this topic in 2016: https://www.postgresql.org/message-id/5aa1df8a-96f5-1d14-46fd-032e32846c71%408kdata.com https://www.postgresql.org/message-id/5aa1df8a-96f5-1d14-46f... But the conclusion was on the lines of "it would be too complicated, closing as WON'T DO".

corty 6y ago
A log containing binary chunks is not a string, because it will probably contain invalid characters, at least if your string encoding has the concept of invalid characters like UTF8 has. Since logs usually are not in any fixed encoding but in whatever the currently running process uses, even the string parts of your log are unlikely to always conform to UTF8. So blob would be the appropriate data type anyways.
- ahachete 6y ago
  Logs, unless specifically being told otherwise, are assumed to be in text format. Even if they may contain binary blobs (which are normally unexpected, and not really conformant), they are still supposed to be text format. Surely you can store anything on a bytea. However, that won't take you very far if you want to query the logs (partially or completely) from the database --which is indeed extremely useful. In any case, Postgres always strived for compliance. And here, it is not accepting a UTF-8 valid character, hence is being non-compliant.
deleted 6y ago
[deleted]

corty 6y ago

A log containing binary chunks is not a string, because it will probably contain invalid characters, at least if your string encoding has the concept of invalid characters like UTF8 has. Since logs usually are not in any fixed encoding but in whatever the currently running process uses, even the string parts of your log are unlikely to always conform to UTF8. So blob would be the appropriate data type anyways.

ahachete 6y ago
Logs, unless specifically being told otherwise, are assumed to be in text format. Even if they may contain binary blobs (which are normally unexpected, and not really conformant), they are still supposed to be text format. Surely you can store anything on a bytea. However, that won't take you very far if you want to query the logs (partially or completely) from the database --which is indeed extremely useful. In any case, Postgres always strived for compliance. And here, it is not accepting a UTF-8 valid character, hence is being non-compliant.

deleted 6y ago

[deleted]