4 ms·

Well, that's the thing - changing NAMEDATALEN is a seemingly small change, but it'll require much more work than just increasing the value. Increasing the value

Well, that's the thing - changing NAMEDATALEN is a seemingly small change, but it'll require much more work than just increasing the value. Increasing the value does not seem like a great option, because (a) how long before people start complaining about the new one and (b) it wastes even more memory. So I assume we'd switch to a variable-length strings, which however affects memory management, changes a lot of other stuff from fixed-length to variable-length, etc. So testing / benchmarking needed and all of that. Which is why people are not enthusiastic about changing it, when there are fairly simple workarounds (assuming keeping the names short is considered to be a workaround).

deleted 8y ago
[deleted]
marcosdumay 8y ago
> (a) how long before people start complaining about the new one Very likely many years, or even never. People don't use large names because they like it, they always prefer small ones. How much memory are we talking about?
- pgaddict 8y ago
  > Very likely many years, or even never. People don't use large names because they like it, they always prefer small ones. Well, we don't have exactly a barrage of complaints about the current limit either. > How much memory are we talking about? Good question. The thing is - it's not just about table names. NameData is used for any object name, so it affects pretty much any system catalog storing name. A simple grep on the repo says it affects about 40 catalogs (out of 80), including pg_attribute, pg_class, pg_operator, pg_proc, pg_type (which tend to be fairly large). So the amount of additional memory may be quite significant, because all of this is cached in various places.
  - anarazel 8y ago
    Yea, I think pg_attribute is likely to be the main issue here. For one, it obviously exists many times per table, and there are workloads with a lot of tables. But also importantly it's included in all tuple descriptors, which in turn get created during query execution in a fair number of places. It's currently ~140 bytes, with ~64bytes of that being the column name - just doubling that would increase the overhead noticeably, and we already have plenty of complaints about pg_attribute. I think it'd be fairly useless to just choose another fixed size, we really ought to make it variable length.
    - pgaddict 8y ago
      Is it ~140 bytes? pahole says it's 112 (without CATALOG_VARLEN). The impact of doubling NameData size would be quite a bit worse, though, thanks to doubling of chunk-size in allocset. At the moment it fits into a 128B chunk (so just ~16B wasted), but by doubling NameData to 128B the struct would suddenly be 176B, which requires 256B chunk (so 80B wasted). Yuck.
      - anarazel 8y ago
        > Is it ~140 bytes? pahole says it's 112 (without CATALOG_VARLEN). Well, but on-disk varlena data is included. pg_column_size() averages 144 bytes for pg_attribute on my system. > The impact of doubling NameData size would be quite a bit worse, though, thanks to doubling of chunk-size in allocset. At the moment it fits into a 128B chunk (so just ~16B wasted), but by doubling NameData to 128B the struct would suddenly be 176B, which requires 256B chunk (so 80B wasted). Yuck. I'm not sure that actually matters that much. Most attributes are allocated as part of TupleDescData, but that allocates all attributes together.
        pgaddict 8y ago
        > Well, but on-disk varlena data is included. pg_column_size() averages 144 bytes for pg_attribute on my system. Sure, but I thought we're talking about in-memory stuff as you've been talking about tuple descriptors. I don't think the on-disk size matters all that much, TBH, it's likely just a tiny fraction of data stored in the cluster. > I'm not sure that actually matters that much. Most attributes are allocated as part of TupleDescData, but that allocates all attributes together. Ah. Good point.

deleted 8y ago

[deleted]

marcosdumay 8y ago

> (a) how long before people start complaining about the new one Very likely many years, or even never. People don't use large names because they like it, they always prefer small ones. How much memory are we talking about?

pgaddict 8y ago
> Very likely many years, or even never. People don't use large names because they like it, they always prefer small ones. Well, we don't have exactly a barrage of complaints about the current limit either. > How much memory are we talking about? Good question. The thing is - it's not just about table names. NameData is used for any object name, so it affects pretty much any system catalog storing name. A simple grep on the repo says it affects about 40 catalogs (out of 80), including pg_attribute, pg_class, pg_operator, pg_proc, pg_type (which tend to be fairly large). So the amount of additional memory may be quite significant, because all of this is cached in various places.
- anarazel 8y ago
  Yea, I think pg_attribute is likely to be the main issue here. For one, it obviously exists many times per table, and there are workloads with a lot of tables. But also importantly it's included in all tuple descriptors, which in turn get created during query execution in a fair number of places. It's currently ~140 bytes, with ~64bytes of that being the column name - just doubling that would increase the overhead noticeably, and we already have plenty of complaints about pg_attribute. I think it'd be fairly useless to just choose another fixed size, we really ought to make it variable length.
  - pgaddict 8y ago
    Is it ~140 bytes? pahole says it's 112 (without CATALOG_VARLEN). The impact of doubling NameData size would be quite a bit worse, though, thanks to doubling of chunk-size in allocset. At the moment it fits into a 128B chunk (so just ~16B wasted), but by doubling NameData to 128B the struct would suddenly be 176B, which requires 256B chunk (so 80B wasted). Yuck.
    - anarazel 8y ago
      > Is it ~140 bytes? pahole says it's 112 (without CATALOG_VARLEN). Well, but on-disk varlena data is included. pg_column_size() averages 144 bytes for pg_attribute on my system. > The impact of doubling NameData size would be quite a bit worse, though, thanks to doubling of chunk-size in allocset. At the moment it fits into a 128B chunk (so just ~16B wasted), but by doubling NameData to 128B the struct would suddenly be 176B, which requires 256B chunk (so 80B wasted). Yuck. I'm not sure that actually matters that much. Most attributes are allocated as part of TupleDescData, but that allocates all attributes together.
      - pgaddict 8y ago
        > Well, but on-disk varlena data is included. pg_column_size() averages 144 bytes for pg_attribute on my system. Sure, but I thought we're talking about in-memory stuff as you've been talking about tuple descriptors. I don't think the on-disk size matters all that much, TBH, it's likely just a tiny fraction of data stored in the cluster. > I'm not sure that actually matters that much. Most attributes are allocated as part of TupleDescData, but that allocates all attributes together. Ah. Good point.