5 ms·

Introducing Abseil, a new common libraries project

85 points by astdb 9y ago

KGIII 9y ago
Previously: https://news.ycombinator.com/item?id=15341493 https://news.ycombinator.com/item?id=15341493 I think there may have been another thread about it, but I'm not seeing it.
- dang 9y ago
  Probably https://news.ycombinator.com/item?id=15339564 https://news.ycombinator.com/item?id=15339564.
  - KGIII 9y ago
    Yup. Perfect! Thanks!
jzelinskie 9y ago
At first I thought this would have a lot of overlap with Facebook's folly[0], but after glancing at the internal packages, Abseil looks lower level: it provides alternatives to STL types with shims for older compilers. It's interesting to see more and more Google internals being open sourced. It's like Blaze being released as Bazel[1] opened the flood gates for many other projects to be shared. Can any Googlers speak to how these projects are synchronized with the internal monorepo? [0]: https://github.com/facebook/folly https://github.com/facebook/folly [1]: https://bazel.build https://bazel.build
- fourth_party 9y ago
  Within google, google’s own open source software is treated as “third party”. https://opensource.google.com/docs/thirdparty/ https://opensource.google.com/docs/thirdparty/
  - jzelinskie 9y ago
    Thanks for the response. Are things moving into third party or are new projects being created in third party and maintained alongside the originals? I and probably many others are more likely to be willing to depend on a library if I know Google internals directly depend on the maintenance of said library.
    - yegle 9y ago
      Likely no one would answer this question considering the info is confidential. I think the best way to understand how Google uses a monorepo is this doc [0]. Disclaimer: yes I work for Google but my work is not related to Abseil team. [0]: https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
      - joshuamorton 9y ago
        Parroting this comment (that is probably confidential), the abseil website does include this statement: >The libraries we are releasing come with a pedigree: many years of experience using these APIs in Google’s production environments. We’ve seen what works and what doesn’t, what designs lead to bugs, performance problems, and misuse. What you see here is what we found to be a good balance between simplicity and meeting the needs of production use and an ever-evolving codebase. [1] Also a googler who has nothing to do with abseil. [1]: https://abseil.io/ https://abseil.io/
  - ehnto 9y ago
    I have always wondered how the monorepo design works in regards to more sensitive pieces of code. For example some of the key algorithms behind search.
    - joshuamorton 9y ago
      >Since Google's source code is one of the company's most important assets, security features are a key consideration in Piper's design. Piper supports file-level access control lists. Most of the repository is visible to all Piper users;d however, important configuration files or files including business-critical algorithms can be more tightly controlled. In addition, read and write access to files in Piper is logged. If sensitive data is accidentally committed to Piper, the file in question can be purged. The read logs allow administrators to determine if anyone accessed the problematic file before it was removed. From [1]. Essentially, there are files and changes that I cannot view the source of. [1]: https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
bsimpson 9y ago
Whoa - there's a Python version too: https://github.com/abseil/abseil-py https://github.com/abseil/abseil-py
- cbhl 9y ago
  Keep in mind that this is a subset of the C++ library. The static analysis tools for C++ are much more mature than for Python, and good static analysis tools are necessary if you want to perform a refactor across the entire monorepo[1] and encourage thousands of other engineers to start coding in your new paradigm[2]. [1] https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext https://cacm.acm.org/magazines/2016/7/204032-why-google-stor... [2] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43322.pdf https://static.googleusercontent.com/media/research.google.c... (Disclaimer: I work on YouTube, which has one of the largest legacy Python code-bases at Google. This is my opinion, not that of my employer.)
randomfool 9y ago
Interesting to compare some of the Chromium code with this, for example: https://cs.chromium.org/chromium/src/net/base/int128.h?dr https://cs.chromium.org/chromium/src/net/base/int128.h?dr https://github.com/abseil/abseil-cpp/blob/master/absl/numeric/int128.h https://github.com/abseil/abseil-cpp/blob/master/absl/numeri...
- esprehn 9y ago
  Chromium's base/ and the various copies of it (ex. inside net/) are derived from a very old fork of the code inside Google that became abseil. In some places it's like looking backwards in time, in others the two have diverged naturally as folks have worked on them separately.
fiokoden 9y ago
Glad to see this is zero configuration. If your software is NOT zero configuration, ask yourself why not.
- majewsky 9y ago
  This is a library, not an application. Can you give an example of any program that does not require configuration? Even yes(1) can be configured to output a different string than "yes".
  - pritambaral 9y ago
    Unrelated to the OP, but I think zero-configuration doesn't mean "you cannot configure it", rather "you shouldn't have to configure it". In that sense, yes(1) is "zero configuration".
    - majewsky 9y ago
      I see. In that sense, it definitely had its merits, although there is the risk of your users never noticing that configuration options are available to accommodate their specific needs better than the defaults.
rurban 9y ago
Very very immature. E.g. the string library only knows about ascii, and does matching via tolower. Not usable for strings, only for memory. It doesn't known about unicode, UTF-8, UTF-16 nor UTF-32. Matching must be done via foldcase (fc) not tolower, There's a string splitter which only does ascii, so if it's UTF8 you get invalid strings, if it's wchar_t it's not usable. The good things are: noexcept, no bloat, FixedArray as better alloca(), InlinedVector similar. folly is much more advanced.
sigi45 9y ago
In germany it sounds like 'einen abseilen' aka 'to go to the toilet and dump one'
- majewsky 9y ago
  Technically, the verb "abseilen" just means "to rope down", but the collision with this idiom is rather unfortunate.
- Geeflow 9y ago
  I am not sure how common that is, though. I am a native speaker and have not heard that expression before.
  - sigi45 9y ago
    I don't think it is common to say it often but at least in Bavaria, people know this.

85 points by astdb 9y ago

Previously: https://news.ycombinator.com/item?id=15341493 https://news.ycombinator.com/item?id=15341493 I think there may have been another thread about it, but I'm not seeing it.

dang 9y ago
Probably https://news.ycombinator.com/item?id=15339564 https://news.ycombinator.com/item?id=15339564.
- KGIII 9y ago
  Yup. Perfect! Thanks!

jzelinskie 9y ago

At first I thought this would have a lot of overlap with Facebook's folly[0], but after glancing at the internal packages, Abseil looks lower level: it provides alternatives to STL types with shims for older compilers. It's interesting to see more and more Google internals being open sourced. It's like Blaze being released as Bazel[1] opened the flood gates for many other projects to be shared. Can any Googlers speak to how these projects are synchronized with the internal monorepo? [0]: https://github.com/facebook/folly https://github.com/facebook/folly [1]: https://bazel.build https://bazel.build

fourth_party 9y ago
Within google, google’s own open source software is treated as “third party”. https://opensource.google.com/docs/thirdparty/ https://opensource.google.com/docs/thirdparty/
- jzelinskie 9y ago
  Thanks for the response. Are things moving into third party or are new projects being created in third party and maintained alongside the originals? I and probably many others are more likely to be willing to depend on a library if I know Google internals directly depend on the maintenance of said library.
  - yegle 9y ago
    Likely no one would answer this question considering the info is confidential. I think the best way to understand how Google uses a monorepo is this doc [0]. Disclaimer: yes I work for Google but my work is not related to Abseil team. [0]: https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...
    - joshuamorton 9y ago
      Parroting this comment (that is probably confidential), the abseil website does include this statement: >The libraries we are releasing come with a pedigree: many years of experience using these APIs in Google’s production environments. We’ve seen what works and what doesn’t, what designs lead to bugs, performance problems, and misuse. What you see here is what we found to be a good balance between simplicity and meeting the needs of production use and an ever-evolving codebase. [1] Also a googler who has nothing to do with abseil. [1]: https://abseil.io/ https://abseil.io/
- ehnto 9y ago
  I have always wondered how the monorepo design works in regards to more sensitive pieces of code. For example some of the key algorithms behind search.
  - joshuamorton 9y ago
    >Since Google's source code is one of the company's most important assets, security features are a key consideration in Piper's design. Piper supports file-level access control lists. Most of the repository is visible to all Piper users;d however, important configuration files or files including business-critical algorithms can be more tightly controlled. In addition, read and write access to files in Piper is logged. If sensitive data is accidentally committed to Piper, the file in question can be purged. The read logs allow administrators to determine if anyone accessed the problematic file before it was removed. From [1]. Essentially, there are files and changes that I cannot view the source of. [1]: https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext https://cacm.acm.org/magazines/2016/7/204032-why-google-stor...

bsimpson 9y ago

Whoa - there's a Python version too: https://github.com/abseil/abseil-py https://github.com/abseil/abseil-py

cbhl 9y ago
Keep in mind that this is a subset of the C++ library. The static analysis tools for C++ are much more mature than for Python, and good static analysis tools are necessary if you want to perform a refactor across the entire monorepo[1] and encourage thousands of other engineers to start coding in your new paradigm[2]. [1] https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/fulltext https://cacm.acm.org/magazines/2016/7/204032-why-google-stor... [2] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43322.pdf https://static.googleusercontent.com/media/research.google.c... (Disclaimer: I work on YouTube, which has one of the largest legacy Python code-bases at Google. This is my opinion, not that of my employer.)

randomfool 9y ago

Interesting to compare some of the Chromium code with this, for example: https://cs.chromium.org/chromium/src/net/base/int128.h?dr https://cs.chromium.org/chromium/src/net/base/int128.h?dr https://github.com/abseil/abseil-cpp/blob/master/absl/numeric/int128.h https://github.com/abseil/abseil-cpp/blob/master/absl/numeri...

esprehn 9y ago
Chromium's base/ and the various copies of it (ex. inside net/) are derived from a very old fork of the code inside Google that became abseil. In some places it's like looking backwards in time, in others the two have diverged naturally as folks have worked on them separately.

fiokoden 9y ago

Glad to see this is zero configuration. If your software is NOT zero configuration, ask yourself why not.

majewsky 9y ago
This is a library, not an application. Can you give an example of any program that does not require configuration? Even yes(1) can be configured to output a different string than "yes".
- pritambaral 9y ago
  Unrelated to the OP, but I think zero-configuration doesn't mean "you cannot configure it", rather "you shouldn't have to configure it". In that sense, yes(1) is "zero configuration".
  - majewsky 9y ago
    I see. In that sense, it definitely had its merits, although there is the risk of your users never noticing that configuration options are available to accommodate their specific needs better than the defaults.

rurban 9y ago

Very very immature. E.g. the string library only knows about ascii, and does matching via tolower. Not usable for strings, only for memory. It doesn't known about unicode, UTF-8, UTF-16 nor UTF-32. Matching must be done via foldcase (fc) not tolower, There's a string splitter which only does ascii, so if it's UTF8 you get invalid strings, if it's wchar_t it's not usable. The good things are: noexcept, no bloat, FixedArray as better alloca(), InlinedVector similar. folly is much more advanced.

sigi45 9y ago

In germany it sounds like 'einen abseilen' aka 'to go to the toilet and dump one'

majewsky 9y ago
Technically, the verb "abseilen" just means "to rope down", but the collision with this idiom is rather unfortunate.
Geeflow 9y ago
I am not sure how common that is, though. I am a native speaker and have not heard that expression before.
- sigi45 9y ago
  I don't think it is common to say it often but at least in Bavaria, people know this.