12 ms·

Common Expression Language

hadrien01 6y ago
At the end of the README: Disclaimer: This is not an official Google product.
- kyrra 6y ago
  I'm a Googler, opinions are my own. A lot of the times engineers at Google will open source libraries or tools they have worked on, which go under the Google GitHub repo, but are attached with that language. This is basically saying that it is owned by Google but it is not something Google is officially supporting. It may continue to get updates, it may not. I've definitely seen some libraries open sourced from Google, that stopped being pushed externally once the primary driver behind it left Google or moved on to other projects. You can actually read some of the process that a Googler will follow when open sourcing software here: https://opensource.google/docs/releasing/ https://opensource.google/docs/releasing/
  - neolog 6y ago
    What is the significance to a user of whether Google is "officially supporting" a product or not?
    - azurezyq 6y ago
      E.g., if google is obliged to address your bugs / issues. It matters a lot in enterprise. This statement is more like a waiver you'd like to sign while using some entertainment equipements.
      - closeparen 6y ago
        An open source maintainer is never obligated to address your bugs / issues. However it is good open source citizenship to be clear about what level of support people can and can't expect.
    - kyrra 6y ago
      If it's not officially supported, it probably means it's just one or two people that open sourced it, and it would fall on them to keep it in sync with any internal work. Officially supported means it's actually owned by some team. They'll dedicate resources to it, meaning it will be accounted for on any project planning or resource management the management needs to do. Officially supported things tend to be bigger.
    - B-Con 6y ago
      "Official" is mostly about who they consider the customers of the product when making a decision. If the project is officially open-sourced, that will be taken into consideration when project priorities, re-orgs, and direction occur at higher levels. "How does this effect our commitment to the community" is a question to address. If it isn't official, then is best-effort by the people who pushed for it to go open-source.
  - bmitc 6y ago
    > It may continue to get updates, it may not. That basically goes for Google's official products as well.
- loosescrews 6y ago
  Google puts that on most of its open source products including both employee personal projects and projects where Google has employees whose job it is to contribute to it.
  - brown9-2 6y ago
    Does any project not include the waiver?
    - jitl 6y ago
      Yes, for example the Firebase Tools CLI is open source (MIT), but it does not include the waiver, because it is an official Google product: https://github.com/firebase/firebase-tools https://github.com/firebase/firebase-tools
- asciimike 6y ago
  It may not be itself, but it's used in a number of them (e.g. Security Rules for Cloud Firestore and Cloud Storage for Firebase).
- partialstate 6y ago
  Unless the open source is part of a paid offering (like Firebase CLI), it's required to be listed as not being an Official Google Product. That said, CEL is used in a number of publicly supported Google Cloud services which means that it's well supported with dedicated maintenance. Case in point, I'm the CEL lead at Google.
tlarkworthy 6y ago
Genuine trinary logic so exceptions are mapped to unknown but can still short circuit the overall expression. https://github.com/google/cel-spec/blob/master/doc/langdef.md#logical-operators https://github.com/google/cel-spec/blob/master/doc/langdef.m...
frou_dh 6y ago
I remember seeing this used inside config files for the Caddy webserver. Google seem to like these executable config languages because they've got another open source one ("Starlark") a few notches up in expressivity.
- throwaway894345 6y ago
  I'm an ardent supporter of executable config languages, especially for the infrastructure-as-code space (the only thing special about this space is that configs tend to be very large, so you're more likely to run into reuse issues), which markets itself as "it's just YAML!" but inevitably all of that copy/pasted YAML becomes unwieldy and you want reusability. At that point, you have a few distinct options: 1. Build an AST on top of YAML a la CloudFormation. Now you're programming in YAML, hurray! 2. Extend your static language with executable features a la Terraform/HCL, basically reinventing (and very badly, at that) more traditional language features 3. Use text templates, a la Helm--now you can generate syntactically invalid configuration! (and absolutely trivially, at that) 4. Use an expression language (familiar, ergonomics) a la Pulumi, Starlark, Nix, Nickel, Dhall, etc Note that in these conversations, someone inevitably shouts "use the simplest tool for the job!" ignoring that static configuration languages (and options 1-3 above) are strictly more complex for the reusability use cases outlined above. EDIT: Pulumi isn't an expression language; rather, it lets you use real languages to generate configuration, and these languages often include powerful expression features. AWS's CDK is also in this category.
  - didip 6y ago
    I am with you, I wish the world would just adopt Lua as their config files.
    - throwaway894345 6y ago
      I haven't used Lua, but I've used Starlark extensively and I will say that static typing is a boon, especially in the infra-as-code space where the feedback loop can be very long.
sandstrom 6y ago
I’m curious how this differs from Sentinel, hashicorps language for similar things. Too bad it isn’t open sourced. https://www.hashicorp.com/sentinel https://www.hashicorp.com/sentinel
- dvtkrlbs 6y ago
  The underlying language the hashicorp configuration language (HCL) is open source.
  - sandstrom 6y ago
    Sentinel and HCL are two different things
- tonyhb 6y ago
  Sentinel is basically OPA - openpolicyagent. OPA is open source and part of CNCF.
hardwaresofton 6y ago
This looks really similar to Open Policy Agent[0], wonder how they compare. [0]: https://www.openpolicyagent.org/ https://www.openpolicyagent.org/
- asciimike 6y ago
  I'll let Tristan or Torin comment more authoritatively, but back in 2017/8 CEL partnered with OPA and I believe CEL was used as the basis for expressions in their new version of Rego. I left the team about that time, so I don't know what exactly happened after that, but I wouldn't be surprised if the two are fairly close. My assumption is that's why CEL is polished up and OSS (I think we first published it a few years ago, why'd it get posted now?)
  - partialstate 6y ago
    OPA Rego and CEL are distinct, but you can see similar thinking in OPA Gatekeeper and CEL Policy Templates (https://github.com/google/cel-policy-templates-go https://github.com/google/cel-policy-templates-go) which are aimed at separating config from policy in order to create a better user experience. Note, the CEL Policy Templates are early in development, but build upon the abstractions provided by CEL.
- tonyhb 6y ago
  I use both in different projects. OPA (and its language Rego) is a good matching and policy engine with declarative blocks, modules, and expressive functions for HTTP headers, JWTs, etc. It's great for security. For building up abstractions and testing arbitrary JSON with complex, pre-defined policies. Testing is built in, and that's great. You can create "functions" and your own DSL for matching/evaluation. If you want ABAC, use OPA. In every case. CEL, and CEL-GO, is entirely different. It allows you to evaluate arbitrary expressions with random data. Think a search (eg. linkedin API's crappy search, or log searching, or random predicates). You would not define complex policies in CEL like you would in OPA. Well, I would not - you can define arbitrary macros and functions in CEL but it is not made for that scale. OPA is more suited for that. Some examples: - In OPA, you can define a policy that matches RBAC, ownership/acl, and ABAC in one file. With multi-tenancy. Think: "as a patient, I can see my data", and "as the patient's guardian, if they're under 18, I can see their data". And "As a doctor in the patient's clinic, I can see their data". And "as a clinical director in sudo mode, I can see their data". All in the same policy package, with tests. - OPA supports "partial evaluation". For example, if you only have a subset of data available, you can evaluate an OPA policy and have OPA tell you whether the policy evaluates to true or what data is missing. This is quite powerful for building up complex auth layers. - In CEL, you can say "all users > 30 days old". Simple, easy, filtering. EG, with a custom date macro, `date(users.created_at) > duration("30d")`. In short, use both. OPA for security and complex policies. CEL for user-defined "expressions".
contravariant 6y ago
I'd be a bit suspicious about the claim that it is not Turing complete. To be fair I can't yet find a way to allow arbitrary computation (though it seems easy to add one with fairly innocuous features). Although you can get it to solve 3-SAT, though only for some predefined number of variables (which it assures can be at least 32). Combinatorics stuff like printing all possible sudokus also seems like it should be feasible. Don't expect your config files to terminate when they use macros, that's all I'm saying.
- asciimike 6y ago
  As the former PM, can confirm it's not Turing complete, and there's a reason many of those innocuous features haven't been added ;)
- chc4 6y ago
  There's a difference between pathologically high complexity functions and Turing completeness. Sub-Turing languages generally don't allow recursion or unbounded loops - your program is always making progress. Solving 3-SAT doesn't sound like it precludes sub-Turing completeness, since you'd have a finite number of solutions you're iterating over. It's still useful for a config language because it makes it harder to accidentally make a config that (in practice) never terminates, and usually allows for easier static analysis and refactoring of the config files through immutability and purity.
  - contravariant 6y ago
    On the one hand you're right, though at some point 'arbitrarily long' and Turing complete become pretty similar. In fact an ordinary computer isn't entirely a Turing machine either, as its memory is limited. Also it means you need to be careful about malicious input, you need to take countermeasures when you evaluate an expression from an untrusted source.
  - blonde_ocean 6y ago
    How do you mean sub-Turing languages don’t allow recursion? Aren’t context-free languages, for example, literally recursive?
deleted 6y ago
[deleted]
brundolf 6y ago
Very cool idea. I'd be curious to find out more about what it is being/might be used for
- rossjudson 6y ago
  Cloud IAM Conditions are expressed in CEL: https://cloud.google.com/iam/docs/conditions-overview#cel https://cloud.google.com/iam/docs/conditions-overview#cel
- aviraldg 6y ago
  It's also used for Firebase security rules: https://firebase.google.com/docs/rules/rules-language https://firebase.google.com/docs/rules/rules-language
- kyessenov 6y ago
  Envoy proxy uses it for conditions in RBAC filter.
- 0xffea 6y ago
  Used in Tekton triggers.
deleted 6y ago
[deleted]
taeric 6y ago
I can't escape the feeling that emacs got this right. Nobody wants their config to be lisp, but it fits the bill for what you needed. Especially combined with the custom sections. So nice.
- remexre 6y ago
  Your config language being turing complete doesn't work if you need to accept/validate config files from untrusted parties
  - taeric 6y ago
    While I don't disagree, you can get rather far by limiting what you allow in the evaluation. There is no reason you have to pull it in in your current environment directly. And then you have an easy mechanism to allow some configs from trusted parties to be a bit more capable, if they need it.
  - contravariant 6y ago
    Though in that case you definitely want to disable macros for this config (as they allow for exponential time/space) and be very careful with any additional functions you expose.
  - filoeleven 6y ago
    edn seems like the right solution, then. https://github.com/edn-format/edn https://github.com/edn-format/edn
scythe 6y ago
This non-Turing-complete language space seems like it should be useful for UI themes and other user scripting in particular: - hard to write exploits, can be shared/installed without warnings - easy(-er) to predict behavior of themes so they won't break in new versions
dathinab 6y ago
> Expr = ConditionalOr ["?" ConditionalOr ":" Expr] ; I believe this was a mistake. "?" based conditionals aren't really a good idea IMHO.
- alfnoodles 6y ago
  Go on... :-). Why don't you like them?
  - dathinab 6y ago
    Nesting them can lead to very not-so-easy to understand logical expressions, furthermore if the parts between ?: is long enough it can also noticeable reduce readability. Instead of `<cond> ? <left> : <right>` I prefer `if <cond> { <left> } else { <right> }` the additional brackets noticeable improve readability and you can extend it to support `if <cond> { <a> } else if <cond2> { <b> } else { <c>` instead of `<cond> ? <a> : <cond2> ? <b> : <c>`. (Oh and that last example might be wrong needing brackets depending on operator precedence...) Through if you don't nest it it doesn't matter (oh and because it's a expression evaluation `else` is not optional but required as you need a value the expression resolves to).
    - alfnoodles 6y ago
      Ah. The biggest difference between ?/: and if/else is that ?/: is an expression (returns a value) and if/else are statements (a step/command/declaration/etc). You can build statements on top of CEL (lots do), but the core Common Expression Language (CEL) doesn't actually have them. Note that python uses if/else for the ternary expression form as well: `a = b if c else d` Though personally, I like to have the condition in the front, instead of in the middle.
      - st1ck 6y ago
        if/else doesn't have to be a statement: max = if a < b then b else a You may also like `jq` syntax with `elif`s: if cond1 then res1 elif cond2 then res2 ... else res end
jmeister 6y ago
Every major investment bank has an in-house variation of this, commonly called “payoff language” or “cashflow language”.
- jpcooper 6y ago
  How are they commonly implemented?
stevekemp 6y ago
I've never used this particular library, but I did put together my own simple evaluation engine and have found it very useful for a range of purposes. Initially it was designed to process incoming slack messages, and sometimes trigger a notification to an on-call engineer, but over time I've found uses for it processing email, scripting simple actions on my desktop, and more. https://github.com/skx/evalfilter/ https://github.com/skx/evalfilter/ These kind of things are pretty simple to write, but sometimes I almost think it is a shame there isn't something more standard. (Lua was kinda winning for that embedded-logic role for a long time, but nowadays we still have the mixture of YAML, HCL, and other niche-specific language/filtering and I imagine the time has passed to pick one standard.)
iso8859-1 6y ago
Looks like it has similar goals to the Dhall config language: https://dhall-lang.org https://dhall-lang.org . But Dhall has functions, so probably more powerful.
habosa 6y ago
Googler here: for those of you who have ever used Firebase this is the language that powers Cloud Firestore / Cloud Storage security rules. Specifically in our rules everything after the "if" is Common Expression Language. See: https://firebase.google.com/docs/firestore/security/rules-conditions https://firebase.google.com/docs/firestore/security/rules-co... The efficiency and safety of CEL enables us to put security rules in the critical path of every database request.
- hhas01 6y ago
  “this is the language that powers Cloud Firestore / Cloud Storage security rules” This really needs to be stated on its front page. Right now it’s all “Hows” and no “Why”. First question anyone looking at it asks: What problem does it solve?/What need does it fill? A real-world use case provides an easy relatable answer. Incidentally, with existing links to protobuf and no halting problem to worry about, it sounds like you’re halfway to having a remote query language a-la SQL too. https://www.researchgate.net/publication/221553413_Safe_Query_Objects_Statically-Typed_Objects_as_Remotely-Executable_Queries https://www.researchgate.net/publication/221553413_Safe_Quer...
  - saagarjha 6y ago
    I actually looked at this thinking it was pretty cool and could come up with usecases immediately–it is a great way to do simple query/matching in a consistent way. If I'm not mistaken, this could be used for something like Gmail's advanced search? There are tons of interfaces where everyone designs their own ad-hoc expressions and having something like this would be very useful in those cases.
  - alfnoodles 6y ago
    The link given if for the spec, this is probably a better "intro" page: https://opensource.google/projects/cel https://opensource.google/projects/cel
jt2190 6y ago
The creator of Apache Ant, James Duncan Davidson, wrote about choosing XML as the “language”: > Now, I never intended for the file format to become a scripting language—after all, my original view of Ant was that there was a declaration of some properties that described the project and that the tasks written in Java performed all the logic. The current maintainers of Ant generally share the same feelings. But when I fused XML and task reflection in Ant, I put together something that is 70-80% of a scripting environment. I just didn't recognize it at the time. To deny that people will use it as a scripting language is equivalent to asking them to pretend that sugar isn't sweet. https://web.archive.org/web/20041217023752///x180.net/Journal/2004/03/31.html https://web.archive.org/web/20041217023752///x180.net/Journa...
nojvek 6y ago
If you’re constantly fighting yaml, consider jsonnet. It’s another project by google and similarly not Turing complete. Works wonderful at generating templates.
- sbarzowski 6y ago
  Well, actually Jsonnet is Turing complete. Anyway, thanks for the shoutout.
  - nojvek 6y ago
    TIL. AFAIK jsonnet can’t do while loops so I assumed it wasn’t Turing complete and I didn’t see a way of writing a non-halting program. But now I see, since it has recursion, there are tricks to make it go in an infinite loop. Thanks for the pointer. I read the rationales again in the doc.
ilaksh 6y ago
What programming languages can you currently use this with?
partialstate 6y ago
The goal of CEL is fast, scalable, and portable expression evaluation. Fast - CEL runs without the need for sandboxing, making it much faster than sandboxed solutions like WebAssembly, Lua, and embedded JavaScript. Scalable - Features like variables and functions would make CEL more expressive, but also less scalable as it's easy to write a few lines of code with functions that consume exponential amounts of memory and compute. CEL is simply the expression and nothing more. Portable - CEL is implemented in Go[0], C++[1], and Python[2] with Java open sourcing in development. There is a public codelab[3] available for Go if anyone is interested. There is also a conformance suite in CEL-Spec to ensure consistent behavior between runtimes and environments. Our objective is to make it possible to bring CEL to K8s, J2EE apps, and C++ proxies. Evaluate at line-rate everywhere. Personally, I hope someone tries to make CEL work on IoT devices some day too. Where? - CEL is usually embedded into larger projects rather than being the one stop shop for solving a particular kind of problem. For example, CEL Policy Templates[4] has an opinionated way of using CEL to validate/evaluate YAML configs. Most of the time CEL is part of a service API. In addition to being used in Firebase's Cloud Firestore / Cloud Storage security rules, it is also used in several other Google Cloud services: - Cloud Armor[5] - IAM Conditions[6] - Cloud Healthcare Consents[7] - Cloud Build Notifiers[8] - Security Token Service[9] - Access Levels[10], and more. CEL is also used in some prominent open source projects like Envoy RBAC[11], Caddyserver[12], Krakend.io[13], and Cloud Custodian[14]. [0]: https://github.com/google/cel-go https://github.com/google/cel-go [1]: https://github.com/google/cel-cpp https://github.com/google/cel-cpp [2]: https://github.com/cloud-custodian/cel-python https://github.com/cloud-custodian/cel-python [3]: https://codelabs.developers.google.com/codelabs/cel-go https://codelabs.developers.google.com/codelabs/cel-go [4]: https://github.com/google/cel-policy-templates-go https://github.com/google/cel-policy-templates-go [5]: https://cloud.google.com/armor/docs/rules-language-reference https://cloud.google.com/armor/docs/rules-language-reference [6]: https://cloud.google.com/iam/docs/conditions-overview https://cloud.google.com/iam/docs/conditions-overview [7]: https://cloud.google.com/healthcare/docs/concepts/consent-model https://cloud.google.com/healthcare/docs/concepts/consent-mo... [8]: https://cloud.google.com/cloud-build/docs/filter-build-notifications https://cloud.google.com/cloud-build/docs/filter-build-notif... [9]: https://cloud.google.com/iam/docs/workload-identity-federation https://cloud.google.com/iam/docs/workload-identity-federati... [10]: https://cloud.google.com/access-context-manager/docs/custom-access-level-spec https://cloud.google.com/access-context-manager/docs/custom-... [11]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/security/rbac_filter.html?highlight=cel https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overv... [12]: https://caddyserver.com/docs/caddyfile/matchers#expression https://caddyserver.com/docs/caddyfile/matchers#expression [13]: https://www.krakend.io/docs/endpoints/common-expression-language-cel https://www.krakend.io/docs/endpoints/common-expression-lang... [14]: https://github.com/cloud-custodian/cel-python https://github.com/cloud-custodian/cel-python

At the end of the README: Disclaimer: This is not an official Google product.

kyrra 6y ago
I'm a Googler, opinions are my own. A lot of the times engineers at Google will open source libraries or tools they have worked on, which go under the Google GitHub repo, but are attached with that language. This is basically saying that it is owned by Google but it is not something Google is officially supporting. It may continue to get updates, it may not. I've definitely seen some libraries open sourced from Google, that stopped being pushed externally once the primary driver behind it left Google or moved on to other projects. You can actually read some of the process that a Googler will follow when open sourcing software here: https://opensource.google/docs/releasing/ https://opensource.google/docs/releasing/
- neolog 6y ago
  What is the significance to a user of whether Google is "officially supporting" a product or not?
  - azurezyq 6y ago
    E.g., if google is obliged to address your bugs / issues. It matters a lot in enterprise. This statement is more like a waiver you'd like to sign while using some entertainment equipements.
    - closeparen 6y ago
      An open source maintainer is never obligated to address your bugs / issues. However it is good open source citizenship to be clear about what level of support people can and can't expect.
  - kyrra 6y ago
    If it's not officially supported, it probably means it's just one or two people that open sourced it, and it would fall on them to keep it in sync with any internal work. Officially supported means it's actually owned by some team. They'll dedicate resources to it, meaning it will be accounted for on any project planning or resource management the management needs to do. Officially supported things tend to be bigger.
  - B-Con 6y ago
    "Official" is mostly about who they consider the customers of the product when making a decision. If the project is officially open-sourced, that will be taken into consideration when project priorities, re-orgs, and direction occur at higher levels. "How does this effect our commitment to the community" is a question to address. If it isn't official, then is best-effort by the people who pushed for it to go open-source.
- bmitc 6y ago
  > It may continue to get updates, it may not. That basically goes for Google's official products as well.
loosescrews 6y ago
Google puts that on most of its open source products including both employee personal projects and projects where Google has employees whose job it is to contribute to it.
- brown9-2 6y ago
  Does any project not include the waiver?
  - jitl 6y ago
    Yes, for example the Firebase Tools CLI is open source (MIT), but it does not include the waiver, because it is an official Google product: https://github.com/firebase/firebase-tools https://github.com/firebase/firebase-tools
asciimike 6y ago
It may not be itself, but it's used in a number of them (e.g. Security Rules for Cloud Firestore and Cloud Storage for Firebase).
partialstate 6y ago
Unless the open source is part of a paid offering (like Firebase CLI), it's required to be listed as not being an Official Google Product. That said, CEL is used in a number of publicly supported Google Cloud services which means that it's well supported with dedicated maintenance. Case in point, I'm the CEL lead at Google.

Genuine trinary logic so exceptions are mapped to unknown but can still short circuit the overall expression. https://github.com/google/cel-spec/blob/master/doc/langdef.md#logical-operators https://github.com/google/cel-spec/blob/master/doc/langdef.m...

frou_dh 6y ago

I remember seeing this used inside config files for the Caddy webserver. Google seem to like these executable config languages because they've got another open source one ("Starlark") a few notches up in expressivity.

throwaway894345 6y ago
I'm an ardent supporter of executable config languages, especially for the infrastructure-as-code space (the only thing special about this space is that configs tend to be very large, so you're more likely to run into reuse issues), which markets itself as "it's just YAML!" but inevitably all of that copy/pasted YAML becomes unwieldy and you want reusability. At that point, you have a few distinct options: 1. Build an AST on top of YAML a la CloudFormation. Now you're programming in YAML, hurray! 2. Extend your static language with executable features a la Terraform/HCL, basically reinventing (and very badly, at that) more traditional language features 3. Use text templates, a la Helm--now you can generate syntactically invalid configuration! (and absolutely trivially, at that) 4. Use an expression language (familiar, ergonomics) a la Pulumi, Starlark, Nix, Nickel, Dhall, etc Note that in these conversations, someone inevitably shouts "use the simplest tool for the job!" ignoring that static configuration languages (and options 1-3 above) are strictly more complex for the reusability use cases outlined above. EDIT: Pulumi isn't an expression language; rather, it lets you use real languages to generate configuration, and these languages often include powerful expression features. AWS's CDK is also in this category.
- didip 6y ago
  I am with you, I wish the world would just adopt Lua as their config files.
  - throwaway894345 6y ago
    I haven't used Lua, but I've used Starlark extensively and I will say that static typing is a boon, especially in the infra-as-code space where the feedback loop can be very long.

sandstrom 6y ago

I’m curious how this differs from Sentinel, hashicorps language for similar things. Too bad it isn’t open sourced. https://www.hashicorp.com/sentinel https://www.hashicorp.com/sentinel

dvtkrlbs 6y ago
The underlying language the hashicorp configuration language (HCL) is open source.
- sandstrom 6y ago
  Sentinel and HCL are two different things
tonyhb 6y ago
Sentinel is basically OPA - openpolicyagent. OPA is open source and part of CNCF.

hardwaresofton 6y ago

This looks really similar to Open Policy Agent[0], wonder how they compare. [0]: https://www.openpolicyagent.org/ https://www.openpolicyagent.org/

asciimike 6y ago
I'll let Tristan or Torin comment more authoritatively, but back in 2017/8 CEL partnered with OPA and I believe CEL was used as the basis for expressions in their new version of Rego. I left the team about that time, so I don't know what exactly happened after that, but I wouldn't be surprised if the two are fairly close. My assumption is that's why CEL is polished up and OSS (I think we first published it a few years ago, why'd it get posted now?)
- partialstate 6y ago
  OPA Rego and CEL are distinct, but you can see similar thinking in OPA Gatekeeper and CEL Policy Templates (https://github.com/google/cel-policy-templates-go https://github.com/google/cel-policy-templates-go) which are aimed at separating config from policy in order to create a better user experience. Note, the CEL Policy Templates are early in development, but build upon the abstractions provided by CEL.
tonyhb 6y ago
I use both in different projects. OPA (and its language Rego) is a good matching and policy engine with declarative blocks, modules, and expressive functions for HTTP headers, JWTs, etc. It's great for security. For building up abstractions and testing arbitrary JSON with complex, pre-defined policies. Testing is built in, and that's great. You can create "functions" and your own DSL for matching/evaluation. If you want ABAC, use OPA. In every case. CEL, and CEL-GO, is entirely different. It allows you to evaluate arbitrary expressions with random data. Think a search (eg. linkedin API's crappy search, or log searching, or random predicates). You would not define complex policies in CEL like you would in OPA. Well, I would not - you can define arbitrary macros and functions in CEL but it is not made for that scale. OPA is more suited for that. Some examples: - In OPA, you can define a policy that matches RBAC, ownership/acl, and ABAC in one file. With multi-tenancy. Think: "as a patient, I can see my data", and "as the patient's guardian, if they're under 18, I can see their data". And "As a doctor in the patient's clinic, I can see their data". And "as a clinical director in sudo mode, I can see their data". All in the same policy package, with tests. - OPA supports "partial evaluation". For example, if you only have a subset of data available, you can evaluate an OPA policy and have OPA tell you whether the policy evaluates to true or what data is missing. This is quite powerful for building up complex auth layers. - In CEL, you can say "all users > 30 days old". Simple, easy, filtering. EG, with a custom date macro, `date(users.created_at) > duration("30d")`. In short, use both. OPA for security and complex policies. CEL for user-defined "expressions".

contravariant 6y ago

I'd be a bit suspicious about the claim that it is not Turing complete. To be fair I can't yet find a way to allow arbitrary computation (though it seems easy to add one with fairly innocuous features). Although you can get it to solve 3-SAT, though only for some predefined number of variables (which it assures can be at least 32). Combinatorics stuff like printing all possible sudokus also seems like it should be feasible. Don't expect your config files to terminate when they use macros, that's all I'm saying.

asciimike 6y ago
As the former PM, can confirm it's not Turing complete, and there's a reason many of those innocuous features haven't been added ;)
chc4 6y ago
There's a difference between pathologically high complexity functions and Turing completeness. Sub-Turing languages generally don't allow recursion or unbounded loops - your program is always making progress. Solving 3-SAT doesn't sound like it precludes sub-Turing completeness, since you'd have a finite number of solutions you're iterating over. It's still useful for a config language because it makes it harder to accidentally make a config that (in practice) never terminates, and usually allows for easier static analysis and refactoring of the config files through immutability and purity.
- contravariant 6y ago
  On the one hand you're right, though at some point 'arbitrarily long' and Turing complete become pretty similar. In fact an ordinary computer isn't entirely a Turing machine either, as its memory is limited. Also it means you need to be careful about malicious input, you need to take countermeasures when you evaluate an expression from an untrusted source.
- blonde_ocean 6y ago
  How do you mean sub-Turing languages don’t allow recursion? Aren’t context-free languages, for example, literally recursive?

deleted 6y ago

[deleted]

brundolf 6y ago

Very cool idea. I'd be curious to find out more about what it is being/might be used for

rossjudson 6y ago
Cloud IAM Conditions are expressed in CEL: https://cloud.google.com/iam/docs/conditions-overview#cel https://cloud.google.com/iam/docs/conditions-overview#cel
aviraldg 6y ago
It's also used for Firebase security rules: https://firebase.google.com/docs/rules/rules-language https://firebase.google.com/docs/rules/rules-language
kyessenov 6y ago
Envoy proxy uses it for conditions in RBAC filter.
0xffea 6y ago
Used in Tekton triggers.

deleted 6y ago

[deleted]

taeric 6y ago

I can't escape the feeling that emacs got this right. Nobody wants their config to be lisp, but it fits the bill for what you needed. Especially combined with the custom sections. So nice.

remexre 6y ago
Your config language being turing complete doesn't work if you need to accept/validate config files from untrusted parties
- taeric 6y ago
  While I don't disagree, you can get rather far by limiting what you allow in the evaluation. There is no reason you have to pull it in in your current environment directly. And then you have an easy mechanism to allow some configs from trusted parties to be a bit more capable, if they need it.
- contravariant 6y ago
  Though in that case you definitely want to disable macros for this config (as they allow for exponential time/space) and be very careful with any additional functions you expose.
- filoeleven 6y ago
  edn seems like the right solution, then. https://github.com/edn-format/edn https://github.com/edn-format/edn

scythe 6y ago

This non-Turing-complete language space seems like it should be useful for UI themes and other user scripting in particular: - hard to write exploits, can be shared/installed without warnings - easy(-er) to predict behavior of themes so they won't break in new versions

dathinab 6y ago

> Expr = ConditionalOr ["?" ConditionalOr ":" Expr] ; I believe this was a mistake. "?" based conditionals aren't really a good idea IMHO.

alfnoodles 6y ago
Go on... :-). Why don't you like them?
- dathinab 6y ago
  Nesting them can lead to very not-so-easy to understand logical expressions, furthermore if the parts between ?: is long enough it can also noticeable reduce readability. Instead of `<cond> ? <left> : <right>` I prefer `if <cond> { <left> } else { <right> }` the additional brackets noticeable improve readability and you can extend it to support `if <cond> { <a> } else if <cond2> { <b> } else { <c>` instead of `<cond> ? <a> : <cond2> ? <b> : <c>`. (Oh and that last example might be wrong needing brackets depending on operator precedence...) Through if you don't nest it it doesn't matter (oh and because it's a expression evaluation `else` is not optional but required as you need a value the expression resolves to).
  - alfnoodles 6y ago
    Ah. The biggest difference between ?/: and if/else is that ?/: is an expression (returns a value) and if/else are statements (a step/command/declaration/etc). You can build statements on top of CEL (lots do), but the core Common Expression Language (CEL) doesn't actually have them. Note that python uses if/else for the ternary expression form as well: `a = b if c else d` Though personally, I like to have the condition in the front, instead of in the middle.
    - st1ck 6y ago
      if/else doesn't have to be a statement: max = if a < b then b else a You may also like `jq` syntax with `elif`s: if cond1 then res1 elif cond2 then res2 ... else res end

jmeister 6y ago

Every major investment bank has an in-house variation of this, commonly called “payoff language” or “cashflow language”.

jpcooper 6y ago
How are they commonly implemented?

stevekemp 6y ago

I've never used this particular library, but I did put together my own simple evaluation engine and have found it very useful for a range of purposes. Initially it was designed to process incoming slack messages, and sometimes trigger a notification to an on-call engineer, but over time I've found uses for it processing email, scripting simple actions on my desktop, and more. https://github.com/skx/evalfilter/ https://github.com/skx/evalfilter/ These kind of things are pretty simple to write, but sometimes I almost think it is a shame there isn't something more standard. (Lua was kinda winning for that embedded-logic role for a long time, but nowadays we still have the mixture of YAML, HCL, and other niche-specific language/filtering and I imagine the time has passed to pick one standard.)

iso8859-1 6y ago

Looks like it has similar goals to the Dhall config language: https://dhall-lang.org https://dhall-lang.org . But Dhall has functions, so probably more powerful.

habosa 6y ago

Googler here: for those of you who have ever used Firebase this is the language that powers Cloud Firestore / Cloud Storage security rules. Specifically in our rules everything after the "if" is Common Expression Language. See: https://firebase.google.com/docs/firestore/security/rules-conditions https://firebase.google.com/docs/firestore/security/rules-co... The efficiency and safety of CEL enables us to put security rules in the critical path of every database request.

hhas01 6y ago
“this is the language that powers Cloud Firestore / Cloud Storage security rules” This really needs to be stated on its front page. Right now it’s all “Hows” and no “Why”. First question anyone looking at it asks: What problem does it solve?/What need does it fill? A real-world use case provides an easy relatable answer. Incidentally, with existing links to protobuf and no halting problem to worry about, it sounds like you’re halfway to having a remote query language a-la SQL too. https://www.researchgate.net/publication/221553413_Safe_Query_Objects_Statically-Typed_Objects_as_Remotely-Executable_Queries https://www.researchgate.net/publication/221553413_Safe_Quer...
- saagarjha 6y ago
  I actually looked at this thinking it was pretty cool and could come up with usecases immediately–it is a great way to do simple query/matching in a consistent way. If I'm not mistaken, this could be used for something like Gmail's advanced search? There are tons of interfaces where everyone designs their own ad-hoc expressions and having something like this would be very useful in those cases.
- alfnoodles 6y ago
  The link given if for the spec, this is probably a better "intro" page: https://opensource.google/projects/cel https://opensource.google/projects/cel

jt2190 6y ago

The creator of Apache Ant, James Duncan Davidson, wrote about choosing XML as the “language”: > Now, I never intended for the file format to become a scripting language—after all, my original view of Ant was that there was a declaration of some properties that described the project and that the tasks written in Java performed all the logic. The current maintainers of Ant generally share the same feelings. But when I fused XML and task reflection in Ant, I put together something that is 70-80% of a scripting environment. I just didn't recognize it at the time. To deny that people will use it as a scripting language is equivalent to asking them to pretend that sugar isn't sweet. https://web.archive.org/web/20041217023752///x180.net/Journal/2004/03/31.html https://web.archive.org/web/20041217023752///x180.net/Journa...

nojvek 6y ago

If you’re constantly fighting yaml, consider jsonnet. It’s another project by google and similarly not Turing complete. Works wonderful at generating templates.

sbarzowski 6y ago
Well, actually Jsonnet is Turing complete. Anyway, thanks for the shoutout.
- nojvek 6y ago
  TIL. AFAIK jsonnet can’t do while loops so I assumed it wasn’t Turing complete and I didn’t see a way of writing a non-halting program. But now I see, since it has recursion, there are tricks to make it go in an infinite loop. Thanks for the pointer. I read the rationales again in the doc.

ilaksh 6y ago

What programming languages can you currently use this with?

partialstate 6y ago

The goal of CEL is fast, scalable, and portable expression evaluation. Fast - CEL runs without the need for sandboxing, making it much faster than sandboxed solutions like WebAssembly, Lua, and embedded JavaScript. Scalable - Features like variables and functions would make CEL more expressive, but also less scalable as it's easy to write a few lines of code with functions that consume exponential amounts of memory and compute. CEL is simply the expression and nothing more. Portable - CEL is implemented in Go[0], C++[1], and Python[2] with Java open sourcing in development. There is a public codelab[3] available for Go if anyone is interested. There is also a conformance suite in CEL-Spec to ensure consistent behavior between runtimes and environments. Our objective is to make it possible to bring CEL to K8s, J2EE apps, and C++ proxies. Evaluate at line-rate everywhere. Personally, I hope someone tries to make CEL work on IoT devices some day too. Where? - CEL is usually embedded into larger projects rather than being the one stop shop for solving a particular kind of problem. For example, CEL Policy Templates[4] has an opinionated way of using CEL to validate/evaluate YAML configs. Most of the time CEL is part of a service API. In addition to being used in Firebase's Cloud Firestore / Cloud Storage security rules, it is also used in several other Google Cloud services: - Cloud Armor[5] - IAM Conditions[6] - Cloud Healthcare Consents[7] - Cloud Build Notifiers[8] - Security Token Service[9] - Access Levels[10], and more. CEL is also used in some prominent open source projects like Envoy RBAC[11], Caddyserver[12], Krakend.io[13], and Cloud Custodian[14]. [0]: https://github.com/google/cel-go https://github.com/google/cel-go [1]: https://github.com/google/cel-cpp https://github.com/google/cel-cpp [2]: https://github.com/cloud-custodian/cel-python https://github.com/cloud-custodian/cel-python [3]: https://codelabs.developers.google.com/codelabs/cel-go https://codelabs.developers.google.com/codelabs/cel-go [4]: https://github.com/google/cel-policy-templates-go https://github.com/google/cel-policy-templates-go [5]: https://cloud.google.com/armor/docs/rules-language-reference https://cloud.google.com/armor/docs/rules-language-reference [6]: https://cloud.google.com/iam/docs/conditions-overview https://cloud.google.com/iam/docs/conditions-overview [7]: https://cloud.google.com/healthcare/docs/concepts/consent-model https://cloud.google.com/healthcare/docs/concepts/consent-mo... [8]: https://cloud.google.com/cloud-build/docs/filter-build-notifications https://cloud.google.com/cloud-build/docs/filter-build-notif... [9]: https://cloud.google.com/iam/docs/workload-identity-federation https://cloud.google.com/iam/docs/workload-identity-federati... [10]: https://cloud.google.com/access-context-manager/docs/custom-access-level-spec https://cloud.google.com/access-context-manager/docs/custom-... [11]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/security/rbac_filter.html?highlight=cel https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overv... [12]: https://caddyserver.com/docs/caddyfile/matchers#expression https://caddyserver.com/docs/caddyfile/matchers#expression [13]: https://www.krakend.io/docs/endpoints/common-expression-language-cel https://www.krakend.io/docs/endpoints/common-expression-lang... [14]: https://github.com/cloud-custodian/cel-python https://github.com/cloud-custodian/cel-python