4 ms·

re: Grafana (i'm the author of the linked issue) - i'm quite excited, i do think there's a world of possibilities here. Two-way sync with a git repo is one pos

re: Grafana (i'm the author of the linked issue) - i'm quite excited, i do think there's a world of possibilities here. Two-way sync with a git repo is one possible path, and we've talked a lot internally at GL about how to best support it. My sense is that we can do it with relatively little friction and likely will - but if you're just syncing with a git repo, there's still a lot of arbitrary, opaque repo layout decisions that still have to be made (how do you map a filesystem position for a dashboard to a position in Grafana? In a way that places the dashboards next to the systems they're intended to observe? With many teams? With many Grafana instances?) which induce new kinds of friction at scale. Fortunately - and not mutually exclusively with the above - by building the system for schema in CUE, we've made a composable thing that we can make into larger systems. That's what we're starting to do with Polly: https://github.com/pollypkg/polly https://github.com/pollypkg/polly Conveniently, my parts of a Grafanaconline talk tomorrow discusses both of these https://grafana.com/go/grafanaconline/2021/dashboards-as-code/ https://grafana.com/go/grafanaconline/2021/dashboards-as-cod... :D

thelastbender12 5y ago
This seems really exciting. I haven't had the chance to use Grafana yet; from the linked issue, am I understanding correctly that you'll be able to serialize dashboards to Cue schema, and hence get all the niceties of a structured representation - versioning, non-visual editing, and reproducibility? I recall seeing another project HN which created dashboards out of a yaml description. This seems like a fantastic idea, given that a lot of business panels and dashboard apps can be implemented with a limited set of UI interactions.
- gervwyk 5y ago
  We've been building dashboards in YAML for a while now using Lowdefy - https://github.com/lowdefy/lowdefy https://github.com/lowdefy/lowdefy It is very flexible. Will publish more examples soon!
  - thelastbender12 5y ago
    Yep, this was the one! It looks great though I hadn't got around to trying it. Do you think something like Cue makes for a better representation for lowdefy apps than YAML, since it seems to offer better abstraction ability and hence easier to compose?
    - gervwyk 5y ago
      It's an interesting thought. I'll definitely spend some time to consider how this could work. Although, writing apps in yaml or json works really well currently. We mostly express logic through Lowdefy operators. We like supporting yaml / json since it is easy to write code to create or update such apps in any language.
- sdboyer 5y ago
  > you'll be able to serialize dashboards to Cue schema It'll be possible to serialize/represent the dashboards in CUE. Here's a handwavy, pseudocode-y example i use in the talk: https://gist.github.com/sdboyer/d76196f94ca78d1e84c739e95e64d3eb https://gist.github.com/sdboyer/d76196f94ca78d1e84c739e95e64... That said, it's not like we're planning on replacing all the "export JSON" buttons in the Grafana UI with "export CUE." One of the interesting properties of defining schemas in CUE is how it allows us to remove schema-defined default values from a dashboard's JSON. The JSON representation can actually look a lot more like a concise CUE representation. > versioning Versioning of the Grafana schema is the essential design goal of the "scuemata" system that is under discussion in that epic issue Versioning of artifacts that are instances of the schema is a key goal with Polly https://docs.google.com/document/d/1GU0DGy-X6z4FVwbJYPsBKRdqApi2RppW0q2U6YUXOp8/edit#heading=h.psq84twzkwrt https://docs.google.com/document/d/1GU0DGy-X6z4FVwbJYPsBKRdq... > non-visual editing Like, editing something other than raw code in an editor? Yes, this is also something directly enabled by the schema (again, see the Polly doc, the "Produce" heading). For data-intensive tasks, such editing experiences are the only way to see your logic in the context of data, and therefore IMO prerequisite for confidence > reproducibility Yup. This already isn't "hard" to do today, but reproducibility gets more complicated at scale - those questions about how to map what's on disk to what's in your Grafana (or whatever app) in my parent comment become more complicated, leading to friction, leading to staleness.
  - thelastbender12 5y ago
    Thank you for the answer! Think I missed the grafana related context, but the design document is really instructive.

thelastbender12 5y ago

This seems really exciting. I haven't had the chance to use Grafana yet; from the linked issue, am I understanding correctly that you'll be able to serialize dashboards to Cue schema, and hence get all the niceties of a structured representation - versioning, non-visual editing, and reproducibility? I recall seeing another project HN which created dashboards out of a yaml description. This seems like a fantastic idea, given that a lot of business panels and dashboard apps can be implemented with a limited set of UI interactions.

gervwyk 5y ago
We've been building dashboards in YAML for a while now using Lowdefy - https://github.com/lowdefy/lowdefy https://github.com/lowdefy/lowdefy It is very flexible. Will publish more examples soon!
- thelastbender12 5y ago
  Yep, this was the one! It looks great though I hadn't got around to trying it. Do you think something like Cue makes for a better representation for lowdefy apps than YAML, since it seems to offer better abstraction ability and hence easier to compose?
  - gervwyk 5y ago
    It's an interesting thought. I'll definitely spend some time to consider how this could work. Although, writing apps in yaml or json works really well currently. We mostly express logic through Lowdefy operators. We like supporting yaml / json since it is easy to write code to create or update such apps in any language.
sdboyer 5y ago
> you'll be able to serialize dashboards to Cue schema It'll be possible to serialize/represent the dashboards in CUE. Here's a handwavy, pseudocode-y example i use in the talk: https://gist.github.com/sdboyer/d76196f94ca78d1e84c739e95e64d3eb https://gist.github.com/sdboyer/d76196f94ca78d1e84c739e95e64... That said, it's not like we're planning on replacing all the "export JSON" buttons in the Grafana UI with "export CUE." One of the interesting properties of defining schemas in CUE is how it allows us to remove schema-defined default values from a dashboard's JSON. The JSON representation can actually look a lot more like a concise CUE representation. > versioning Versioning of the Grafana schema is the essential design goal of the "scuemata" system that is under discussion in that epic issue Versioning of artifacts that are instances of the schema is a key goal with Polly https://docs.google.com/document/d/1GU0DGy-X6z4FVwbJYPsBKRdqApi2RppW0q2U6YUXOp8/edit#heading=h.psq84twzkwrt https://docs.google.com/document/d/1GU0DGy-X6z4FVwbJYPsBKRdq... > non-visual editing Like, editing something other than raw code in an editor? Yes, this is also something directly enabled by the schema (again, see the Polly doc, the "Produce" heading). For data-intensive tasks, such editing experiences are the only way to see your logic in the context of data, and therefore IMO prerequisite for confidence > reproducibility Yup. This already isn't "hard" to do today, but reproducibility gets more complicated at scale - those questions about how to map what's on disk to what's in your Grafana (or whatever app) in my parent comment become more complicated, leading to friction, leading to staleness.
- thelastbender12 5y ago
  Thank you for the answer! Think I missed the grafana related context, but the design document is really instructive.