8 ms·

Am I the only one who is always slightly disappointed that neither the README file on Github nor the landing page at the website tells me why I would want to us

Am I the only one who is always slightly disappointed that neither the README file on Github nor the landing page at the website tells me why I would want to use the software in question? What problem it solves? Why might "a distributed, fault-tolerant observability pipeline" be interesting to programmers or anyone else? It seems like you've already got to be familiar with the problem space to understand what this is and what need it fulfills. I'm not picking on this package. I see it all the time. Can someone here explain to me what the use case is for this software?

jmillikin 8y ago
The use case: you have more than a hundred machines emitting lots of monitoring data, much of which is uninteresting except in aggregate form. Instead of paying to store millions of data points and then computing the aggregates later, Veneur can calculate things like percentiles itself and only forwards those. It also has a separate, related purpose as a statsd protocol transport. You run it on the statsd port and it receives standard (or DataDog-extended) UDP traffic, then it forwards metrics via TCP to another backend. This has reliability benefits when operating over a network that might drop UDP packets, such as the public internet.
- tchaffee 8y ago
  Thanks. That's pretty good. Every README needs to start with something like this: "If you monitor applications, Veneur can save you money and increase reliability"
  - mrkurt 8y ago
    That sounds like a startup's elevator pitch, not a README for an OSS project.
    - tchaffee 8y ago
      It kind of does sound like the first line of an elevator pitch, doesn't it? And as a programmer it helps me quickly decide if I should be care about it at all, and how to categorize it. Take a look at the software this "pipeline" is supposed to integrate with. It's a commercial product. Their landing page could not be clearer: "Modern monitoring & analytics. See inside any stack, any app, at any scale, anywhere" Perfect. Now I know what they do in plain English, how to mentally categorize their product, and even how to bookmark the page for the future when I might need their product. React is an OSS product, and it comes close "A declarative, efficient, and flexible JavaScript library for building user interfaces." Tells me an awful lot with only one jargon word: declarative. But I can still categorize it mentally and bookmark it even if I don't know what that words means. Make it easy for people to understand what you do, and you'll get more interest. People will bother to bookmark it. And maybe come back.
abathur 8y ago
I regularly have the same experience. When I run into one, I optimistically assume I'd know if I needed it (but, unfortunately, it means I'm very unlikely to remember a specific package if I do eventually run into the problem. If they don't already exist, maybe there's room in the universe for some README best-practices (like those floating around for writing commit messages, user stories, issue reports, etc.) that might nudge more maintainers to include at least one lucid example of a problem it could solve.
mrkurt 8y ago
It's not always possible to describe a problem in a way you will understand if you aren't already familiar with it. And that's ok! Reading something like "distributed, fault-tolerant pipeline for observability data" and ¯\_(ツ)_/¯ could be good response both for you and the people who built the thing. It's definitely ok that you have to dig a little more to wrap your head around it. In short, it's reasonable to expect that people who see a project already understand the problem space and write for the ones who can say "yep I need this". This particular project is probably only useful to people who know what observability data means.
- aaronbrethorst 8y ago
  Even though Albert Einstein seemingly didn't actually say this, I still think this aphorism is appropriate: If you can't explain it simply, you don't understand it well enough.
  - mrkurt 8y ago
    Is it? Because there's an awful lot of base knowledge you need to understand any kind of OSS project. Can you explain what nginx does to someone who doesn't have a smart phone in any kind of meaningful way?
    - aaronbrethorst 8y ago
      I have fifteen years of professional experience in software development and I didn't have the slightest clue what Veneur did from its description of "A distributed, fault-tolerant pipeline for observability data." - Distributed - Got this one, but it's an adjective and doesn't mean much on its own. - Fault-tolerant - Same. - Pipeline - OK, great, totally meaningless noun. I guess stuff goes in it? - Observability data - What the heck is observability data? Nginx is a web server, plus a heck of a lot more. Their website does a terrific job of explaining it. https://www.nginx.com/resources/glossary/nginx/ https://www.nginx.com/resources/glossary/nginx/
      - mrkurt 8y ago
SuperKlaus 8y ago
Thanks asking this. I came back here after looking at the README in the repo and was still clueless as to why I'd want to use Veneur.

jmillikin 8y ago

The use case: you have more than a hundred machines emitting lots of monitoring data, much of which is uninteresting except in aggregate form. Instead of paying to store millions of data points and then computing the aggregates later, Veneur can calculate things like percentiles itself and only forwards those. It also has a separate, related purpose as a statsd protocol transport. You run it on the statsd port and it receives standard (or DataDog-extended) UDP traffic, then it forwards metrics via TCP to another backend. This has reliability benefits when operating over a network that might drop UDP packets, such as the public internet.

tchaffee 8y ago
Thanks. That's pretty good. Every README needs to start with something like this: "If you monitor applications, Veneur can save you money and increase reliability"
- mrkurt 8y ago
  That sounds like a startup's elevator pitch, not a README for an OSS project.
  - tchaffee 8y ago
    It kind of does sound like the first line of an elevator pitch, doesn't it? And as a programmer it helps me quickly decide if I should be care about it at all, and how to categorize it. Take a look at the software this "pipeline" is supposed to integrate with. It's a commercial product. Their landing page could not be clearer: "Modern monitoring & analytics. See inside any stack, any app, at any scale, anywhere" Perfect. Now I know what they do in plain English, how to mentally categorize their product, and even how to bookmark the page for the future when I might need their product. React is an OSS product, and it comes close "A declarative, efficient, and flexible JavaScript library for building user interfaces." Tells me an awful lot with only one jargon word: declarative. But I can still categorize it mentally and bookmark it even if I don't know what that words means. Make it easy for people to understand what you do, and you'll get more interest. People will bother to bookmark it. And maybe come back.

abathur 8y ago

I regularly have the same experience. When I run into one, I optimistically assume I'd know if I needed it (but, unfortunately, it means I'm very unlikely to remember a specific package if I do eventually run into the problem. If they don't already exist, maybe there's room in the universe for some README best-practices (like those floating around for writing commit messages, user stories, issue reports, etc.) that might nudge more maintainers to include at least one lucid example of a problem it could solve.

mrkurt 8y ago

It's not always possible to describe a problem in a way you will understand if you aren't already familiar with it. And that's ok! Reading something like "distributed, fault-tolerant pipeline for observability data" and ¯\_(ツ)_/¯ could be good response both for you and the people who built the thing. It's definitely ok that you have to dig a little more to wrap your head around it. In short, it's reasonable to expect that people who see a project already understand the problem space and write for the ones who can say "yep I need this". This particular project is probably only useful to people who know what observability data means.

aaronbrethorst 8y ago
Even though Albert Einstein seemingly didn't actually say this, I still think this aphorism is appropriate: If you can't explain it simply, you don't understand it well enough.
- mrkurt 8y ago
  Is it? Because there's an awful lot of base knowledge you need to understand any kind of OSS project. Can you explain what nginx does to someone who doesn't have a smart phone in any kind of meaningful way?
  - aaronbrethorst 8y ago
    I have fifteen years of professional experience in software development and I didn't have the slightest clue what Veneur did from its description of "A distributed, fault-tolerant pipeline for observability data." - Distributed - Got this one, but it's an adjective and doesn't mean much on its own. - Fault-tolerant - Same. - Pipeline - OK, great, totally meaningless noun. I guess stuff goes in it? - Observability data - What the heck is observability data? Nginx is a web server, plus a heck of a lot more. Their website does a terrific job of explaining it. https://www.nginx.com/resources/glossary/nginx/ https://www.nginx.com/resources/glossary/nginx/
    - mrkurt 8y ago

SuperKlaus 8y ago

Thanks asking this. I came back here after looking at the README in the repo and was still clueless as to why I'd want to use Veneur.