6 ms·

Not sure I entirely follow what's going on here. Is there some context behind this that's useful to know?

by icepat 4y ago

tonyhb 4y ago
If I understand correctly, this is a receiver — which means you can take your existing DD instrumentation in their format and have it translated to OTel using this code. It lowers the switching cost to get off of DD.
- trynewideas 4y ago
  And Datadog competitor Grafana sponsored it: https://twitter.com/boostchicken/status/1618692475845238784 https://twitter.com/boostchicken/status/1618692475845238784
  - deleted 4y ago
    [deleted]
- xen2xen1 4y ago
  Protect the moat at all costs!
  - benatkin 4y ago
    It's worse than that, they want security through obscurity too. They feel like someone is inappropriately tinkering with the agent they want customers to install. It's open source: https://github.com/DataDog/datadog-agent https://github.com/DataDog/datadog-agent ...but the downloads are behind a login: https://github.com/DataDog/datadog-agent#datadog-agent https://github.com/DataDog/datadog-agent#datadog-agent
    - judge2020 4y ago
      Not really - the script it gives you in-app is DD_API_KEY= DD_SITE="datadoghq.com" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script_agent7.sh)" They tell you to sign in because installing without a key leads to non-working agents and support tickets. As for the libraries themselves, they're all on the regular package manager for that language, eg. pypi.
      - benatkin 4y ago
        Ah, that's good to know, thanks. I didn't have that link, but I can see it doesn't have an API key, so they can't change the download based on who's downloading it. That's good for transparency. Ditto the package managers. I see some Linux package manager public keys in there too. Edit: I missed the environment variable before "curl". The .sh is downloaded without the API key but the rest could be done using the API key, since it is passed to the script.
- benatkin 4y ago
  They want to keep their agent secret. Secret Agent Man...
  - angstrom 4y ago
    They've given you a pull number and taken 'way your name
xen2xen1 4y ago
Wondered the same. Assume this is a threat to their business? An open source competitor can do something cool so they want to kill the functionality?
- hosh 4y ago
  We use DD, and we have to use Otel for our Elixir apps to ingest spans into DD because there are no DD SDKs for Elixir or Erlang. From the perspective of a customer, I can tell you that DD already has quite a bit of a moat. Their main competitive advantage, and what got us into using it, is being able to correlate data across APM, custom metrics, and logging through the use of tagging. They then densely link data together across the platform. There is also a built-in Jupyter-style notebooks. By correlating data like that, you get more value out of ingesting as much data into DD as you can. There are some additional services we're not using, such as auto-correlation with ML (and alerting for anomaly detection), and security monitoring that also looks across the entire platform using their ML tech. Like AWS/GCP/Azure, it can get expensive, quite fast, using on-demand pricing, so there are negotiated annual contracts. Right now, our team is small, and to replicate the functionality we do use, using self-hosted open-source tooling, we might as well hire another engineer for just setting up and maintaining such a platform. I get it that, you want to defend the moat and that eroding the little things can lead to eroding the big things. As I see it though, if you need those correlations, you'll need a certain scale and team size before it makes sense to build out something like that for yourself.
  - aeyes 4y ago
    > negotiated annual contract Maybe we are too small but Datadog is one of the few vendors which we haven't been able to negotiate down in years. The price has always been whats on the website. I honestly don't even mind, with some vendors it feels like you are on a basar and they always tell you that their final discountns had to get approval by the CEO.
    - hosh 4y ago
      I don't know if we got a discount for an annual contract, or if the annual contract pricing was published on the website. But we were able to work out things like, using seasonal pricing because our traffic is seasonal.
ren_engineer 4y ago
OpenTelemetry basically allows for vendor neutral instrumentation for your metrics, logs, and traces. It makes it easy to switch to a cheaper or better service with minimal lock-in, which is obviously bad for DataDog and other companies business model
- MuffinFlavored 4y ago
  > It makes it easy to switch to a cheaper or better service with minimal lock-in Got any examples? I tried running my own "stack" for a project I wanted alerting on. I landed on Jaeger all-in-one (wasted time on Zipkin, the UI just was nowhere near as good as it ought to be) Docker container in docker-compose with COLLECTOR_OTLP_ENABLED.
  - rad_gruchalski 4y ago
    I used the collector to proxy otel traces to jaeger zipkin for uniform istio tracing.
  - xcasperx 4y ago
    Check out telemetryhub.com (admittedly I work here). We offer a free trial and don't charge per a seat.
    - vertis 4y ago
      I understand the need to charge money for things, but the $150 per month pretty much rules out any bootstrapped startups.
      - MuffinFlavored 4y ago
        I wish I knew the right answer between "spin up a cloud VPS Linux server and run your own with Docker compose" and pay $150/mo for something like DataDog
  - m-o11y 4y ago
    If you're looking for something a bit more "few-clicks-and-you-are-up-and-running", check out OpsVerse ObserveNow: https://opsverse.io/observenow-observability/ https://opsverse.io/observenow-observability/ .. Entirely powered by OSS tools, ingestion-driven pricing, and without the hassle of managing the stack and scaling up.
grepfru_it 4y ago
This comment highlights my problem with singular project tracking issues. Once they get so massive like this, people suddenly "can't follow along". Though not specific to GitHub but in this case the ui hampers readability and the comments are very much in-the-moment. This then leaves me to ask the questions, (a) should we stop writing such mammoth comments in issues or (b) should we leverage generative AI to summarize a weekly or monthly timeline of issue interaction. What is the point of having historical issues if it's too complex to follow along? This was a question I regularly asked at GitHub that usually resulted in blank stares. GitHub's own answer to this is to force engineers to use a /slash command to post a summary of the week's updates. Clunky, but it works.
- dylan604 4y ago
  but if you're actively working on a problem you're in it, and the comments probably make sense. coming in from outside via web search or shared link, yeah, it's probably hard to jump in without full context of the lead up to it. once the problem is fixed, do you really care that much after the fact and have moved on?
  - grepfru_it 4y ago
    How do you manage historical context ?

Not sure I entirely follow what's going on here. Is there some context behind this that's useful to know?

by icepat 4y ago

Not sure I entirely follow what's going on here. Is there some context behind this that's useful to know?

If I understand correctly, this is a receiver — which means you can take your existing DD instrumentation in their format and have it translated to OTel using this code. It lowers the switching cost to get off of DD.

trynewideas 4y ago
And Datadog competitor Grafana sponsored it: https://twitter.com/boostchicken/status/1618692475845238784 https://twitter.com/boostchicken/status/1618692475845238784
- deleted 4y ago
  [deleted]
xen2xen1 4y ago
Protect the moat at all costs!
- benatkin 4y ago
  It's worse than that, they want security through obscurity too. They feel like someone is inappropriately tinkering with the agent they want customers to install. It's open source: https://github.com/DataDog/datadog-agent https://github.com/DataDog/datadog-agent ...but the downloads are behind a login: https://github.com/DataDog/datadog-agent#datadog-agent https://github.com/DataDog/datadog-agent#datadog-agent
  - judge2020 4y ago
    Not really - the script it gives you in-app is DD_API_KEY= DD_SITE="datadoghq.com" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script_agent7.sh)" They tell you to sign in because installing without a key leads to non-working agents and support tickets. As for the libraries themselves, they're all on the regular package manager for that language, eg. pypi.
    - benatkin 4y ago
      Ah, that's good to know, thanks. I didn't have that link, but I can see it doesn't have an API key, so they can't change the download based on who's downloading it. That's good for transparency. Ditto the package managers. I see some Linux package manager public keys in there too. Edit: I missed the environment variable before "curl". The .sh is downloaded without the API key but the rest could be done using the API key, since it is passed to the script.
benatkin 4y ago
They want to keep their agent secret. Secret Agent Man...
- angstrom 4y ago
  They've given you a pull number and taken 'way your name

xen2xen1 4y ago

Wondered the same. Assume this is a threat to their business? An open source competitor can do something cool so they want to kill the functionality?

hosh 4y ago
We use DD, and we have to use Otel for our Elixir apps to ingest spans into DD because there are no DD SDKs for Elixir or Erlang. From the perspective of a customer, I can tell you that DD already has quite a bit of a moat. Their main competitive advantage, and what got us into using it, is being able to correlate data across APM, custom metrics, and logging through the use of tagging. They then densely link data together across the platform. There is also a built-in Jupyter-style notebooks. By correlating data like that, you get more value out of ingesting as much data into DD as you can. There are some additional services we're not using, such as auto-correlation with ML (and alerting for anomaly detection), and security monitoring that also looks across the entire platform using their ML tech. Like AWS/GCP/Azure, it can get expensive, quite fast, using on-demand pricing, so there are negotiated annual contracts. Right now, our team is small, and to replicate the functionality we do use, using self-hosted open-source tooling, we might as well hire another engineer for just setting up and maintaining such a platform. I get it that, you want to defend the moat and that eroding the little things can lead to eroding the big things. As I see it though, if you need those correlations, you'll need a certain scale and team size before it makes sense to build out something like that for yourself.
- aeyes 4y ago
  > negotiated annual contract Maybe we are too small but Datadog is one of the few vendors which we haven't been able to negotiate down in years. The price has always been whats on the website. I honestly don't even mind, with some vendors it feels like you are on a basar and they always tell you that their final discountns had to get approval by the CEO.
  - hosh 4y ago
    I don't know if we got a discount for an annual contract, or if the annual contract pricing was published on the website. But we were able to work out things like, using seasonal pricing because our traffic is seasonal.

ren_engineer 4y ago

OpenTelemetry basically allows for vendor neutral instrumentation for your metrics, logs, and traces. It makes it easy to switch to a cheaper or better service with minimal lock-in, which is obviously bad for DataDog and other companies business model

MuffinFlavored 4y ago
> It makes it easy to switch to a cheaper or better service with minimal lock-in Got any examples? I tried running my own "stack" for a project I wanted alerting on. I landed on Jaeger all-in-one (wasted time on Zipkin, the UI just was nowhere near as good as it ought to be) Docker container in docker-compose with COLLECTOR_OTLP_ENABLED.
- rad_gruchalski 4y ago
  I used the collector to proxy otel traces to jaeger zipkin for uniform istio tracing.
- xcasperx 4y ago
  Check out telemetryhub.com (admittedly I work here). We offer a free trial and don't charge per a seat.
  - vertis 4y ago
    I understand the need to charge money for things, but the $150 per month pretty much rules out any bootstrapped startups.
    - MuffinFlavored 4y ago
      I wish I knew the right answer between "spin up a cloud VPS Linux server and run your own with Docker compose" and pay $150/mo for something like DataDog
- m-o11y 4y ago
  If you're looking for something a bit more "few-clicks-and-you-are-up-and-running", check out OpsVerse ObserveNow: https://opsverse.io/observenow-observability/ https://opsverse.io/observenow-observability/ .. Entirely powered by OSS tools, ingestion-driven pricing, and without the hassle of managing the stack and scaling up.

grepfru_it 4y ago

This comment highlights my problem with singular project tracking issues. Once they get so massive like this, people suddenly "can't follow along". Though not specific to GitHub but in this case the ui hampers readability and the comments are very much in-the-moment. This then leaves me to ask the questions, (a) should we stop writing such mammoth comments in issues or (b) should we leverage generative AI to summarize a weekly or monthly timeline of issue interaction. What is the point of having historical issues if it's too complex to follow along? This was a question I regularly asked at GitHub that usually resulted in blank stares. GitHub's own answer to this is to force engineers to use a /slash command to post a summary of the week's updates. Clunky, but it works.

dylan604 4y ago
but if you're actively working on a problem you're in it, and the comments probably make sense. coming in from outside via web search or shared link, yeah, it's probably hard to jump in without full context of the lead up to it. once the problem is fixed, do you really care that much after the fact and have moved on?
- grepfru_it 4y ago
  How do you manage historical context ?