8 ms·
Pydantic Logfire
- OutOfHere 2y agoI would rather use something five times worse or simpler that I could self-host than be forced to rely on some service.
- LtWorf 2y agoBut VC needs to make money.
- OutOfHere 2y agoI believe it is in the best interest of VCs to allow free self-hosting because this is what will encourage developers to first experiment with the software for their personal projects. If developers find that they're happy with it, these same developers will later happily encourage some of their commercial clients and employers to use the paid service.
- LtWorf 2y agoThey want to lock in people.
- serjester 2y agoI love pydantic but I really have to wonder why they chose this route. There's already a ton of companies that do this really well and I'm trying to figure out how their solution is any different. The llm integration seems promising but if you care about LLM observability you probably also care about evals, guardrails and a million other things that are very specific to LLM's. Is it possible to build all this under one platform? I do hope I'm wrong for the sake of pydantic-core.
- thenipper 2y agoAgreed. I use pydantic daily and I'm in the position to try this out... but why? It really seems to be a weird divergence from their core product.
- Yenrabbit 2y agoThey also have a UI framework (fastui). They have funding and devs looking to be useful, I think it's fine that they get to try out new things that work well with their core product.
- deleted 2y ago[deleted]
- wferrell 2y agoWho do you like who does this well? I think Pydantic is great software and so I am inclined to see if this too will be great software.
- clintonb 2y agoMy last company used Splunk and SignalFx. They were fine, but I wasn't necessarily in the weeds with configuration and usage. The Observability Team made it easy to use. My current company, a much small startup, primarily uses Datadog and we are starting to better integrage Honeycomb. We mostly abandoned Google Cloud Monitoring because the UI/UX are not that great. Honeycomb is a paradigm, so took some time for us/me to understand. It's growing on us. Despite the ability to completely blow up our bill (which is now better controlled), Datadog is a good product that lets us quickly answer questions when things go wrong. It's not perfect, but it's the best we have right now. The UI is intuitive, and facilitates more discovery (esp. for metrics and their attributes).
- cipherself 2y agoI am kinda confused by your comment, OP is about tracing as far as I understood, but you're referencing Google Cloud Monitoring (whereas the comparable thing would be Google Cloud Trace) and then again (esp. for metrics and their attributes). but OP isn't about metrics at all rather traces.
- gazpacho 2y agoHi, I'm one of the developers of Logfire. We do support metrics! Our vision is a single pane of glass observability where you can easily go from metrics <-> traces both via correlation and by creating metrics/dashboards from traces. We also support logging as an integrated concept into tracing (you can emit events without a duration that are just like a log but carry context of where they were emitted from relative to a trace).
- clintonb 2y agoGoogle Cloud Monitoring (https://console.cloud.google.com/monitoring https://console.cloud.google.com/monitoring), perhaps not what you call it, supports the big three signals—logs, traces, metrics.
- scolvin 2y agoPydantic creator here. I understand why this might be your reaction, but let me just share my thoughts: Can we build all the LLM things people want? Yes I think so, early feedback is that Logfire is already much more comprehensive than LLM specific solutions. How is our solution any different? AFAIK: * no one else offers opinionated wrapper for OTel to make it nicer to use, but with all the advantages * no one else does auto-tracing (basically profiling) using import hooks the way we do * no one else has dedicated instrumentation for OpenAI or Pydantic * no one else provides metadata about python objects the way we do so we can render a semi-faithful repr of complex objects (Pydantic models, dataclasses, dataframes etc.) in the web UI * no one else (except Sentry, who are doing something different) makes it really easy to start using it * no one else lets you query observability data with SQL In my mind, the obvious alternative would be to do the thing most OSS companies do, and build "Pydantic Cloud", then start adding features to that instead of the open source package. I didn't want to do that, and I don't think our users would like that either. In the end I decided to build Logfire for two reasons: 1. I wanted it, and have wanted it for years 2. I think building a commercial product like this, then using OSS to spread the word and drive adoption is a really exciting way to incentivize us to make the open source as good as possible — good as-in permissive and good as-in well maintained. And it means that our commercial success is not in tension with the adoption of our open source, which has been a recurring issue for companies trying to capitalize on their open source brand.
- hakanderyal 2y agoUseful product, great marketing site. This is obviously a labor of love. Keep it up! Sadly I've moved on from Python world. Funny anecdote, using Pydantic everywhere to improve maintainability made me realize I'm fighting an uphill battle with Python and I should move to a statically typed language, so I switched to C#. Thanks for your work.
- sirsinsalot 2y agoHonestly, after spending the last 10 years developing in Python, I'm considering doing something similar. I've lived the problems. They suck for engineering software with longevity.
- zbentley 2y agoThis smells a bit like someone is using the brand of a famous open-source tool they built to promote their startup. If so, that's lame; if the new business is unrelated to your open source project, branding it as Pydantic Logfire seems a bit disingenuous. I think there's a less-misleading way to use open-source reputation for business credibility, though: "so-and-so business, by the creators of so-and-so project". Knowing that respected and skilled folks are working on a business is great! It's a fine distinction, but I think it matters.
- scolvin 2y agoIf anyone is mislead about whether Pydantic is Logfire or visa-versa, I'll eat my hat. Other than that, see my answer to the other comment.
- nsteel 2y agoFor the record, I don't agree with the grandparent comment at all. None of this is "lame". It's a path to sustainable OSS and it's more obvious than ever that we need these pathways. But to be fair, the branding here is a little weird. If you were living under a rock and hadn't heard of pydantic then the website reads like pydantic is the company and logfire is a product of said company. That's fine. But then you've also got a product called pydantic. Or is it now called pydantic pydantic? I realise it's kind of an extension to pydantic but the AI focus doesn't make it feel that way and so I think that's where the GP is coming from. Sorry to nitpick, I hope it does well.
- esafak 2y agoThis is just how you brand when you have one product. If he made up a new brand to subsume both products nobody would recognize it, so he wisely decided to recycle the existing brand; familiar, and liked. Google was originally just the search engine, without any qualifier. Now it's Google Search. In the same way, pydantic can become "Pydantic Validate" or something.
- scolvin 2y agoI think that's fair. It's a hard problem. We could have called the company a completely new name and everyone would have been confused, or just called us "pydantic". I think Pydantic being the company name, and a standalone entity, and there being other products is fairly common. I think if Logfire is successful, it will end up just being known as "Logfire".
- MapleWalnut 2y agoIt'll be hard to position this against Sentry. Sentry's a joy to use and their performance product is so helpful in debugging performance issues
- Lucasoato 2y agoOne of the Sentry inconvenience is self-hosting: it relies on so many services it can be very complicated to maintain.
- esafak 2y agoDoes the self-hosted version have all the features?
- easton 2y agoyes, every feature (but billing, iirc) is in self hosted. which means you need all of their backend dependencies (kafka, zookeeper, other things that are probably easier to manage than those two)
- whit537 2y agoWe've started working on addressing this, fwiw: https://github.com/getsentry/team-ospo/issues/232 https://github.com/getsentry/team-ospo/issues/232
- mdaniel 2y agoI draw ones attention to the actual Open Source glitchtip which has a much more sane deployment, akin to the good old days of Sentry before they got Big Data-itis: https://gitlab.com/glitchtip/glitchtip-backend/-/blob/v4.0.8/docker-compose.yml https://gitlab.com/glitchtip/glitchtip-backend/-/blob/v4.0.8... (or its helm version, similarly not JFC https://gitlab.com/glitchtip/glitchtip-helm-chart/-/tree/61c97590fc761301279b12150a64c130e4627eb3 https://gitlab.com/glitchtip/glitchtip-helm-chart/-/tree/61c... )
- 2y ago
- drchaim 2y agoThe ideal business model (from a user's point of view) for successful libraries like Pydantic, Ruff, etc., is like a foundation or sponsoring. But (almost) nobody pays for them (me included), so their creators have raised VC money, citing: 'We have xxM users and yyyyyM downloads per month.' Not to blame library authors here, they are trying their thing, which is indeed admirable. It's just that open source is difficult to sustain. Wishing them all the best
- deleted 2y ago[deleted]
- LtWorf 2y agoDownloads is a terrible metric. Easy to fake to increase them, while projects with many skilled users result with 0 since they will use distributions or caches instead of wasting time re-downloading the same thing over and over.
- jmduke 2y agoOP: as someone who runs a Python business and is a happy user of Pydantic (via Django-ninja) — something I would want this splash page to convey, at least implicitly, is “why use this over Sentry?” As far as I can tell, the answer is: an OpenAI binding and stronger Otel support. (That being said, I am a sucker for all new tooling in this genre and am excited to play around with it!)
- jmduke 2y agoResponding to my own post: this comment (https://news.ycombinator.com/item?id=40214695 https://news.ycombinator.com/item?id=40214695) I think is a very compelling answer. I love the design language of the splash page, but I think the copy could be improved by a little less "here is why performance insights are useful" (which I think most people are kind of already on board with) and more "here is why our performance insights are _particularly_ more useful than the competition".
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- cqqxo4zV46cp 2y agoAlso, I’m pretty sure that Sentry has OpenAI bindings now?
- megaman821 2y agoDoes this do logging or just traces? More to the point, is it economically to store tens of millions of log lines a month?
- cipherself 2y agoFrom the screenshots, it seems very much to be about tracing.
- dmontagu 2y agoI'm one of the Pydantic/Logfire devs — we are actually aiming to unify the ideas of logging and tracing a bit more than seems common in most such tools — our logfire SDK actually emits spans for all of the traditional logging statements rather than relying on OTel logging infrastructure (which seems to be kind of a legacy-oriented afterthought relative to the tracing APIs/infrastructure). So, yes, I would say we do support logging (certainly on a conceptual level), though the point is to try to move toward a world where as much as possible is done with spans, if only as an implementation detail. We also have explicit integrations with popular logging packages (see the bottom of https://docs.pydantic.dev/logfire/integrations/#opentelemetry-integrations https://docs.pydantic.dev/logfire/integrations/#opentelemetr...), so if you use the standard library `logging` module, `structlog`, or `loguru`, you should be able to set up your logs to ship to Logfire with minimal modifications to your existing code. While we have not announced our pricing yet, I am confident that tens of millions of log lines a month will absolutely be economical to store. (Also, I'll just add that we also support metrics, though our metrics support is not as well-integrated yet as tracing. But soon!)
- laborcontract 2y agoI was just looking for a logging service. Really enjoy Pydantic so this looks like it may be a delight. Try not to surprise us on pricing, though.
- barapa 2y agoI'm already using it and I think it's great. Well done Samuel and team!
- logfire_q 2y agocan i self host logfire? based on the “logfire auth” in the docs i’m thinking no?
- mariocesar 2y agoIt is a great tool! Congratulations on launching it! Could you clarify if it's possible to self-host the service?
- scolvin 2y agoWe'll offer self hosting for enterprise. I want the entry level for "enterprise" to be much lower than some other companies in this space.
- cqqxo4zV46cp 2y agoFor your consideration: I work in an industry which commonly involves data residency requirements / restrictions. Contrary to the belief seemingly held by every VC-funded cookie cutter-business-model SaaS company out there, this doesn’t make us “enterprise-y”, nor does it mean that we are flush with cash to splash on “contact sales”-tier Enterprise SaaS subscriptions. This ham-fisted market segmentation is a constant frustration for me. Please, please, please keep this in mind.
- mulmboy 2y agoIs self hosting possible? Can't see any mention in the docs
- ryanisnan 2y agoIt looks like it's on the roadmap, but my interpretation of the project would be that this is would be behind a licensed paywall. https://docs.pydantic.dev/logfire/roadmap/#on-premise-deployment https://docs.pydantic.dev/logfire/roadmap/#on-premise-deploy...
- gazpacho 2y agoI’d like to address a lot of the questions about self hosting: we do not currently have any way to self host, but we do plan on figuring that out eventually. Offering self hosting introduces challenges around updates, support, etc that we don’t want to slow us down while we refine the product to its initial state. We are cognizant of data restrictions and other factors that would require self hosting. That said, we don’t at this time plan on completely open sourcing the entire product. It would be more of a licensing setup. Of course this is all subject to change in the future based on feedback and product direction. The most important thing is that you share with us WHY you want to self host. Is it compliance? What does that compliance mean (min/max data retention, right to be forgotten, data at rest needs to be geographically located or you have to own it, etc)? This will help us build the “right” kind of features in this area so that it works for you and your company.
- kbumsik 2y ago> WHY you want to self host. In my case, I am hesitate to add another cloud service in the Sub-Processor list to meet GDPR. Actually, if it is an Otel wrapper, I don't think you don't need to offer self-hosting the server at all. Making easy to integrate with other Otel ecosystems would be enough. > It would be more of a licensing setup. At least this is much better than FOSS first than switching to a non-FOSS license like what happened to hashicorp and redis.
- feydaykyn 2y agoGDPR on our side too, we need to reduce subcontractors as much as possible to handle all the requirements you listed. Also, we have to renegotiate with our own customers every time we want to had a new subcontractor, so we avoid doing so at all costs. I like the prefect.io approach to have the control plane on the external provider, with the data and workers being run on the customer infrastructure. It seems fair for both sides: - as the subcontractor, you keep trace of the real usage, without having to handle end user data which is a pain to manage, so you don't have to offer outrageous license pricing to compensate for being stolen (looking at you Grafana) . - as the company, you comply with gdpr, while alleviate the operating costs, and also supporting the companies providing the tooling you need.