abelanger

9 ms·

91.
by abelanger 2y ago
Nice! I haven't looked closely, but some initial questions/comments: 1. Are you ordering the jobs by any parameter? I don't see an ORDER BY in this clause: https://github.com/oneapplab/lq/blob/8
92.
by abelanger 2y ago
> and DBOS lets you write worklflows as code instead of explicit DAGs To clarify, Hatchet supports both DAGs and workflows as code: see https://docs.hatchet.run/home/child-spawning and https://docs.hatche
93.
by abelanger 2y ago
We use gRPC on our workers. All API specs can be found here: https://github.com/hatchet-dev/hatchet/tree/main/api-contrac... However, the SDKs are very tightly integrated with the runtime in each languag
94.
by abelanger 2y ago
Really appreciate the candid feedback, and glad to hear you like the product. We ran a broken links checker against our docs, but it's possible we missed something. Is there anywhere you're seeing a broken link? Re SDK specs -- I
95.
by abelanger 2y ago
Re DBOS - yep, this is exactly what the child spawning feature is meant for: https://docs.hatchet.run/home/child-spawning The core idea being that you write the "parent" task as a durable task, and you invoke
96.
by abelanger 2y ago
Yeah, part of this rewrite was separating our monitoring tables from all of our queue tables to avoid problems like table bloat. At one point we considered partitioning on the status of a queue item (basically active | inactive) and aggress
97.
by abelanger 2y ago
Thanks! There are SDKs for Python, Typescript and Go. We've gotten a lot of requests for other SDKs which we're tracking here: https://github.com/hatchet-dev/hatchet/discussions/436
98.
by abelanger 2y ago
I'm not sure of the exact threshold, but the pathological case seemed to be (1) many tasks in the backlog, (2) many workers, (3) workers long-polling the task tables at approximately the same time. This would consistently lead to very
99.
by abelanger 2y ago
Yep, durable execution-wise we're targeting a very similar use-case with a very different philosophy on whether the orchestrator (the part of the durable execution engine which invokes tasks) should run in-process or as a separate serv
100.
Show HN: Hatchet v1 – A task orchestration platform built on Postgres(github.com)
240 points by abelanger 2y ago|74 comments
101.
The tech fantasy that powers AI is running on fumes(nytimes.com)
9 points by abelanger 2y ago|2 comments
102.
Vercel's Fluid compute is now GA(vercel.com)
2 points by abelanger 2y ago|0 comments
103.
by abelanger 2y ago
You can get around some of this coordination by doing more work locally, in-process -- but you're risking the availability of your primary API. The work that you're doing as a background job or workflow may: 1. Be unbounded -- 1 A
104.
by abelanger 2y ago
Definitely agree that the dev experience is better with a library, particularly for lightweight and low-volume tasks (Hatchet is also moving in the same direction, we'll be releasing library-only mode this month). And I really like the
105.
by abelanger 2y ago
Disclaimer: I'm a co-founder of Hatchet ( https://github.com/hatchet-dev/hatchet ), which is a Postgres-backed task queue that supports durable execution. > Because a step transition is just a Postgres write (~1m
106.
Use Postgres for your events table(docs.hatchet.run)
6 points by abelanger 2y ago|0 comments
107.
by abelanger 2y ago
I've found that I rely most heavily on LLMs when: 1. I'm developing a utility package that's easily testable and I'm certain of the interface. I'll write the interface for the package in my editor, then ask an LLM t
108.
ChatGPT Spoiled My Semester(ben.page)
6 points by abelanger 2y ago|1 comments
109.
Email Marketing to Developers(ronakganatra.com)
1 points by abelanger 2y ago|0 comments
110.
Migrating to sqlc and pgx for better performance(docs.hatchet.run)
3 points by abelanger 2y ago|0 comments
111.
Volcano: A cloud-native batch processing system(github.com)
3 points by abelanger 2y ago|0 comments
112.
RedCode: Risky Code Execution and Generation Benchmark for Code Agents(arxiv.org)
2 points by abelanger 2y ago|0 comments
113.
Frogs kick back against lethal fungus(knowablemagazine.org)
3 points by abelanger 2y ago|0 comments
114.
by abelanger 2y ago
It's not clear how many records were generated in the sample dataset for these benchmarks. The methodology [1] and Github repo [2] show how to seed tables of 1k records, but it's not clear what size the results on the website were
115.
When the World Feels Messy, I Turn to PowerWash Simulator(nytimes.com)
2 points by abelanger 2y ago|0 comments
116.
Hatchet (YC W24) is hiring a developer-focused product engineer(ycombinator.com)
1 points by abelanger 2y ago
117.
by abelanger 2y ago
Hatchet ( https://hatchet.run ) | New York City (IN-PERSON) | Full-time We're hiring a founding engineer to help us with development on our open-source, distributed task queue: https://github.com/hatchet-dev&#
118.
by abelanger 2y ago
Yep, we have support for "retry 5 times, then give up" ( https://docs.hatchet.run/home/features/retries/simple ) and "text me" - you can use either our built-in alerting features which integ
119.
by abelanger 2y ago
You're right, if all you need is a queue with a small number of workers connected at low volume, you don't need Hatchet or any other managed queue - you can get some pretty performant behavior with something like: https:/&#x
120.
by abelanger 2y ago
Thanks! Yes, our recommended approach is to write a parent workflow which calls child workflows registered on a different worker. We have users who are managing a set of Python functions from a Typescript backend with this approach. It&#x27

More ›