4 ms·
I work closely with the team responsible for a large, self-hosted GitHub Enterprise instance. This is good advice for clients/consumers of GH data, but it can v
by iamEAP 17d ago
I work closely with the team responsible for a large, self-hosted GitHub Enterprise instance. This is good advice for clients/consumers of GH data, but it can very easily lead to a lot of strain on the server-side. It’s not obvious what fields that you request are simple reads from tables or actually end up invoking git under the hood.
You could argue the rate limit guards should better reflect that, but that’s just not the reality of the system. Likely speaks to a lot of stability issues GitHub has been facing lately.
- enormousness 17d agoIsn't it kind of part and parcel of any GraphQL deployment to update it efficiently?
- kccqzy 17d agoThat is true for almost any GraphQL backend not just GHE.
- dieselgate 17d ago> It’s not obvious what fields that you request are simple reads from tables or actually end up invoking git under the hood. I'm not familiar with graphql but what would make something "invoke git", is it a technical thing or hyperbole?
- iamEAP 17d agoMy over-simplified explanation: the graphql server will map a field on a request to a resolver. The resolver can execute whatever code it needs to return the value, up to and including calls into libgit. So asking for the title of a PR might just be an extra column selected on a DB query. But calculating mergability status of that PR might be something else entirely.
- bob1029 17d agoAll of the queries my agents use select fields like issue title, body, labels, createdAt, updatedAt, etc. That's about it. I would hope that stuff is cached and efficient to read. I do not think GraphQL is a good way to interact with git. Running git on the CLI is the best way to interact with git.
- 0xbadcafebee 17d agoIt's definitely one source of GitHub's issues. They never implemented limits properly and their codebase is likely decades of cruft and impossible to retrofit quickly (to say nothing of their horrific mysql clusters). If you want to scale and survive it, you have to build the thing right the first time.
- bombcar 17d agoYou have to be so uptight about performance when you're small that people look at you funny, or your shit is going to fall right over under webscale (ai scale?) load.
- bbor 17d agoWell 1. GitHub is the broken & bad one, and 2. isn't that kind of on them? If they're getting a ton of requests that read data awkwardly, the fix seems like the quintessential data engineering task.
- jeremyjh 17d agoYou sound like someone who has not been informed about the problem GitHub is facing. No one else is facing anything close to it, so there is no meaning to "the bad one". https://cursor.com/blog/git-at-any-scale https://cursor.com/blog/git-at-any-scale
- spockz 17d agoI put my ssh credential behind a tpm and now I notice that every time I switch a chat in ChatGPT it tries to see the current remote status of the branch. That also doesn’t help with load. So besides what agents are doing themselves the AI companies also have a responsibility of being good citizens.
- jeremyjh 17d agoVS Code does this as well - or maybe the GitHub plugin does it. Either way, GitHub owns it.
- RomanKornev 17d ago> lot of strain on the server-side That's why GitHub assigns "points" to each requests and deducts based on the data shape you request. For simple requests it's 1-to-1, but can quickly balloon