27 ms·

How and why GraphQL will influence the Sourcehut alpha

awinter-py 6y ago
> The value-add is difficult to understand yup
z3t4 6y ago
Im a Sourcehut user and all I care about is that I can host my public Mercurial repositories, and that web frontend is somewhat usable (it currently is). I dont care what languages is ised to build it nor do i care about apis and other features. If i where to chose I would like an api that i could use with curl, eg. Dont complicate things.
- alexchamberlain 6y ago
  I think this is simply a technical blog post - it could be about any site. It's simply an insight to the way another engineer is thinking, so you can challenge your own.
  - z3t4 6y ago
    It was not a comment about the blog post! It was just my thought as a user of the product. He should probably spend his time marketing as he already got a working product, rather then rewriting stuff.
xvilka 6y ago
Does this mean that SourceHut will become completely written in Go? Amazing news if so!
- CameronNemo 6y ago
  Why? I am currently converting a Flask/SQLAlchemy application to Go and gRPC. Go has not been that much of a win from my perspective. Lots of rough edges and missing functionality in libraries available, a lackluster type system that does enough to get in the way but not enough to properly express programmatic intention, and an atypical non-ideal error handling model have left me with little reason to champion Go. Not that it is a terrible language, I just do not see the value add compared to even Python (let alone Kotlin or Rust).
  - philosopher1234 6y ago
    Could you name some specific rough edges and missing functionality that have frustrated you? I'm a big fan of Go and have a good experience with it, so I'd be very curious to hear from someone with an opposite experience.
    - CameronNemo 6y ago
      On several occasions I have gotten panics (segfaults), many times without using CGo. This is not something I would expect from a "high level" language that has a garbage collector. The constant use of generated code is another real pain point, particularly when I am writing business logic that needs to operate on generated types for which there is no interface available that does what I need (and why would there be? how would the library/codegen-tool author know all the permutations of business logic that might be out there?). The sql library has some fairly annoying documentation, e.g. > Scan implements the Scanner interface. https://golang.org/pkg/database/sql/#NullBool.Scan https://golang.org/pkg/database/sql/#NullBool.Scan > Scanner is an interface used by Scan. https://golang.org/pkg/database/sql/#Scanner https://golang.org/pkg/database/sql/#Scanner There is only really one concrete example of how to use Scan on the godoc page. The introspection capabilities (reflect package) are quite obtuse and error prone (basically requiring 100% test coverage because you lose nearly the entire type system), yet absolutely critical to implementing anything notably complex with Go.
      - philosopher1234 6y ago
        >The constant use of generated code is another real pain point, particularly when I am writing business logic that needs to operate on generated types for which there is no interface available that does what I need (and why would there be? how would the library/codegen-tool author know all the permutations of business logic that might be out there?). Not sure what you mean here. Is there a particular codegen tool you found lacking? >The introspection capabilities (reflect package) are quite obtuse and error prone (basically requiring 100% test coverage because you lose nearly the entire type system), yet absolutely critical to implementing anything notably complex with Go. Ah, the lack of generics. I haven't really written any particularly large projects, where have you had to use reflection when working with go in your projects? I hope my questions don't come across as dismissive. I think the typical go response to your complaints are "it wont come up" (a YAGNI variant), so I'm always interested to hear about the cases where that argument fails.
ianamartin 6y ago
Well, that's too bad. I always thought this was a cool project. But if you can't dev your way into decent performance for a small alpha project using python/flask/sql, I don't think your tools are the problem. And I guarantee that a graphql isn't the solution. So, I mean, good luck.
- ghoshbishakh 6y ago
  what do you feel the problem is?
  - ianamartin 6y ago
    There are lots of production sites that serve 10,000x as much traffic as sourcehut that are built on Python/flask/sqlalchemy serving RESTful APIs. If you can't make that combination work well, there's another place to look for problems besides your tool kit. You might need to ask yourself if you really understand the tools you're trying to use. But like I said, this has always been a very cool project. My "good luck" was meant more as actual good luck than a Morgan Freeman You're-trying-to-blackmail-batman kind of good luck.
    - deleted 6y ago
      [deleted]
    - xrisk 6y ago
      What are some production websites that run Python stacks? I’m curious. On a tangential note: if anybody has blog posts on scaling Flask/SQLAlchemy or Django stacks I would appreciate it.
      - searchableguy 6y ago
        Dropbox is a notable one born here.
        procinct 6y ago
RcouF1uZ4gsC 6y ago
> My work on SourceHut has, on the whole, really soured my opinion of Python as a serious language for large projects. A lot of the things thank make Python great for small projects, really bite you on a large or long-lived project. For me, the two biggest are lack of types and indentation for scoping. It is really easy to mess up white space during an edit or refactor. In many languages you would just reformat. In python, you have to be careful that a statement suddenly did not end up outside or inside an if block.
- frou_dh 6y ago
  Python has a better static typesystem than Go does: https://mypy.readthedocs.io/en/stable/kinds_of_types.html#union-types https://mypy.readthedocs.io/en/stable/kinds_of_types.html#un... https://mypy.readthedocs.io/en/stable/kinds_of_types.html#optional-types-and-the-none-type https://mypy.readthedocs.io/en/stable/kinds_of_types.html#op... https://mypy.readthedocs.io/en/stable/generics.html https://mypy.readthedocs.io/en/stable/generics.html https://mypy.readthedocs.io/en/stable/protocols.html https://mypy.readthedocs.io/en/stable/protocols.html
  - ianamartin 6y ago
    Even though I like Python a lot, I clicked on a reply to this comment because I was absolutely certain I was going to have to disagree with you. That just didn't seem possible. But . . . I was wrong. That third link with generics makes me cry when I think about the go code I've written.
    - searchableguy 6y ago
      Are those tears result of overflowing with joy or sadness?
      - ianamartin 6y ago
        Can it be both?
    - 6y ago
tmpz22 6y ago
Big OOF. Even the flow of the article is muddied to the point where it takes significant effort to dig out the main points for OP's migration from REST to GraphQL. What I got from it: > The system would become more stable with the benefit of static typing, and more scalable with a faster and lighter-weight implementation. OK static typing I'm with you so far. Faster and lighter weight? I'm not so sure, sounds like you're having troubles with Flask and SQLAlchemy completely unrelated to REST. All production graphQL implementations I've seen are very heavy when they add in authentication and more advanced query capabilities. Is this REALLY so superior to REST? > Another (potential) advantage of GraphQL is the ability to compose many different APIs into a single, federated GraphQL schema. I guess the discoverability of GraphQL is better, but 90% of APIs on the internet prove that large REST APIs are very effective and achieve the same thing. > I also mentioned earlier that I am unsatisfied with the Python/Flask/SQLAlchemy design that underlies most of SourceHut’s implementation. The performance characteristics of this design are rather poor, and I have limited options for improvement. The reliability is also not something I am especially confident in. This is where you completely lose me. It's fine if you hate ORMs, its fine if you hate the SQLAlchemy API, but you're blaming your hammer for the fact that you think you built a shoddy house. Going out and buying a hammer won't fix the fact that you're lining up your nails all wrong. > The GraphQL services are completely standalone, and it is possible to deploy them independently of the web application... > ...it is my intention to build out experimental replacement backends which are routed through GraphQL instead. I think these two captions go together, are you describing microservices? This can be achieved just fine with REST using simple load balancing strategies based on url routing or similar. > Almost all of SourceHut today is built with Python, Flask, and SQLAlchemy, which is great for quickly building a working prototype. This has been an effective approach to building a “good” service and understanding the constraints of the problem space. However, it’s become clear to me that this approach isn’t going to cut it in the long term, where the goal is not just “good”, but “excellent” This is a classic example of using a handful of annoying issues to justify an exciting large re-write that doesn't actually address the main issues you are having. If you are struggling with the SQLAlchemy library you will find alternative (and perhaps larger) struggles in all GraphQL implementations. Best of luck, this is a road I would not follow you on. Seriously though I wish you the best and hope your product succeeds despite this.
WhatIsDukkha 6y ago
I don't understand the attraction to Graphql. (I do understand it if maybe you actually want the things that gRPC or Thrift etc gives you) It seems like exactly the ORM solution/problem but even more abstract and less under control since it pushes the orm out to browser clients and the frontend devs. ORM suffer from being at beyond arms length from the query analyzer in the database server. https://en.wikipedia.org/wiki/Query_optimization https://en.wikipedia.org/wiki/Query_optimization A query optimizer that's been tuned over decades by pretty serious people. Bad queries, overfetching, sudden performance cliffs everywhere. Graphql actually adds another query language on top of the normal orm problem. (Maybe the answer is that graphql is so simple by design that it has no dark corners but that seems like a matter of mathematical proof that I haven't seen alluded to). Why is graphql not going to have exactly this problem as we see people actually start to work seriously with it? Four or five implementations in javascript, haskell and now go. From what I could see none of them were mentioning query optimization as an aspiration.
- searchableguy 6y ago
  That is upto the graphql framework and the consumers of them. Graphql is just a query language. You need to have data loader (batching) on the backend to avoid n+1 queries and some other similar stuff with cache to improve the performance. You also have cache and batching on the frontend usually. Apollo client (most popular graphql client in js) uses a normalized caching strategy (overkill and a pain). For rate/abuse limiting, graphql requires a completely different approach. It's either point based on the numbers of nodes or edges you request so you can calculate the burden of the query before you execute it or deep introspection to avoid crashing your database. Query white listing is another option. There are few other pain points you need to implement when you scale up. So yeah defo not needed if it's only a small project.
  - kabes 6y ago
    "You have to calculate the burden of the query before you execute it so you don't end up crashing your database." This sounds like disaster waiting to happen.
tleb_ 6y ago
I think this article misses the explanation of why GraphQL over REST. I usually don't like "x versus y" articles but here both have been tested on SourceHut, the hindsight should probably appear as useful. Thanks Drew and others for SourceHut.
- PostThisTooFast 6y ago
  What is SourceHut?
tannhaeuser 6y ago
Tbh I'd expected a little better than framing this question in a "REST vs GraphQL" discussion coming from sourcehut.org. If you control your backend, you can aggregate whatever payloads you please into a single HTTP response, and don't have to subscribe to a (naive) "RESTful" way where you have network roundtrips for every single "resource", a practice criticized by Roy Fielding (who coined the term "REST") himself and rooted in a mindset I'd call based more on cultural beliefs rather than engineering. That said, a recent discussion [1] convinced me there are practical benefits in using GraphQL if you're working with "modern" SPA frameworks, and your backend team can't always deliver the ever-changing interfaces you need so you're using a backend-for-fronted (an extra fronted-facing backend that wraps your actual backend) approach anyway, though it could be argued that organizational issues play a larger role here. [1]: https://news.ycombinator.com/item?id=23119810 https://news.ycombinator.com/item?id=23119810
- jamil7 6y ago
  I like GraphQL but if you're just serving a single SPA I wonder about all this busy work we still have to do. Why haven't we gone a step further and just abstracted all the networking and serialisation steps away and our models are synced for us in the background. Maybe the apollo team is heading in this direction but their offline story isn't great yet. Edit: I remember now that the Apollo team is made up of members of the former Meteor team which worked in a similar way using a client side database.
  - searchableguy 6y ago
    You can use rxdb for replication and offline support.
    - jamil7 6y ago
      Yeah good point, WatermelonDB, rxdb and pouchdb all fit this model. This to me feels like the future for web and mobile.
      - 6y ago
zapf 6y ago
I like my front end team to not write queries. I have seen too many front end developers write queries equivalent to "select * from users, t1, t2, ... tn, left outer join on users.id = t1.user_id... etc etc etc". I think that is just bullshit. I prefer to tell the frontend people - these are the endpoints that backend team provides, go live with it. If you really have issues, talk to them, and they might just add a new endpoint for you. If you let frontend developers dictate backend queries, all you will get is a big master cluster fuck. I am talking average joe developers that we usually find to work on startups.
- runawaybottle 6y ago
  I think Frontend has the right to take over certain domains of the backend. A very clear example of this is templating. Templating used to be a backend task, but it became very clear it was the frontend developers that needed power over this domain. Going over to a backend dev to make adjustments to the templates is an inefficiency that had to be solved, period. I agree with you that writing optimal queries or data modeling should not be shifted over to the frontend. With that said, there are basic aspects of this equation that can be factored out. There should be a layer where frontend can at the very least pick or mix the data they need. That doesn’t necessarily mean Graphql, but could also mean a simple middle stack layer where one can do this basic thing without being able to shoot themselves in the foot. There’s a responsible way to do this, and most likely will be an ongoing discussion.
cletus 6y ago
So I've now had the opportunity to use both GraphQL and protocol buffers ("protobufs" is the more typical term) professionally and I have some thoughts on this. 1. Protobufs use integer IDs for fields. GraphQL uses string names. IMHO this is a clear win for protobufs. Changing the name of a field of GraphQL is essentially impossible. Once a name is there it's there forever (eg mobile client versions are out there forever) so you're going to have to return null from it and create a new one. In protobufs, the name you see in code is nothing more than the client's bindings. Get a copy of the .proto file, change a name (but not the ID number) and recompile and everything will work. The wire format is the same; 2. People who talk about auto-generating GraphQL wrappers for Postgres database schemas (not the author of this post, to be clear, but it's common enough) are missing the point entirely. The whole point of GraphQL is to span heterogeneous and independent data sources; 3. Protobuf's notions of required vs optional fields was a design mistake that's now impossible to rectify without breaking changes. Maybe protobuf v3/gRPC did this. I'm honestly not sure. 4. Protobuf is just a wire format plus a way of generating language bindings for it. There are RPC extensions for this (Stubby internally at Google; gRPC externally and no they're not the same thing). GraphQL is a query language. I do think it's better than protobufs in this regard; 5. GraphQL fragments are one of these things that are probably a net positive but they aren't as good as they might appear. You will find in any large codebase that there are key fragments that if you change in any way you'll generate a massive recompile across hundreds or thousands of callsites. And if just one caller uses one of the fields in that fragment, you can't remove it; 6. GraphQL does kind of support union types (eg foo as Bar1, foo as Bar2) but it's awkward and my understanding is the mobile code generated is... less than ideal. Still, it's better than not having it. The protobuf equivalent is to have many optional submessages and there's no way to express that only one of them will be popualated; 7. Under the hood I believe the GraphQL query is stored on the server and identified by ID but the bindings for it are baked into the client. Perhaps this is just how FB uses it? It always struck me as somewhat awkward. Perhaps certain GraphQL queries are particularly large? I never bothered to look into the reason for this but given that the bindings are baked into the code it doesn't seem to gain you much; 8. GraphQL usage in Facebook is pervasive and it has first class support in iOS, Android and React. This is in stark contrast to protobufs where protobuf v2 in Google is probably there forever and protobuf v3/gRPC is largely for the outsiders. It's been several years now since I worked at Google but I would be shocked if this had changed or there was even an intention of changing it at this point; 9. The fact that you can do a GraphQL mutation and declare what fields are returned is, IMHO, very nice. It saves really awkward create/update then re-query hops. 10. This is probably a problem only for Google internally but another factor on top of protobuf version was the API version. Build artifacts were declared with this API version, which was actually a huge headache if you wanted to bring in dependencies, some of which were Java APIv1 and others Java APIv2. I don't really understand why you had to make this kind of decision in creating build artifacts. Again, maybe this has improved. I would be surprised however. Lastly, as for Sourcehut, I had a look at their home page. I'm honestly still not exactly sure what they are or what value they create. There are 3 pricing plans that provide access to all features so I'd have to dig in to find the difference (hint: I didn't). So it's hard for me to say if GraphQL is an appropriate choice for them. At least their pages loaded fast. That's a good sign.
jorams 6y ago
In my experience GraphQL can be much nicer to implement than REST, and it offers a good structure around things that many REST APIs implement in particular ways (like selecting which fields you want). The pain you'll experience depends heavily on your data model and the abuse potential that brings. I think the biggest problem with GraphQL is the JavaScript ecosystem around it, and all of its implicit context. It seems to be built entirely on specific servers and clients, instead of on the general concepts. Relay[1], a popular client-side library, adds all kinds of requirements in addition to the use of GraphQL. One of those is that until version 8, it required all mutation inputs and outputs to contain a "clientMutationId", which had to be round-tripped. It was an obvious hack for some client-side problem which added requirements to the backend. Somehow it had a specification written for it instead of being fixed before release. This hack is now in public APIs, like every single mutation in the GitHub API v4. GraphQL also includes "subscriptions", which are described incredibly vaguely and frankly underspecified. There are all kinds of libraries and frameworks that "support subscriptions", but in practice they mean they just support the websocket transport[2] created by Apollo GraphQL. If you just use it as a way to implement a well-structured API, and use the simplest tools possible to get you there, it's a pleasure to work with. [1]: https://relay.dev/ https://relay.dev/ [2]: https://github.com/apollographql/subscriptions-transport-ws https://github.com/apollographql/subscriptions-transport-ws
- kasbah 6y ago
  I don't think Relay is used that much outside of Facebook. The community seems to have settled on Apollo. Personally I find Apollo over-engineered. When I couldn't delete things from the cache, because of a bug, and was faced with digging into the complex code-base, I ended up just using straight JSON with a HTTP client/fetch and caching in a simple JS object. Other users of my API [1] just use straight HTTP with JSON as well. GraphQL clients seem to solve something we are not encountering. If urql [2] or gqless [3] work well when I try them I'd be up for changing my mind though. [1]: https://github.com/kitspace/partinfo https://github.com/kitspace/partinfo [2]: https://github.com/FormidableLabs/urql https://github.com/FormidableLabs/urql [3]: https://gqless.dev/ https://gqless.dev/
  - 6y ago
leadingthenet 6y ago
Might I suggest taking a look at FastAPI? It’s been a 10x+ improvement on Flask, in my experience.
say_it_as_it_is 6y ago
Where are the GraphQL lessons learned? The author hasn't even implemented a solution with it yet, but that hasn't stopped him from declaring it to the world. I don't find an announcement useful. Maybe GraphQL adopters aren't sharing their experiences with it in production because they're realizing its faults? People are quick to announce successes and very reluctant to own, let alone share, costly mistakes. Also, people change jobs so often that those who influence a roll-out won't even be around long enough for the post-mortem. GraphQL publicity is consequently positively biased. If the HN community were to follow up with posters who announced their use of GraphQL the last two years, maybe we can find out how things are going?
- dgellow 6y ago
  The author isn’t the one who shared the link on HN.
- gbear605 6y ago
  The reason this post was written was for users of Sourcehut, especially for people writing to its API. This post isn’t particularly relevant or explanatory for other audiences, but I don’t think it’s supposed to be so that’s fine.
  - say_it_as_it_is 6y ago
    I understand what you mean. That makes sense.
orf 6y ago
The author says that he has soured on Python for “serious, large projects”. While it’s clearly personal opinion, and that’s fair enough , I can’t help but think his choice of framework hasn’t helped him and has likely caused significant slowdown when delivering features. Looking through some of the code for Sourcehut, there’s an insane amount of boilerplate or otherwise redundant code[1]. The shared code library is a mini-framework, with custom email and validation components[2][3]. In the ‘main’ project we can see the views that power mailing lists and projects[4][5]. I’m totally biased, but I can’t help but think “why Flask, and why not Django” after seeing all of this. Most of the repeated view boilerplate would have gone ([1] could be like 20 lines), the author could have used Django rest framework to get a quality API with not much work (rather than building it yourself[6]) and the pluggable apps at the core of Django seem a perfect fit. I see this all the time with flasks projects. They start off small and light, and as long as they stay that way then Flask is a great choice. But they often don’t, and as the grow in complexity you end up re-inventing a framework like Django but worse whilst getting fatigued by “Python” being bad. 1. https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrht/blueprints/public.py https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrh... 2. https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/email.py https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/emai... 3. https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/validation.py https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/vali... 4. https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/blueprints/mailing_lists.py https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/bl... 5. https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/blueprints/projects.py https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/bl... 6. https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrht/blueprints/api/pastes.py https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrh...
- StavrosK 6y ago
  Exactly agreed. I basically only use Flask for things I want to explicitly be single-file these days. For anything larger, I reach for Django, because I know that if I need at least one thing from it (and I always need the ORM/migrations/admin), it will have been worth it. My current favorite way of building APIs is this Frankenstein's monster of Django/FastAPI, which actually works quite well so far: https://www.stavros.io/posts/fastapi-with-django/ https://www.stavros.io/posts/fastapi-with-django/ FastAPI is a much better way of writing APIs than DRF, I wish it were a Django library, but hopefully compatibility will improve as Django adds async support.
mehdix 6y ago
> With these, you can deploy a SourceHut instance with no frontend at all, using the GraphQL APIs exclusively. This reminds me of Kubernetes' design. You have an API server which is practially the Kubernetes from user's perspective. `kubectl` is just one out of possibily many clients that talk to this API. Edit: typos.
FpUser 6y ago
I've been using home grown RPC for my servers (POST + JSON) for HTTP or binary serialization for lower level access. Works like a charm for years. Never felt like missing anything.
ddevault 6y ago
Author here. Wow, there is a ton of "didn't RTFA" comments here. It seems like half of this thread saw GraphQL in the title, it knocked two gears into place in their head, and they started writing up their own little essay about how bad it is. I evaluated GraphQL twice before, and discarded it for many of the reasons brought up here. Even this time around, I give a rather lackluster review of it and mention that there are still many caveats. It's not magic, and I'm not entirely in love with it - but it's better than REST. Query optimization, scalability, authentication, and many other issues raised here were part of the research effort and I would not have moved forward with it if I did not feel that they were adequately addressed. Before assuming some limitation you have had with it in the past applies to sr.ht, I would recommend reading through the code: https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/api https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/api https://git.sr.ht/~sircmpwn/gql.sr.ht https://git.sr.ht/~sircmpwn/gql.sr.ht If you're curious for a more detailed run-down of my problems with REST, Python, Flask, and SQLAlchemy, I answered similar comments last night on the Lobsters thread: https://lobste.rs/s/me5emr/how_why_graphql_will_influence_sourcehut https://lobste.rs/s/me5emr/how_why_graphql_will_influence_so... I would also like to point out that the last time I thought a rewrite was in order, we got wlroots, which is now the most successful project in its class. Cheers.
AmericanChopper 6y ago
GraphQL as a query language is simply better than REST in most cases imo. REST has too much client side state, which not only has the potential to make things harder for clients to consume, but also has all the inconsistent states to handle where your consumer gets part way through a multiple-REST method workflow, and then bails. REST also absolutely sucks for mutating arrays. Really I just look at GraphQL as a nice RPC framework. The graph theory operations like field level resolvers are mostly useless. But if you treat each relationship as a node rather than each field, you can get it to work very nicely with a normalized data set. I haven’t found it hard to preserve join efficiency in the backend either, and it so far hasn’t forced me into redundant query operations. Just as long as you don’t use appsync. Really, don’t even bother.
- GordonS 6y ago
  > GraphQL as a query language is simply better than REST in most cases imo. REST has too much client side state, which not only has the potential to make things harder for clients to consume, but also has all the inconsistent states to handle where your consumer gets part way through a multiple-REST method workflow, and then bails. How much client state you maintain seems to me to be orthogonal to GraphQL/REST. Take your example or a multiple-REST workflow. I presume your point was that the workflow could be implemented by a single GraphQL query/mutation/whatever - but just the same, you can put as much code and logic as you like behind a REST call?
  - AmericanChopper 6y ago
    You could do that, but if you start creating endpoints for transactions rather than method -> resource endpoints, then you’re not really making a REST interface anymore. But even ignoring REST purity, I’d argue that GraphQL is better suited to that design pattern in general.
    - GordonS 6y ago
      When most people refer to REST APIs, they really mean "HTTP APIs". I really don't think we should reach for GraphQL just because of the ideological notion of daring to not adhere 100% to REST.
geitir 6y ago
One thing I've noticed is it seems redux and graphql are solving the same problem in an entirely different way
- iso-8859-1 6y ago
  Redux brands itself as a tool for managing state. How does GraphQL manage state?
  - geitir 6y ago
    The way I see it is Redux is a local normalized store which can then be used via the selector pattern to generate custom objects / views. With GraphQL you are requesting that derived data to be generated for you via the query language. Regarding managing state I don't see GraphQL helping with that at all.
crabmusket 6y ago
> Another (potential) advantage of GraphQL is the ability to compose many different APIs into a single, federated GraphQL schema. If anyone else can share experiences of this sort of problems and solution, I'd be really interested to hear it. I've written non-GQL APIs before that back onto other internal and external services; what am I missing?
- maddyboo 6y ago
  I think Drew is referring to composition in terms of API end-user code, not sr.ht code, e.g. making it possible for the user write a single GQL query that combines data from multiple sr.ht services.
  - crabmusket 6y ago
    Ah, I think that makes a little more sense. Thanks!
- GordonS 6y ago
  I've mentioned my (negative) experience elsewhere in this thread. It wasn't actually me doing the implementation, but the result was a horrendous, unmaintainable and slow code base. Ultimately, no part of the team benefited from the use of GraphQL instead of REST. To be fair, the same devs that built it using GraphQL would likely have made many of the same mistakes with a REST API, but I do feel it would at least have been easier to reason about the code.
pknopf 6y ago
I firmly believe GraphQL is a fad. It will never work, exposing the ORM to your clients. I'm curious, why don't we see more public-facing APIs using gRPC?
- iso-8859-1 6y ago
  gRPC is awkward on web because it assumes HTTP/2. The web is still half HTTP/1. Also, gRPC does almost everything, but not quite, it doesn't know how to synchronize state in general. It is a heavy weight choice considering that you may only need protobuf. And protobuf is not natively supported in browsers, web devs can't agree on serialization libs, they can only agree on the serialization format. And they didn't choose protobuf ;)
- zimpenfish 6y ago
  Currently working on a public-facing gRPC API and it's much more faffy than a REST API. Browsers don't talk gRPC which means you need to support gRPC-web endpoints. You probably need a gRPC aware proxy too and most of those have their own challenges. Some stuff doesn't like protobufs which means you need JSON transport which means protoc plugins. etc.etc. It's all just more work for (IMHO) little gain.
hn_throwaway_99 6y ago
Based on the top comments in this thread, I can see my previous attempts to dispel mistaken beliefs around graphql have failed, but I'll still try! 1. The biggest mistake GraphQL made was putting 'QL' in the name so people think it's a query language comparable to SQL. It's not: https://news.ycombinator.com/item?id=23120997 https://news.ycombinator.com/item?id=23120997 2. Some benefits of GraphQL over REST: https://news.ycombinator.com/item?id=23124862 https://news.ycombinator.com/item?id=23124862
- detaro 6y ago
  I suspect as long as https://graphql.org https://graphql.org says what it does, you're going to have a hard time with that fight...
deleted 6y ago
[deleted]
iooi 6y ago
I wanted to learn GraphQL recently and I wrote a small library to automagically generate GraphQL schemas from SQLAlchemy models. [1] It's inspired by Hasura, the schema is almost the same. It's not optimized at all, but it's a nice way to quickly get started with GraphQL and expose your existing models. [1] https://github.com/gzzo/graphql-sqlalchemy https://github.com/gzzo/graphql-sqlalchemy
hpen 6y ago
I tried GraphQL but my resolvers became extremely complicated. Am I missing something or is this just how it goes?
camgunz 6y ago
A lot of places have a hell of a time dealing with very nested graphql queries that effectively DoS your app servers, or a resolver that’s very slow. Caching is also an open question. But for the simplest systems, I’d hesitate to recommend graphql at this point.

> The value-add is difficult to understand yup

z3t4 6y ago

Im a Sourcehut user and all I care about is that I can host my public Mercurial repositories, and that web frontend is somewhat usable (it currently is). I dont care what languages is ised to build it nor do i care about apis and other features. If i where to chose I would like an api that i could use with curl, eg. Dont complicate things.

alexchamberlain 6y ago
I think this is simply a technical blog post - it could be about any site. It's simply an insight to the way another engineer is thinking, so you can challenge your own.
- z3t4 6y ago
  It was not a comment about the blog post! It was just my thought as a user of the product. He should probably spend his time marketing as he already got a working product, rather then rewriting stuff.

xvilka 6y ago

Does this mean that SourceHut will become completely written in Go? Amazing news if so!

CameronNemo 6y ago
Why? I am currently converting a Flask/SQLAlchemy application to Go and gRPC. Go has not been that much of a win from my perspective. Lots of rough edges and missing functionality in libraries available, a lackluster type system that does enough to get in the way but not enough to properly express programmatic intention, and an atypical non-ideal error handling model have left me with little reason to champion Go. Not that it is a terrible language, I just do not see the value add compared to even Python (let alone Kotlin or Rust).
- philosopher1234 6y ago
  Could you name some specific rough edges and missing functionality that have frustrated you? I'm a big fan of Go and have a good experience with it, so I'd be very curious to hear from someone with an opposite experience.
  - CameronNemo 6y ago
    On several occasions I have gotten panics (segfaults), many times without using CGo. This is not something I would expect from a "high level" language that has a garbage collector. The constant use of generated code is another real pain point, particularly when I am writing business logic that needs to operate on generated types for which there is no interface available that does what I need (and why would there be? how would the library/codegen-tool author know all the permutations of business logic that might be out there?). The sql library has some fairly annoying documentation, e.g. > Scan implements the Scanner interface. https://golang.org/pkg/database/sql/#NullBool.Scan https://golang.org/pkg/database/sql/#NullBool.Scan > Scanner is an interface used by Scan. https://golang.org/pkg/database/sql/#Scanner https://golang.org/pkg/database/sql/#Scanner There is only really one concrete example of how to use Scan on the godoc page. The introspection capabilities (reflect package) are quite obtuse and error prone (basically requiring 100% test coverage because you lose nearly the entire type system), yet absolutely critical to implementing anything notably complex with Go.
    - philosopher1234 6y ago
      >The constant use of generated code is another real pain point, particularly when I am writing business logic that needs to operate on generated types for which there is no interface available that does what I need (and why would there be? how would the library/codegen-tool author know all the permutations of business logic that might be out there?). Not sure what you mean here. Is there a particular codegen tool you found lacking? >The introspection capabilities (reflect package) are quite obtuse and error prone (basically requiring 100% test coverage because you lose nearly the entire type system), yet absolutely critical to implementing anything notably complex with Go. Ah, the lack of generics. I haven't really written any particularly large projects, where have you had to use reflection when working with go in your projects? I hope my questions don't come across as dismissive. I think the typical go response to your complaints are "it wont come up" (a YAGNI variant), so I'm always interested to hear about the cases where that argument fails.

ianamartin 6y ago

Well, that's too bad. I always thought this was a cool project. But if you can't dev your way into decent performance for a small alpha project using python/flask/sql, I don't think your tools are the problem. And I guarantee that a graphql isn't the solution. So, I mean, good luck.

ghoshbishakh 6y ago
what do you feel the problem is?
- ianamartin 6y ago
  There are lots of production sites that serve 10,000x as much traffic as sourcehut that are built on Python/flask/sqlalchemy serving RESTful APIs. If you can't make that combination work well, there's another place to look for problems besides your tool kit. You might need to ask yourself if you really understand the tools you're trying to use. But like I said, this has always been a very cool project. My "good luck" was meant more as actual good luck than a Morgan Freeman You're-trying-to-blackmail-batman kind of good luck.
  - deleted 6y ago
    [deleted]
  - xrisk 6y ago
    What are some production websites that run Python stacks? I’m curious. On a tangential note: if anybody has blog posts on scaling Flask/SQLAlchemy or Django stacks I would appreciate it.
    - searchableguy 6y ago
      Dropbox is a notable one born here.
      - procinct 6y ago

RcouF1uZ4gsC 6y ago

> My work on SourceHut has, on the whole, really soured my opinion of Python as a serious language for large projects. A lot of the things thank make Python great for small projects, really bite you on a large or long-lived project. For me, the two biggest are lack of types and indentation for scoping. It is really easy to mess up white space during an edit or refactor. In many languages you would just reformat. In python, you have to be careful that a statement suddenly did not end up outside or inside an if block.

frou_dh 6y ago
Python has a better static typesystem than Go does: https://mypy.readthedocs.io/en/stable/kinds_of_types.html#union-types https://mypy.readthedocs.io/en/stable/kinds_of_types.html#un... https://mypy.readthedocs.io/en/stable/kinds_of_types.html#optional-types-and-the-none-type https://mypy.readthedocs.io/en/stable/kinds_of_types.html#op... https://mypy.readthedocs.io/en/stable/generics.html https://mypy.readthedocs.io/en/stable/generics.html https://mypy.readthedocs.io/en/stable/protocols.html https://mypy.readthedocs.io/en/stable/protocols.html
- ianamartin 6y ago
  Even though I like Python a lot, I clicked on a reply to this comment because I was absolutely certain I was going to have to disagree with you. That just didn't seem possible. But . . . I was wrong. That third link with generics makes me cry when I think about the go code I've written.
  - searchableguy 6y ago
    Are those tears result of overflowing with joy or sadness?
    - ianamartin 6y ago
      Can it be both?
  - 6y ago

tmpz22 6y ago

Big OOF. Even the flow of the article is muddied to the point where it takes significant effort to dig out the main points for OP's migration from REST to GraphQL. What I got from it: > The system would become more stable with the benefit of static typing, and more scalable with a faster and lighter-weight implementation. OK static typing I'm with you so far. Faster and lighter weight? I'm not so sure, sounds like you're having troubles with Flask and SQLAlchemy completely unrelated to REST. All production graphQL implementations I've seen are very heavy when they add in authentication and more advanced query capabilities. Is this REALLY so superior to REST? > Another (potential) advantage of GraphQL is the ability to compose many different APIs into a single, federated GraphQL schema. I guess the discoverability of GraphQL is better, but 90% of APIs on the internet prove that large REST APIs are very effective and achieve the same thing. > I also mentioned earlier that I am unsatisfied with the Python/Flask/SQLAlchemy design that underlies most of SourceHut’s implementation. The performance characteristics of this design are rather poor, and I have limited options for improvement. The reliability is also not something I am especially confident in. This is where you completely lose me. It's fine if you hate ORMs, its fine if you hate the SQLAlchemy API, but you're blaming your hammer for the fact that you think you built a shoddy house. Going out and buying a hammer won't fix the fact that you're lining up your nails all wrong. > The GraphQL services are completely standalone, and it is possible to deploy them independently of the web application... > ...it is my intention to build out experimental replacement backends which are routed through GraphQL instead. I think these two captions go together, are you describing microservices? This can be achieved just fine with REST using simple load balancing strategies based on url routing or similar. > Almost all of SourceHut today is built with Python, Flask, and SQLAlchemy, which is great for quickly building a working prototype. This has been an effective approach to building a “good” service and understanding the constraints of the problem space. However, it’s become clear to me that this approach isn’t going to cut it in the long term, where the goal is not just “good”, but “excellent” This is a classic example of using a handful of annoying issues to justify an exciting large re-write that doesn't actually address the main issues you are having. If you are struggling with the SQLAlchemy library you will find alternative (and perhaps larger) struggles in all GraphQL implementations. Best of luck, this is a road I would not follow you on. Seriously though I wish you the best and hope your product succeeds despite this.

WhatIsDukkha 6y ago

I don't understand the attraction to Graphql. (I do understand it if maybe you actually want the things that gRPC or Thrift etc gives you) It seems like exactly the ORM solution/problem but even more abstract and less under control since it pushes the orm out to browser clients and the frontend devs. ORM suffer from being at beyond arms length from the query analyzer in the database server. https://en.wikipedia.org/wiki/Query_optimization https://en.wikipedia.org/wiki/Query_optimization A query optimizer that's been tuned over decades by pretty serious people. Bad queries, overfetching, sudden performance cliffs everywhere. Graphql actually adds another query language on top of the normal orm problem. (Maybe the answer is that graphql is so simple by design that it has no dark corners but that seems like a matter of mathematical proof that I haven't seen alluded to). Why is graphql not going to have exactly this problem as we see people actually start to work seriously with it? Four or five implementations in javascript, haskell and now go. From what I could see none of them were mentioning query optimization as an aspiration.

searchableguy 6y ago
That is upto the graphql framework and the consumers of them. Graphql is just a query language. You need to have data loader (batching) on the backend to avoid n+1 queries and some other similar stuff with cache to improve the performance. You also have cache and batching on the frontend usually. Apollo client (most popular graphql client in js) uses a normalized caching strategy (overkill and a pain). For rate/abuse limiting, graphql requires a completely different approach. It's either point based on the numbers of nodes or edges you request so you can calculate the burden of the query before you execute it or deep introspection to avoid crashing your database. Query white listing is another option. There are few other pain points you need to implement when you scale up. So yeah defo not needed if it's only a small project.
- kabes 6y ago
  "You have to calculate the burden of the query before you execute it so you don't end up crashing your database." This sounds like disaster waiting to happen.

tleb_ 6y ago

I think this article misses the explanation of why GraphQL over REST. I usually don't like "x versus y" articles but here both have been tested on SourceHut, the hindsight should probably appear as useful. Thanks Drew and others for SourceHut.

PostThisTooFast 6y ago
What is SourceHut?

tannhaeuser 6y ago

Tbh I'd expected a little better than framing this question in a "REST vs GraphQL" discussion coming from sourcehut.org. If you control your backend, you can aggregate whatever payloads you please into a single HTTP response, and don't have to subscribe to a (naive) "RESTful" way where you have network roundtrips for every single "resource", a practice criticized by Roy Fielding (who coined the term "REST") himself and rooted in a mindset I'd call based more on cultural beliefs rather than engineering. That said, a recent discussion [1] convinced me there are practical benefits in using GraphQL if you're working with "modern" SPA frameworks, and your backend team can't always deliver the ever-changing interfaces you need so you're using a backend-for-fronted (an extra fronted-facing backend that wraps your actual backend) approach anyway, though it could be argued that organizational issues play a larger role here. [1]: https://news.ycombinator.com/item?id=23119810 https://news.ycombinator.com/item?id=23119810

jamil7 6y ago
I like GraphQL but if you're just serving a single SPA I wonder about all this busy work we still have to do. Why haven't we gone a step further and just abstracted all the networking and serialisation steps away and our models are synced for us in the background. Maybe the apollo team is heading in this direction but their offline story isn't great yet. Edit: I remember now that the Apollo team is made up of members of the former Meteor team which worked in a similar way using a client side database.
- searchableguy 6y ago
  You can use rxdb for replication and offline support.
  - jamil7 6y ago
    Yeah good point, WatermelonDB, rxdb and pouchdb all fit this model. This to me feels like the future for web and mobile.
    - 6y ago

zapf 6y ago

I like my front end team to not write queries. I have seen too many front end developers write queries equivalent to "select * from users, t1, t2, ... tn, left outer join on users.id = t1.user_id... etc etc etc". I think that is just bullshit. I prefer to tell the frontend people - these are the endpoints that backend team provides, go live with it. If you really have issues, talk to them, and they might just add a new endpoint for you. If you let frontend developers dictate backend queries, all you will get is a big master cluster fuck. I am talking average joe developers that we usually find to work on startups.

runawaybottle 6y ago
I think Frontend has the right to take over certain domains of the backend. A very clear example of this is templating. Templating used to be a backend task, but it became very clear it was the frontend developers that needed power over this domain. Going over to a backend dev to make adjustments to the templates is an inefficiency that had to be solved, period. I agree with you that writing optimal queries or data modeling should not be shifted over to the frontend. With that said, there are basic aspects of this equation that can be factored out. There should be a layer where frontend can at the very least pick or mix the data they need. That doesn’t necessarily mean Graphql, but could also mean a simple middle stack layer where one can do this basic thing without being able to shoot themselves in the foot. There’s a responsible way to do this, and most likely will be an ongoing discussion.

cletus 6y ago

So I've now had the opportunity to use both GraphQL and protocol buffers ("protobufs" is the more typical term) professionally and I have some thoughts on this. 1. Protobufs use integer IDs for fields. GraphQL uses string names. IMHO this is a clear win for protobufs. Changing the name of a field of GraphQL is essentially impossible. Once a name is there it's there forever (eg mobile client versions are out there forever) so you're going to have to return null from it and create a new one. In protobufs, the name you see in code is nothing more than the client's bindings. Get a copy of the .proto file, change a name (but not the ID number) and recompile and everything will work. The wire format is the same; 2. People who talk about auto-generating GraphQL wrappers for Postgres database schemas (not the author of this post, to be clear, but it's common enough) are missing the point entirely. The whole point of GraphQL is to span heterogeneous and independent data sources; 3. Protobuf's notions of required vs optional fields was a design mistake that's now impossible to rectify without breaking changes. Maybe protobuf v3/gRPC did this. I'm honestly not sure. 4. Protobuf is just a wire format plus a way of generating language bindings for it. There are RPC extensions for this (Stubby internally at Google; gRPC externally and no they're not the same thing). GraphQL is a query language. I do think it's better than protobufs in this regard; 5. GraphQL fragments are one of these things that are probably a net positive but they aren't as good as they might appear. You will find in any large codebase that there are key fragments that if you change in any way you'll generate a massive recompile across hundreds or thousands of callsites. And if just one caller uses one of the fields in that fragment, you can't remove it; 6. GraphQL does kind of support union types (eg foo as Bar1, foo as Bar2) but it's awkward and my understanding is the mobile code generated is... less than ideal. Still, it's better than not having it. The protobuf equivalent is to have many optional submessages and there's no way to express that only one of them will be popualated; 7. Under the hood I believe the GraphQL query is stored on the server and identified by ID but the bindings for it are baked into the client. Perhaps this is just how FB uses it? It always struck me as somewhat awkward. Perhaps certain GraphQL queries are particularly large? I never bothered to look into the reason for this but given that the bindings are baked into the code it doesn't seem to gain you much; 8. GraphQL usage in Facebook is pervasive and it has first class support in iOS, Android and React. This is in stark contrast to protobufs where protobuf v2 in Google is probably there forever and protobuf v3/gRPC is largely for the outsiders. It's been several years now since I worked at Google but I would be shocked if this had changed or there was even an intention of changing it at this point; 9. The fact that you can do a GraphQL mutation and declare what fields are returned is, IMHO, very nice. It saves really awkward create/update then re-query hops. 10. This is probably a problem only for Google internally but another factor on top of protobuf version was the API version. Build artifacts were declared with this API version, which was actually a huge headache if you wanted to bring in dependencies, some of which were Java APIv1 and others Java APIv2. I don't really understand why you had to make this kind of decision in creating build artifacts. Again, maybe this has improved. I would be surprised however. Lastly, as for Sourcehut, I had a look at their home page. I'm honestly still not exactly sure what they are or what value they create. There are 3 pricing plans that provide access to all features so I'd have to dig in to find the difference (hint: I didn't). So it's hard for me to say if GraphQL is an appropriate choice for them. At least their pages loaded fast. That's a good sign.

jorams 6y ago

In my experience GraphQL can be much nicer to implement than REST, and it offers a good structure around things that many REST APIs implement in particular ways (like selecting which fields you want). The pain you'll experience depends heavily on your data model and the abuse potential that brings. I think the biggest problem with GraphQL is the JavaScript ecosystem around it, and all of its implicit context. It seems to be built entirely on specific servers and clients, instead of on the general concepts. Relay[1], a popular client-side library, adds all kinds of requirements in addition to the use of GraphQL. One of those is that until version 8, it required all mutation inputs and outputs to contain a "clientMutationId", which had to be round-tripped. It was an obvious hack for some client-side problem which added requirements to the backend. Somehow it had a specification written for it instead of being fixed before release. This hack is now in public APIs, like every single mutation in the GitHub API v4. GraphQL also includes "subscriptions", which are described incredibly vaguely and frankly underspecified. There are all kinds of libraries and frameworks that "support subscriptions", but in practice they mean they just support the websocket transport[2] created by Apollo GraphQL. If you just use it as a way to implement a well-structured API, and use the simplest tools possible to get you there, it's a pleasure to work with. [1]: https://relay.dev/ https://relay.dev/ [2]: https://github.com/apollographql/subscriptions-transport-ws https://github.com/apollographql/subscriptions-transport-ws

kasbah 6y ago
I don't think Relay is used that much outside of Facebook. The community seems to have settled on Apollo. Personally I find Apollo over-engineered. When I couldn't delete things from the cache, because of a bug, and was faced with digging into the complex code-base, I ended up just using straight JSON with a HTTP client/fetch and caching in a simple JS object. Other users of my API [1] just use straight HTTP with JSON as well. GraphQL clients seem to solve something we are not encountering. If urql [2] or gqless [3] work well when I try them I'd be up for changing my mind though. [1]: https://github.com/kitspace/partinfo https://github.com/kitspace/partinfo [2]: https://github.com/FormidableLabs/urql https://github.com/FormidableLabs/urql [3]: https://gqless.dev/ https://gqless.dev/
- 6y ago

leadingthenet 6y ago

Might I suggest taking a look at FastAPI? It’s been a 10x+ improvement on Flask, in my experience.

say_it_as_it_is 6y ago

Where are the GraphQL lessons learned? The author hasn't even implemented a solution with it yet, but that hasn't stopped him from declaring it to the world. I don't find an announcement useful. Maybe GraphQL adopters aren't sharing their experiences with it in production because they're realizing its faults? People are quick to announce successes and very reluctant to own, let alone share, costly mistakes. Also, people change jobs so often that those who influence a roll-out won't even be around long enough for the post-mortem. GraphQL publicity is consequently positively biased. If the HN community were to follow up with posters who announced their use of GraphQL the last two years, maybe we can find out how things are going?

dgellow 6y ago
The author isn’t the one who shared the link on HN.
gbear605 6y ago
The reason this post was written was for users of Sourcehut, especially for people writing to its API. This post isn’t particularly relevant or explanatory for other audiences, but I don’t think it’s supposed to be so that’s fine.
- say_it_as_it_is 6y ago
  I understand what you mean. That makes sense.

orf 6y ago

The author says that he has soured on Python for “serious, large projects”. While it’s clearly personal opinion, and that’s fair enough , I can’t help but think his choice of framework hasn’t helped him and has likely caused significant slowdown when delivering features. Looking through some of the code for Sourcehut, there’s an insane amount of boilerplate or otherwise redundant code[1]. The shared code library is a mini-framework, with custom email and validation components[2][3]. In the ‘main’ project we can see the views that power mailing lists and projects[4][5]. I’m totally biased, but I can’t help but think “why Flask, and why not Django” after seeing all of this. Most of the repeated view boilerplate would have gone ([1] could be like 20 lines), the author could have used Django rest framework to get a quality API with not much work (rather than building it yourself[6]) and the pluggable apps at the core of Django seem a perfect fit. I see this all the time with flasks projects. They start off small and light, and as long as they stay that way then Flask is a great choice. But they often don’t, and as the grow in complexity you end up re-inventing a framework like Django but worse whilst getting fatigued by “Python” being bad. 1. https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrht/blueprints/public.py https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrh... 2. https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/email.py https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/emai... 3. https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/validation.py https://git.sr.ht/~sircmpwn/core.sr.ht/tree/master/srht/vali... 4. https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/blueprints/mailing_lists.py https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/bl... 5. https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/blueprints/projects.py https://git.sr.ht/~sircmpwn/hub.sr.ht/tree/master/hubsrht/bl... 6. https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrht/blueprints/api/pastes.py https://git.sr.ht/~sircmpwn/paste.sr.ht/tree/master/pastesrh...

StavrosK 6y ago
Exactly agreed. I basically only use Flask for things I want to explicitly be single-file these days. For anything larger, I reach for Django, because I know that if I need at least one thing from it (and I always need the ORM/migrations/admin), it will have been worth it. My current favorite way of building APIs is this Frankenstein's monster of Django/FastAPI, which actually works quite well so far: https://www.stavros.io/posts/fastapi-with-django/ https://www.stavros.io/posts/fastapi-with-django/ FastAPI is a much better way of writing APIs than DRF, I wish it were a Django library, but hopefully compatibility will improve as Django adds async support.

mehdix 6y ago

> With these, you can deploy a SourceHut instance with no frontend at all, using the GraphQL APIs exclusively. This reminds me of Kubernetes' design. You have an API server which is practially the Kubernetes from user's perspective. `kubectl` is just one out of possibily many clients that talk to this API. Edit: typos.

FpUser 6y ago

I've been using home grown RPC for my servers (POST + JSON) for HTTP or binary serialization for lower level access. Works like a charm for years. Never felt like missing anything.

ddevault 6y ago

Author here. Wow, there is a ton of "didn't RTFA" comments here. It seems like half of this thread saw GraphQL in the title, it knocked two gears into place in their head, and they started writing up their own little essay about how bad it is. I evaluated GraphQL twice before, and discarded it for many of the reasons brought up here. Even this time around, I give a rather lackluster review of it and mention that there are still many caveats. It's not magic, and I'm not entirely in love with it - but it's better than REST. Query optimization, scalability, authentication, and many other issues raised here were part of the research effort and I would not have moved forward with it if I did not feel that they were adequately addressed. Before assuming some limitation you have had with it in the past applies to sr.ht, I would recommend reading through the code: https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/api https://git.sr.ht/~sircmpwn/git.sr.ht/tree/master/api https://git.sr.ht/~sircmpwn/gql.sr.ht https://git.sr.ht/~sircmpwn/gql.sr.ht If you're curious for a more detailed run-down of my problems with REST, Python, Flask, and SQLAlchemy, I answered similar comments last night on the Lobsters thread: https://lobste.rs/s/me5emr/how_why_graphql_will_influence_sourcehut https://lobste.rs/s/me5emr/how_why_graphql_will_influence_so... I would also like to point out that the last time I thought a rewrite was in order, we got wlroots, which is now the most successful project in its class. Cheers.

AmericanChopper 6y ago

GraphQL as a query language is simply better than REST in most cases imo. REST has too much client side state, which not only has the potential to make things harder for clients to consume, but also has all the inconsistent states to handle where your consumer gets part way through a multiple-REST method workflow, and then bails. REST also absolutely sucks for mutating arrays. Really I just look at GraphQL as a nice RPC framework. The graph theory operations like field level resolvers are mostly useless. But if you treat each relationship as a node rather than each field, you can get it to work very nicely with a normalized data set. I haven’t found it hard to preserve join efficiency in the backend either, and it so far hasn’t forced me into redundant query operations. Just as long as you don’t use appsync. Really, don’t even bother.

GordonS 6y ago
> GraphQL as a query language is simply better than REST in most cases imo. REST has too much client side state, which not only has the potential to make things harder for clients to consume, but also has all the inconsistent states to handle where your consumer gets part way through a multiple-REST method workflow, and then bails. How much client state you maintain seems to me to be orthogonal to GraphQL/REST. Take your example or a multiple-REST workflow. I presume your point was that the workflow could be implemented by a single GraphQL query/mutation/whatever - but just the same, you can put as much code and logic as you like behind a REST call?
- AmericanChopper 6y ago
  You could do that, but if you start creating endpoints for transactions rather than method -> resource endpoints, then you’re not really making a REST interface anymore. But even ignoring REST purity, I’d argue that GraphQL is better suited to that design pattern in general.
  - GordonS 6y ago
    When most people refer to REST APIs, they really mean "HTTP APIs". I really don't think we should reach for GraphQL just because of the ideological notion of daring to not adhere 100% to REST.

geitir 6y ago

One thing I've noticed is it seems redux and graphql are solving the same problem in an entirely different way

iso-8859-1 6y ago
Redux brands itself as a tool for managing state. How does GraphQL manage state?
- geitir 6y ago
  The way I see it is Redux is a local normalized store which can then be used via the selector pattern to generate custom objects / views. With GraphQL you are requesting that derived data to be generated for you via the query language. Regarding managing state I don't see GraphQL helping with that at all.

crabmusket 6y ago

> Another (potential) advantage of GraphQL is the ability to compose many different APIs into a single, federated GraphQL schema. If anyone else can share experiences of this sort of problems and solution, I'd be really interested to hear it. I've written non-GQL APIs before that back onto other internal and external services; what am I missing?

maddyboo 6y ago
I think Drew is referring to composition in terms of API end-user code, not sr.ht code, e.g. making it possible for the user write a single GQL query that combines data from multiple sr.ht services.
- crabmusket 6y ago
  Ah, I think that makes a little more sense. Thanks!
GordonS 6y ago
I've mentioned my (negative) experience elsewhere in this thread. It wasn't actually me doing the implementation, but the result was a horrendous, unmaintainable and slow code base. Ultimately, no part of the team benefited from the use of GraphQL instead of REST. To be fair, the same devs that built it using GraphQL would likely have made many of the same mistakes with a REST API, but I do feel it would at least have been easier to reason about the code.

pknopf 6y ago

I firmly believe GraphQL is a fad. It will never work, exposing the ORM to your clients. I'm curious, why don't we see more public-facing APIs using gRPC?

iso-8859-1 6y ago
gRPC is awkward on web because it assumes HTTP/2. The web is still half HTTP/1. Also, gRPC does almost everything, but not quite, it doesn't know how to synchronize state in general. It is a heavy weight choice considering that you may only need protobuf. And protobuf is not natively supported in browsers, web devs can't agree on serialization libs, they can only agree on the serialization format. And they didn't choose protobuf ;)
zimpenfish 6y ago
Currently working on a public-facing gRPC API and it's much more faffy than a REST API. Browsers don't talk gRPC which means you need to support gRPC-web endpoints. You probably need a gRPC aware proxy too and most of those have their own challenges. Some stuff doesn't like protobufs which means you need JSON transport which means protoc plugins. etc.etc. It's all just more work for (IMHO) little gain.

hn_throwaway_99 6y ago

Based on the top comments in this thread, I can see my previous attempts to dispel mistaken beliefs around graphql have failed, but I'll still try! 1. The biggest mistake GraphQL made was putting 'QL' in the name so people think it's a query language comparable to SQL. It's not: https://news.ycombinator.com/item?id=23120997 https://news.ycombinator.com/item?id=23120997 2. Some benefits of GraphQL over REST: https://news.ycombinator.com/item?id=23124862 https://news.ycombinator.com/item?id=23124862

detaro 6y ago
I suspect as long as https://graphql.org https://graphql.org says what it does, you're going to have a hard time with that fight...

deleted 6y ago

[deleted]

iooi 6y ago

I wanted to learn GraphQL recently and I wrote a small library to automagically generate GraphQL schemas from SQLAlchemy models. [1] It's inspired by Hasura, the schema is almost the same. It's not optimized at all, but it's a nice way to quickly get started with GraphQL and expose your existing models. [1] https://github.com/gzzo/graphql-sqlalchemy https://github.com/gzzo/graphql-sqlalchemy

hpen 6y ago

I tried GraphQL but my resolvers became extremely complicated. Am I missing something or is this just how it goes?

camgunz 6y ago

A lot of places have a hell of a time dealing with very nested graphql queries that effectively DoS your app servers, or a resolver that’s very slow. Caching is also an open question. But for the simplest systems, I’d hesitate to recommend graphql at this point.