16 ms·

Migrating Netflix to GraphQL safely

235 points by theptip 3y ago

latchkey 3y ago
I wonder how many hidden n+1 problems there are. Reading the blog post, it looks like they mostly only tested existing pathways as part of the migration. Who knows what random requests could do.
- gavinray 3y ago
  This is Netflix, not some amateur-hour implementation copied from a blogpost. The de-facto GraphQL Java library comes with dataloaders built-in anyways: https://www.graphql-java.com/documentation/batching/ https://www.graphql-java.com/documentation/batching/
  - voz_ 3y ago
    While Netflix has a good reputation, we should not give blanket benefit of the doubt, especially since the parent comment was a rather interesting and inquisitive one.
- rand_r 3y ago
  Yes, this is the biggest problem with GraphQL that I haven't addressed anywhere. The architectural beauty of GraphQL is that you can write isolated, functional nodes that only have to know about their object type. For example, the User node only has to know how to get the user's last_login, active status, and return a reference to another node type like a profile Image. Inside the User node, you are free to write code (e.g. Java or Python) that can compute which Image node is the right one to return. i.e. you can write: class UserNode: def get_user_profile_image() -> ImageNode: return some_arbitrary_function() There is no way I've seen to square this kind of arbitrary node computation with the desire to write an optimized SQL query that selects all needed information in one big-ass statement.
  - valenterry 3y ago
    The exact same problem exists in an typical REST api as well. In both cases you have to add a specialized endpoint to retrieve multiple results.
  - striking 3y ago
    You could use https://github.com/join-monster/join-monster https://github.com/join-monster/join-monster, but setting that aside it's usually enough to use Dataloaders that resolve based on some source ID, with some server-local caching on top if latency becomes an issue.
- 727374 3y ago
  Dataloader (https://github.com/graphql/dataloader https://github.com/graphql/dataloader) eliminates many n+1 issues. Basically, entity requests are queued up during evaluation of the query and then you can batch them up for fulfillment. It’s a great addition to other asynchronous contexts as well. WRT random requests, there are libraries for query cost estimation you can use as a gate.
  - theptip 3y ago
    Interesting, hadn’t seen that package, thanks. The general pattern seems quite useful, is it implemented in any SQL ORMs? Seems like you could do the same batching/coalesce strategy for async Postgres for example, but I don’t see anything after a quick scan of the docs in SQLAlchemy. (Seems like it would be feasible since they already batch requests in unit-of-work, they just don’t coalesce to bulk operations AFAICT.)
    - IceDane 3y ago
      Using something like `pothos` for your graphql backend, you can get tight integration with `prisma`, which will practically eliminate any N+1 issues. For fields which hit external services, you can define types as "loadable" so that every time those are requested in a batch, they are loaded efficiently to avoid n+1.
  - latchkey 3y ago
    Correct, there are solutions out there, but unknown if they implemented them.
nf17 3y ago
Is JS/Express/Apollo still the goto graphQL server implementation or C#(HotChocolate), Java, go etc are widely used too? On the client side(for mobile), AFAIK iOS(Swift) and Android(Kotlin) does not support GraphQL natively? are there 3rd party libraries that can be used? Edit: Added Apollo
- gavinray 3y ago
  The "go-to" GraphQL server implementation varies based on language. Here are the popular ones to my knowledge: - Node.js: GraphQL Yoga + Fastify, Apollo Server - Java/Kotlin: Netflix DGS (built on top of Spring Boot) - Scala: Caliban, Sangria - Python: Strawberry, Graphene Django - C#: HotChocolate > "On the client side(for mobile), AFAIK iOS(Swift) and Android(Kotlin) does not support GraphQL natively?" GraphQL queries are just HTTP POST queries with a JSON body. They're supported everywhere. If you want specialized tooling for them, Kotlin and Swift both have great strongly-typed GraphQL libraries. Apollo publishes libraries for both: - https://www.apollographql.com/docs/kotlin/ https://www.apollographql.com/docs/kotlin/ - https://www.apollographql.com/docs/ios/ https://www.apollographql.com/docs/ios/
  - a_wild_dandan 3y ago
    It appears that fastify can't keep their website up.
    - preommr 3y ago
      Apparently something got messed up with their cloudflare setup and the person with access is at a wedding or something. https://twitter.com/fastifyjs/status/1670145210347577344?cxt=HHwWgICzoYqOxq0uAAAA https://twitter.com/fastifyjs/status/1670145210347577344?cxt...
  - migf 3y ago
    Apollo is also available for Elixir, where I have seen it much loved.
  - re-thc 3y ago
revskill 3y ago
How to avoid N+1 query ? You don't join SQL.
- x86x87 3y ago
  brave to assume they are avoiding this problem :/
- lolive 3y ago
  I am not sure I understand that n+1 select issue with graphQL. On my project, we generate SPARQL queries [to simplify, it is not so different from a GraphQL query] and we send it to a SQL translator that then sends that SQL to a Spark cluster. It is reasonably efficient, and extremely flexible. Of course, this is a single database use case, but I don’t see any n+1 select involved here. And I do not understand how single database graphQL servers cannot work exactly the same.
- ptramo 3y ago
  The most common practice is to turn N+1 into 1+1 using dataloaders (https://github.com/graphql/dataloader https://github.com/graphql/dataloader for JS, there are equivalents for most implementations). The N resolvers invoke a single batched loader which receives a list of keys and returns a list of values.
toomim 3y ago
Falcor was cool. I'm sad seeing it go away.
- jFriedensreich 3y ago
  I would not say it is more going away now than in the last 2 years of not getting upstream support from netflix. If anything this step at least makes clear this is a dead end at netflix and we can think about how to go forward with what is/was great about falcor!
itake 3y ago
I don’t see the value ins Graphql. Most apis rarely change and the complexity of supporting queries is not worth the time saved. Graphql was a cool experiment, but I’m sticking with rest/protobufs
- devmor 3y ago
  For an API that centers around filters and nested subfilters to retrieve exponentially specific data, graphql can end up saving you a lot of work. For anything else, it seems pointless.
- rand_r 3y ago
  The original motivation was to reduce the number of network requests made on cellular network-connected mobile devices, both to save on power consumption and reduce latency for the user. It off-loads a lot of complexity and computation from the client to the server, and make sense if want to create as good a mobile experience as you can. In terms of code architecture, it drastically simplifies frontend code, and overall reduces backend code and boilerplate. It's extremely elegant and nice to use over creating custom endpoints for every use-case, but the N+1 SQL query problem is a giant thorn.
  - itake 3y ago
    You can still design an immutable REST api to match the client's needs. Apollo has really nice out-of-the-box caching that can be tricky to replicate with rest. I see your point about reducing network requests (graphql lets you co-mingle mutations and queries), but I don't think that feature is used often enough, b/c what if one query fails, then everyone has to fail).
    - evangow 3y ago
      You can make a REST endpoint match a client's needs, but if you have multiple clients (for example, 1 for mobile and 1 for desktop) that all need different amounts of data (for example, the mobile client shows a simplified view with less data), then you would need to write 2 different REST endpoints to handle each clients. Multiply that by the number of pages with a difference.
x86x87 3y ago
I never understood why people go for graphql. In my experience, it's a nightmare to 1/ secure 2/ version 3/ ensure qos Securing it properly should make it a no-go in like 95% of cases. you get amazing flexibility on the front end with a heavy heavy cost on the backend. Also, in general being able to say "i want to do whatever" and request everything at once is an anti-pattern IMHO, especially after HTTP2 became mainstream and doing multiple requests is reasonably fast. Versioning? Forget about it. Now you have "all the versions" and good luck figuring out what is used, what is not used or god forbid deprecate something. you're going to have a bad time. As far as qos goes, if any client can range from "i'm asking a simple thing" to "i would like the whole world please - and btw you're only going to figure things out as you pull them in" qos becomes a pipe dream. Last, I don't understand what Netflix does that is so complex that would warrant something like gql. I just don't. To the naive developer in me it seems that they 1) need to have a basic api to get whatever catalog they have + a few apps build on top of that 2) have figured out how to do streaming exceptionally well in order to scale to all the people watching. That's is and although a gross oversimplification I cannot think about a scenario where gql is needed. I think in their desire to reinvent the wheel Netflix has jumped the shark. A lot of really good ideas have come out of the innovation they did in the past but lately it seems like they are doing things for the sake of doing them or they are pissing people off with password household policies and whatnot.
- downWidOutaFite 3y ago
  My main issue with it is client-side caching doesn't map cleanly to graphql. I want well-defined objects to cache.
  - gareve 3y ago
    that's solved with Relay's graphstore
    - x86x87 3y ago
      There's a solution to everything /s We started with a somewhat simple problem and now let's keep throwing things in. How about something I can just curl? Is that too much to ask?
sibeliuss 3y ago
Definitely not working anywhere that doesn't have a GraphQL api. Once you've used it at scale there's no going back.
- runeks 3y ago
  Go on…
jFriedensreich 3y ago
This is a sad but expected day. For years i seem to have been the last lonely falcor user who is still excited about this and thinks this is one of the best abstractions for a key class of state management applications, that are not a great fit to graphQL at all. - Falcor can be schemaless with optional json schema where it makes sense - Falcor supports simpler to reason about and implement features that map perfectly to javascript objects so you can do zero boilerplate databinding with ES proxies without writing query and mutation wrappers - Does not try to be a qyery language but only expose things you need to request the data that needs to be rendered in the UI I think not even the creators of falcor fully understood how perfect it is if you just add in the last missing piece of fully automating the connection from the falcor model to the place in the UI templates where the data is needed. Most people tried doing this manually in component boilderplate which lead initially to preloading tens of json paths eg falcor.preload([`todos.done.length`,`todos.done[0..10].name`, `todos.done[0..10].date`, ...etc.]) and then people developing GraphQl like syntax for these eg. somthing like load(` todos { done 0...10 {name, data} } etc. `) at which point devs thought they might as well be using graphQL and get more features The key mistake was that you need to instead think about your UI components ITSELF as the data quueries eg you can automatically derive all falcor quieries from a component like this without ANY boilerplate, and this is only possible with falcor, as everything else has so many features that it would not be possible to map the UI code to the intended data queries {#each todos as todo} <div>{todo.name} ({todo.date})</div> {/each} It does not look like netflix will hand over falcor to a community and the codebase needs lots of work especially around tooling and overuse of some patterns like a custom observable implementation but i am still on the fence about building a new minimal derived library or forking
CSDude 3y ago
Maybe now GraphQL api can stop auto completing movies that does not exist in their catalog.
- paulddraper 3y ago
  ? It autocompletes so you find out more quickly that Netflix doesn't carry it, and what similar options are I get that you are annoyed Netflix didn't have the movie you wanted, but autocomplete doesn't make that worse.
  - 9dev 3y ago
    You’re right, but it’s annoying for sure. Instead of telling me they don’t have a title, they complete its full name (as if to say „I know what you want, but I’m not gonna give it to ya“), yet show me some arbitrary stuff named similar. It’s just a poor solution.
    - kiwijamo 3y ago
      Usually, the search results are movies with similar genre to the one I was looking for, not just simply similar spelling etc. I assume they look up stuff like what other movies people who had seen the searched-for movie have seen. So if I was looking for Movie A and Netflix knows that people who've seen Movie A tend to like Movie B, C, D, etc (and these are available in your part of the world) it'll show these movies instead. I think this is nicer and better than just failing with zero results.
  - CSDude 3y ago
    I don't look for similar movies, I look for the exact movie I want, it's just a redirection of their poor catalog. If that was the intended use case, they'd have proper catalogs to drill down instead of a very long, arbitrary vertical canvas of always changing, non consistent categories.
    - paulddraper 3y ago
      Sounds like a personal preference. When I search for "How to Train Your Dragon 2", I find Netflix doesn't have it. But it has the original, plus a few series, so I turn on one of those instead.
saurik 3y ago
Why? What problem did this solve for them? I use Netflix a lot, and I use it on AppleTV where I get automatic updates to the device and the app... the app has never changed except to get slower over the years (and I mean a lot slower). There is absolutely no new functionality they have ever deployed... isn't the app "done"? Is there some serious scaling issue on their backend that is causing the perceived frontend slowness that this will fix? Is this going to massively decrease the costs of the backend? Or is it just that the engineers are all bored and can't accept the idea that the product is either 1) done or 2) needs a new direction (instead of some new tech)?
- mkwarman 3y ago
  I have definitely noticed and been frustrated by what you're calling out here. There always has to be something to work on, even if that thing might invalidate a significant amount of previous effort or cause regression. As a developer myself, I guess it is good for me, because it means job security. And maybe that's why there's always something to be added or changed or re-created on the products I've helped develop - to ensure all the individual contributors have something to keep them busy. But it definitely is frustrating that it seems like an app being called "done" is extremely rare nowadays. Though I suppose developers didn't really have the choice to continue development indefinitely back before the internet allowed "miscellaneous bug fixes and feature updates" patches to be pushed weekly. Edit: I just noticed who I was replying to. Thanks for your work on Cydia and in the iOS jailbraking scene at large. I have lots of fond memories jailbraking and installing all kinds of weird tweaks on my iPhone 4 back in the day.
- Dudester230602 3y ago
  It is surely solving a promotion problem and a bonus-getting problem for someone in their hierarchy. Plus it is making the world a better place through constructing elegant hierarchies for maximum code reuse and extensibility and through minimal message-oriented transport layers.
- thephyber 3y ago
  > isn't the app "done"? Netflix is an ecosystem, not just the frontend app for Apple TV. Netflix is pretty close to a vertical monopoly for some of its content. Fund-raising, planning, development, post-production, translations/dubbing, preparation for distribution, and distribution. Your idea of viewing a video content on the app is only the last 20%. Also worth reminding people that the AppleTV app is not the only app Netflix supports for viewing. There have been dozens/hundreds of different apps and most are still actively supported. Also, they are a company, so they are always looking to be able to optimize and cut costs. It’s worth staying curious and asking why before assuming you know more than the engineers who have to work “behind the curtain”.
valenterry 3y ago
I have a lot of experience with GraphQL and I love it. It saves a ton of development time. However, for those who wonder what the disadvantages are, here are some. They are from real world production experience and they are not what people usually claim wrongly (like graphql would be harder to secure/version, ...) No, some real pain points are: 1. Lack of a Map-type. There is no native type to return a list/array of objects but not a map with unique keys or even a set instead of a list. You can fallback to "convention" or custom types (like plain json) but it sucks. 2. Uploading big files (in chunks) is much more tricky than expected 3. There are union-types for the returned data. For example, I can query for an animal and get either a cat or a dolphin and if I use typescript or another capable language then the compiler will tell me that. But I cannot send such a request to the server, saying "here I send you an animal, the data is this ...". The server will have to provide a separate query/endpoint per animal. Very very annoying. 4. While graphql is awesome when it comes to batching requests, what it is not so awesome at is to improve performance by allowing the server to return the part of the data is already resolved. So while reducing the number http requests, now the previous slowest http request will become the bottleneck for all. This means the page might load subjectively faster because some parts are rendered later than possible. There is work going on here (like @defer) but it's not fully spec'd and also the solution doesn't work for some cases There are more points but this gives you some idea of the ACTUAL problems. Otherwise, from development and setup time over maintenance, analytics, versioning/deprecation, tooling and so on, graphql is much better than your standard REST api in my opinion.
- scotty79 3y ago
  > Lack of a Map-type. Why do you need special map types for transferring maps? Doesn't list of key-value pairs suffice? Also, why am I wrong to even ask the above questions?
  - tadfisher 3y ago
    A (single-level) Map is generally understood to have unique keys, which allows for some invariants derived from this property, such as a one-to-one or one-to-many object relationship, without runtime checks in client code. I do think it's worthwhile to ask the question, though. I have a strong personal preference to avoid encoding invariants like this in wire representations, because they are properties of the _schema_ and not the wire type (JSON object, in my most commonly-encountered case). Of course, the entire point of GraphQL is to encode invariants in the data schema, so it's entirely reasonable to ask for a built-in Map type instead of building your own for every project.
gumballindie 3y ago
Curious how they handle GQL injections. Most devs using graphql have no clue what they are as the majority just cargo cults new frameworks to keep themselves busy.
- valenterry 3y ago
  What are GQL injections? You mean something like SQL injections that come through the graphql API?
  - gumballindie 3y ago
    Exactly. Suppose you have a front end system that captures user input, and then the backend communicates with another system using the input data. If not properly escaped and is concatenated then it can lead to an injection attack just like regular sql.
    - valenterry 3y ago
      I find the terminology strange. The exact same can happen with a standard REST api. In fact, it's even more likely since there is no type system to assist there. Would you then call it REST injections too?
cmgriffing 3y ago
I think the important thing for people to recognize is that Falcor, the tool used in their mobile apps, was already GraphQL-like. So, this isn't a giant change.
cakoose 3y ago
> We migrated Netflix’s mobile apps to GraphQL with zero downtime Is "zero downtime" relevant here? For example, when migrating a database that is the source-of-truth for data, then yeah, downtime is something you have to contend with. But for a client-server API change, isn't it trivial? Companies do this all the time -- expose a new version of an API then gradually migrate clients over. (It's definitely not a trivial task overall! For example, you need thorough testing to make sure you don't break existing functionality and the article does talk about that.)
- why-el 3y ago
  I don't know, you said isn't trivial then you said it's not trivial! :D It's a colossal effort if you are at Netflix size, they support probably the largest number of client devices on the planet.
lumb63 3y ago
This is cool, but why? What’s the business use case or problem they’re trying to solve? I would have liked to hear about that. Without that critical piece, it sounds like Netflix engineers spent a lot of time and effort just to migrate to a new framework. It also bears mentioning that Netflix is, or pretty much should be, complete. They don’t need to make any changes to the service; if so, it’s very few. I’m sure they need talented engineers to scale up their operations, and great SREs, but what are they still writing code for?
- XorNot 3y ago
  This has been a perennial question about Netflix for a while now: they've tended to want to brand themselves as a tech company, but that hasn't been true for a long time now and was questionable to start with. If they were a tech company that'd be a SaaS provider for video streaming, in which case all of this would make sense - the proliferation of streaming providers certainly provides the use-case that someone could offer that as a branded service. But the time for that has come and gone as well - the other streaming services exist, and they work fine.
- WorldMaker 3y ago
  The business case seems to be to finally kill Falcor [1] which had a lot of similarities to GraphQL but a much smaller maintenance and developer community than GraphQL and I would assume looked a lot like tech debt to Netflix at this point. [1] https://github.com/Netflix/falcor https://github.com/Netflix/falcor
ralphstodomingo 3y ago
A lot of the sentiment coming from non-gql people in this thread start with: "gql is impossible to X, Y, Z" which is immediately untrue by inference if you base it from how well existing companies in the wild have adapted to it, and provable on your own if you just went out and tried it yourself. "from what I can see in the landing page, REST can do everything it does anyway" is partially true, but is not useful, because how else would have gql picked up adoption if it can't provide similar benefits as REST? I'm smelling a lot of cope from people who have buried their heads so deep in the sand that they cannot fathom a world that doesn't prefer REST anymore. At least try the tech for your own and make informed opinions.
- klabb3 3y ago
  My problem isn’t that graphql can’t do X. It’s the opposite, that I don’t know what it’s supposed to do really well – the core value prop. I still haven’t heard any one-sentence explanation of what it is, and importantly, what it isn’t. To me it sounds like they try to solve numerous problems: (1) be a query language, ie an improvement over sql, (2) a type safe alternative to REST ie web APIs (3) allow frontend developers to write queries without having to write endpoints, (4) keep clients lightweight, (5) be a single entry point to this hyper graph thing, and more. None of these things seem compelling in isolation: simply replacing REST adds resolver complexity and doesn’t handle things like file uploads. Substituting sql restricts you to whatever sql is generated and you end up with two query planners. Allowing frontend to query db directly is insecure so you must have fool proof row level security or more resolver complexity. Clients I guess started out thin but now there are all these intelligent caching layers. And this hyper graph thing is only valuable with other graphql services. So, I guess graphql can be good if you marry it and go 100% all in? But to me it’s a smell when technology claims that your pains can be resolved by using more of the same tech. How am I wrong?
  - jerrygenser 3y ago
    It's perfectly fine to use graphql for fetching metadata and then using REST API for file upload and downloads. On a gql server, you implement resolvers for the schema. There might be some frameworks that abstract SQL away from the resolver implementation but a resolver may need to make a combination of sql and network calls to other services. Or only network calls to other services. > Allowing frontend to query db directly is insecure so you must have fool proof row level security or more resolver complexity. I'm not sure why you are thinking that graphql means the frontend is querying the db directly; a graphql server implementation is implement a callback for each graph node that returns the object. It's the server code logic to implement permission checks and the actual SQL or network calls to resolve the object. What part of this results in frontend code querying SQL directly? It seems like you are getting caught up in the name of "query language" which may be partially a misnomer. It's more like a data fetching protocol
villgax 3y ago
Not once have I seen a graphql blog/usage since like 2018 onwards, just die already
sidcool 3y ago
The main article does not mention in detail why the decision was taken to use GraphQL. But the HN comments are a treasure trove right here.

235 points by theptip 3y ago

I wonder how many hidden n+1 problems there are. Reading the blog post, it looks like they mostly only tested existing pathways as part of the migration. Who knows what random requests could do.

gavinray 3y ago
This is Netflix, not some amateur-hour implementation copied from a blogpost. The de-facto GraphQL Java library comes with dataloaders built-in anyways: https://www.graphql-java.com/documentation/batching/ https://www.graphql-java.com/documentation/batching/
- voz_ 3y ago
  While Netflix has a good reputation, we should not give blanket benefit of the doubt, especially since the parent comment was a rather interesting and inquisitive one.
rand_r 3y ago
Yes, this is the biggest problem with GraphQL that I haven't addressed anywhere. The architectural beauty of GraphQL is that you can write isolated, functional nodes that only have to know about their object type. For example, the User node only has to know how to get the user's last_login, active status, and return a reference to another node type like a profile Image. Inside the User node, you are free to write code (e.g. Java or Python) that can compute which Image node is the right one to return. i.e. you can write: class UserNode: def get_user_profile_image() -> ImageNode: return some_arbitrary_function() There is no way I've seen to square this kind of arbitrary node computation with the desire to write an optimized SQL query that selects all needed information in one big-ass statement.
- valenterry 3y ago
  The exact same problem exists in an typical REST api as well. In both cases you have to add a specialized endpoint to retrieve multiple results.
- striking 3y ago
  You could use https://github.com/join-monster/join-monster https://github.com/join-monster/join-monster, but setting that aside it's usually enough to use Dataloaders that resolve based on some source ID, with some server-local caching on top if latency becomes an issue.
727374 3y ago
Dataloader (https://github.com/graphql/dataloader https://github.com/graphql/dataloader) eliminates many n+1 issues. Basically, entity requests are queued up during evaluation of the query and then you can batch them up for fulfillment. It’s a great addition to other asynchronous contexts as well. WRT random requests, there are libraries for query cost estimation you can use as a gate.
- theptip 3y ago
  Interesting, hadn’t seen that package, thanks. The general pattern seems quite useful, is it implemented in any SQL ORMs? Seems like you could do the same batching/coalesce strategy for async Postgres for example, but I don’t see anything after a quick scan of the docs in SQLAlchemy. (Seems like it would be feasible since they already batch requests in unit-of-work, they just don’t coalesce to bulk operations AFAICT.)
  - IceDane 3y ago
    Using something like `pothos` for your graphql backend, you can get tight integration with `prisma`, which will practically eliminate any N+1 issues. For fields which hit external services, you can define types as "loadable" so that every time those are requested in a batch, they are loaded efficiently to avoid n+1.
- latchkey 3y ago
  Correct, there are solutions out there, but unknown if they implemented them.

nf17 3y ago

Is JS/Express/Apollo still the goto graphQL server implementation or C#(HotChocolate), Java, go etc are widely used too? On the client side(for mobile), AFAIK iOS(Swift) and Android(Kotlin) does not support GraphQL natively? are there 3rd party libraries that can be used? Edit: Added Apollo

gavinray 3y ago
The "go-to" GraphQL server implementation varies based on language. Here are the popular ones to my knowledge: - Node.js: GraphQL Yoga + Fastify, Apollo Server - Java/Kotlin: Netflix DGS (built on top of Spring Boot) - Scala: Caliban, Sangria - Python: Strawberry, Graphene Django - C#: HotChocolate > "On the client side(for mobile), AFAIK iOS(Swift) and Android(Kotlin) does not support GraphQL natively?" GraphQL queries are just HTTP POST queries with a JSON body. They're supported everywhere. If you want specialized tooling for them, Kotlin and Swift both have great strongly-typed GraphQL libraries. Apollo publishes libraries for both: - https://www.apollographql.com/docs/kotlin/ https://www.apollographql.com/docs/kotlin/ - https://www.apollographql.com/docs/ios/ https://www.apollographql.com/docs/ios/
- a_wild_dandan 3y ago
  It appears that fastify can't keep their website up.
  - preommr 3y ago
    Apparently something got messed up with their cloudflare setup and the person with access is at a wedding or something. https://twitter.com/fastifyjs/status/1670145210347577344?cxt=HHwWgICzoYqOxq0uAAAA https://twitter.com/fastifyjs/status/1670145210347577344?cxt...
- migf 3y ago
  Apollo is also available for Elixir, where I have seen it much loved.
- re-thc 3y ago

revskill 3y ago

How to avoid N+1 query ? You don't join SQL.

x86x87 3y ago
brave to assume they are avoiding this problem :/
lolive 3y ago
I am not sure I understand that n+1 select issue with graphQL. On my project, we generate SPARQL queries [to simplify, it is not so different from a GraphQL query] and we send it to a SQL translator that then sends that SQL to a Spark cluster. It is reasonably efficient, and extremely flexible. Of course, this is a single database use case, but I don’t see any n+1 select involved here. And I do not understand how single database graphQL servers cannot work exactly the same.
ptramo 3y ago
The most common practice is to turn N+1 into 1+1 using dataloaders (https://github.com/graphql/dataloader https://github.com/graphql/dataloader for JS, there are equivalents for most implementations). The N resolvers invoke a single batched loader which receives a list of keys and returns a list of values.

toomim 3y ago

Falcor was cool. I'm sad seeing it go away.

jFriedensreich 3y ago
I would not say it is more going away now than in the last 2 years of not getting upstream support from netflix. If anything this step at least makes clear this is a dead end at netflix and we can think about how to go forward with what is/was great about falcor!

itake 3y ago

I don’t see the value ins Graphql. Most apis rarely change and the complexity of supporting queries is not worth the time saved. Graphql was a cool experiment, but I’m sticking with rest/protobufs

devmor 3y ago
For an API that centers around filters and nested subfilters to retrieve exponentially specific data, graphql can end up saving you a lot of work. For anything else, it seems pointless.
rand_r 3y ago
The original motivation was to reduce the number of network requests made on cellular network-connected mobile devices, both to save on power consumption and reduce latency for the user. It off-loads a lot of complexity and computation from the client to the server, and make sense if want to create as good a mobile experience as you can. In terms of code architecture, it drastically simplifies frontend code, and overall reduces backend code and boilerplate. It's extremely elegant and nice to use over creating custom endpoints for every use-case, but the N+1 SQL query problem is a giant thorn.
- itake 3y ago
  You can still design an immutable REST api to match the client's needs. Apollo has really nice out-of-the-box caching that can be tricky to replicate with rest. I see your point about reducing network requests (graphql lets you co-mingle mutations and queries), but I don't think that feature is used often enough, b/c what if one query fails, then everyone has to fail).
  - evangow 3y ago
    You can make a REST endpoint match a client's needs, but if you have multiple clients (for example, 1 for mobile and 1 for desktop) that all need different amounts of data (for example, the mobile client shows a simplified view with less data), then you would need to write 2 different REST endpoints to handle each clients. Multiply that by the number of pages with a difference.

x86x87 3y ago

I never understood why people go for graphql. In my experience, it's a nightmare to 1/ secure 2/ version 3/ ensure qos Securing it properly should make it a no-go in like 95% of cases. you get amazing flexibility on the front end with a heavy heavy cost on the backend. Also, in general being able to say "i want to do whatever" and request everything at once is an anti-pattern IMHO, especially after HTTP2 became mainstream and doing multiple requests is reasonably fast. Versioning? Forget about it. Now you have "all the versions" and good luck figuring out what is used, what is not used or god forbid deprecate something. you're going to have a bad time. As far as qos goes, if any client can range from "i'm asking a simple thing" to "i would like the whole world please - and btw you're only going to figure things out as you pull them in" qos becomes a pipe dream. Last, I don't understand what Netflix does that is so complex that would warrant something like gql. I just don't. To the naive developer in me it seems that they 1) need to have a basic api to get whatever catalog they have + a few apps build on top of that 2) have figured out how to do streaming exceptionally well in order to scale to all the people watching. That's is and although a gross oversimplification I cannot think about a scenario where gql is needed. I think in their desire to reinvent the wheel Netflix has jumped the shark. A lot of really good ideas have come out of the innovation they did in the past but lately it seems like they are doing things for the sake of doing them or they are pissing people off with password household policies and whatnot.

downWidOutaFite 3y ago
My main issue with it is client-side caching doesn't map cleanly to graphql. I want well-defined objects to cache.
- gareve 3y ago
  that's solved with Relay's graphstore
  - x86x87 3y ago
    There's a solution to everything /s We started with a somewhat simple problem and now let's keep throwing things in. How about something I can just curl? Is that too much to ask?

sibeliuss 3y ago

Definitely not working anywhere that doesn't have a GraphQL api. Once you've used it at scale there's no going back.

runeks 3y ago
Go on…

jFriedensreich 3y ago

This is a sad but expected day. For years i seem to have been the last lonely falcor user who is still excited about this and thinks this is one of the best abstractions for a key class of state management applications, that are not a great fit to graphQL at all. - Falcor can be schemaless with optional json schema where it makes sense - Falcor supports simpler to reason about and implement features that map perfectly to javascript objects so you can do zero boilerplate databinding with ES proxies without writing query and mutation wrappers - Does not try to be a qyery language but only expose things you need to request the data that needs to be rendered in the UI I think not even the creators of falcor fully understood how perfect it is if you just add in the last missing piece of fully automating the connection from the falcor model to the place in the UI templates where the data is needed. Most people tried doing this manually in component boilderplate which lead initially to preloading tens of json paths eg falcor.preload([`todos.done.length`,`todos.done[0..10].name`, `todos.done[0..10].date`, ...etc.]) and then people developing GraphQl like syntax for these eg. somthing like load(` todos { done 0...10 {name, data} } etc. `) at which point devs thought they might as well be using graphQL and get more features The key mistake was that you need to instead think about your UI components ITSELF as the data quueries eg you can automatically derive all falcor quieries from a component like this without ANY boilerplate, and this is only possible with falcor, as everything else has so many features that it would not be possible to map the UI code to the intended data queries {#each todos as todo} <div>{todo.name} ({todo.date})</div> {/each} It does not look like netflix will hand over falcor to a community and the codebase needs lots of work especially around tooling and overuse of some patterns like a custom observable implementation but i am still on the fence about building a new minimal derived library or forking

CSDude 3y ago

Maybe now GraphQL api can stop auto completing movies that does not exist in their catalog.

paulddraper 3y ago
? It autocompletes so you find out more quickly that Netflix doesn't carry it, and what similar options are I get that you are annoyed Netflix didn't have the movie you wanted, but autocomplete doesn't make that worse.
- 9dev 3y ago
  You’re right, but it’s annoying for sure. Instead of telling me they don’t have a title, they complete its full name (as if to say „I know what you want, but I’m not gonna give it to ya“), yet show me some arbitrary stuff named similar. It’s just a poor solution.
  - kiwijamo 3y ago
    Usually, the search results are movies with similar genre to the one I was looking for, not just simply similar spelling etc. I assume they look up stuff like what other movies people who had seen the searched-for movie have seen. So if I was looking for Movie A and Netflix knows that people who've seen Movie A tend to like Movie B, C, D, etc (and these are available in your part of the world) it'll show these movies instead. I think this is nicer and better than just failing with zero results.
- CSDude 3y ago
  I don't look for similar movies, I look for the exact movie I want, it's just a redirection of their poor catalog. If that was the intended use case, they'd have proper catalogs to drill down instead of a very long, arbitrary vertical canvas of always changing, non consistent categories.
  - paulddraper 3y ago
    Sounds like a personal preference. When I search for "How to Train Your Dragon 2", I find Netflix doesn't have it. But it has the original, plus a few series, so I turn on one of those instead.

saurik 3y ago

Why? What problem did this solve for them? I use Netflix a lot, and I use it on AppleTV where I get automatic updates to the device and the app... the app has never changed except to get slower over the years (and I mean a lot slower). There is absolutely no new functionality they have ever deployed... isn't the app "done"? Is there some serious scaling issue on their backend that is causing the perceived frontend slowness that this will fix? Is this going to massively decrease the costs of the backend? Or is it just that the engineers are all bored and can't accept the idea that the product is either 1) done or 2) needs a new direction (instead of some new tech)?

mkwarman 3y ago
I have definitely noticed and been frustrated by what you're calling out here. There always has to be something to work on, even if that thing might invalidate a significant amount of previous effort or cause regression. As a developer myself, I guess it is good for me, because it means job security. And maybe that's why there's always something to be added or changed or re-created on the products I've helped develop - to ensure all the individual contributors have something to keep them busy. But it definitely is frustrating that it seems like an app being called "done" is extremely rare nowadays. Though I suppose developers didn't really have the choice to continue development indefinitely back before the internet allowed "miscellaneous bug fixes and feature updates" patches to be pushed weekly. Edit: I just noticed who I was replying to. Thanks for your work on Cydia and in the iOS jailbraking scene at large. I have lots of fond memories jailbraking and installing all kinds of weird tweaks on my iPhone 4 back in the day.
Dudester230602 3y ago
It is surely solving a promotion problem and a bonus-getting problem for someone in their hierarchy. Plus it is making the world a better place through constructing elegant hierarchies for maximum code reuse and extensibility and through minimal message-oriented transport layers.
thephyber 3y ago
> isn't the app "done"? Netflix is an ecosystem, not just the frontend app for Apple TV. Netflix is pretty close to a vertical monopoly for some of its content. Fund-raising, planning, development, post-production, translations/dubbing, preparation for distribution, and distribution. Your idea of viewing a video content on the app is only the last 20%. Also worth reminding people that the AppleTV app is not the only app Netflix supports for viewing. There have been dozens/hundreds of different apps and most are still actively supported. Also, they are a company, so they are always looking to be able to optimize and cut costs. It’s worth staying curious and asking why before assuming you know more than the engineers who have to work “behind the curtain”.

valenterry 3y ago

I have a lot of experience with GraphQL and I love it. It saves a ton of development time. However, for those who wonder what the disadvantages are, here are some. They are from real world production experience and they are not what people usually claim wrongly (like graphql would be harder to secure/version, ...) No, some real pain points are: 1. Lack of a Map-type. There is no native type to return a list/array of objects but not a map with unique keys or even a set instead of a list. You can fallback to "convention" or custom types (like plain json) but it sucks. 2. Uploading big files (in chunks) is much more tricky than expected 3. There are union-types for the returned data. For example, I can query for an animal and get either a cat or a dolphin and if I use typescript or another capable language then the compiler will tell me that. But I cannot send such a request to the server, saying "here I send you an animal, the data is this ...". The server will have to provide a separate query/endpoint per animal. Very very annoying. 4. While graphql is awesome when it comes to batching requests, what it is not so awesome at is to improve performance by allowing the server to return the part of the data is already resolved. So while reducing the number http requests, now the previous slowest http request will become the bottleneck for all. This means the page might load subjectively faster because some parts are rendered later than possible. There is work going on here (like @defer) but it's not fully spec'd and also the solution doesn't work for some cases There are more points but this gives you some idea of the ACTUAL problems. Otherwise, from development and setup time over maintenance, analytics, versioning/deprecation, tooling and so on, graphql is much better than your standard REST api in my opinion.

scotty79 3y ago
> Lack of a Map-type. Why do you need special map types for transferring maps? Doesn't list of key-value pairs suffice? Also, why am I wrong to even ask the above questions?
- tadfisher 3y ago
  A (single-level) Map is generally understood to have unique keys, which allows for some invariants derived from this property, such as a one-to-one or one-to-many object relationship, without runtime checks in client code. I do think it's worthwhile to ask the question, though. I have a strong personal preference to avoid encoding invariants like this in wire representations, because they are properties of the _schema_ and not the wire type (JSON object, in my most commonly-encountered case). Of course, the entire point of GraphQL is to encode invariants in the data schema, so it's entirely reasonable to ask for a built-in Map type instead of building your own for every project.

gumballindie 3y ago

Curious how they handle GQL injections. Most devs using graphql have no clue what they are as the majority just cargo cults new frameworks to keep themselves busy.

valenterry 3y ago
What are GQL injections? You mean something like SQL injections that come through the graphql API?
- gumballindie 3y ago
  Exactly. Suppose you have a front end system that captures user input, and then the backend communicates with another system using the input data. If not properly escaped and is concatenated then it can lead to an injection attack just like regular sql.
  - valenterry 3y ago
    I find the terminology strange. The exact same can happen with a standard REST api. In fact, it's even more likely since there is no type system to assist there. Would you then call it REST injections too?

cmgriffing 3y ago

I think the important thing for people to recognize is that Falcor, the tool used in their mobile apps, was already GraphQL-like. So, this isn't a giant change.

cakoose 3y ago

> We migrated Netflix’s mobile apps to GraphQL with zero downtime Is "zero downtime" relevant here? For example, when migrating a database that is the source-of-truth for data, then yeah, downtime is something you have to contend with. But for a client-server API change, isn't it trivial? Companies do this all the time -- expose a new version of an API then gradually migrate clients over. (It's definitely not a trivial task overall! For example, you need thorough testing to make sure you don't break existing functionality and the article does talk about that.)

why-el 3y ago
I don't know, you said isn't trivial then you said it's not trivial! :D It's a colossal effort if you are at Netflix size, they support probably the largest number of client devices on the planet.

lumb63 3y ago

This is cool, but why? What’s the business use case or problem they’re trying to solve? I would have liked to hear about that. Without that critical piece, it sounds like Netflix engineers spent a lot of time and effort just to migrate to a new framework. It also bears mentioning that Netflix is, or pretty much should be, complete. They don’t need to make any changes to the service; if so, it’s very few. I’m sure they need talented engineers to scale up their operations, and great SREs, but what are they still writing code for?

XorNot 3y ago
This has been a perennial question about Netflix for a while now: they've tended to want to brand themselves as a tech company, but that hasn't been true for a long time now and was questionable to start with. If they were a tech company that'd be a SaaS provider for video streaming, in which case all of this would make sense - the proliferation of streaming providers certainly provides the use-case that someone could offer that as a branded service. But the time for that has come and gone as well - the other streaming services exist, and they work fine.
WorldMaker 3y ago
The business case seems to be to finally kill Falcor [1] which had a lot of similarities to GraphQL but a much smaller maintenance and developer community than GraphQL and I would assume looked a lot like tech debt to Netflix at this point. [1] https://github.com/Netflix/falcor https://github.com/Netflix/falcor

ralphstodomingo 3y ago

A lot of the sentiment coming from non-gql people in this thread start with: "gql is impossible to X, Y, Z" which is immediately untrue by inference if you base it from how well existing companies in the wild have adapted to it, and provable on your own if you just went out and tried it yourself. "from what I can see in the landing page, REST can do everything it does anyway" is partially true, but is not useful, because how else would have gql picked up adoption if it can't provide similar benefits as REST? I'm smelling a lot of cope from people who have buried their heads so deep in the sand that they cannot fathom a world that doesn't prefer REST anymore. At least try the tech for your own and make informed opinions.

klabb3 3y ago
My problem isn’t that graphql can’t do X. It’s the opposite, that I don’t know what it’s supposed to do really well – the core value prop. I still haven’t heard any one-sentence explanation of what it is, and importantly, what it isn’t. To me it sounds like they try to solve numerous problems: (1) be a query language, ie an improvement over sql, (2) a type safe alternative to REST ie web APIs (3) allow frontend developers to write queries without having to write endpoints, (4) keep clients lightweight, (5) be a single entry point to this hyper graph thing, and more. None of these things seem compelling in isolation: simply replacing REST adds resolver complexity and doesn’t handle things like file uploads. Substituting sql restricts you to whatever sql is generated and you end up with two query planners. Allowing frontend to query db directly is insecure so you must have fool proof row level security or more resolver complexity. Clients I guess started out thin but now there are all these intelligent caching layers. And this hyper graph thing is only valuable with other graphql services. So, I guess graphql can be good if you marry it and go 100% all in? But to me it’s a smell when technology claims that your pains can be resolved by using more of the same tech. How am I wrong?
- jerrygenser 3y ago
  It's perfectly fine to use graphql for fetching metadata and then using REST API for file upload and downloads. On a gql server, you implement resolvers for the schema. There might be some frameworks that abstract SQL away from the resolver implementation but a resolver may need to make a combination of sql and network calls to other services. Or only network calls to other services. > Allowing frontend to query db directly is insecure so you must have fool proof row level security or more resolver complexity. I'm not sure why you are thinking that graphql means the frontend is querying the db directly; a graphql server implementation is implement a callback for each graph node that returns the object. It's the server code logic to implement permission checks and the actual SQL or network calls to resolve the object. What part of this results in frontend code querying SQL directly? It seems like you are getting caught up in the name of "query language" which may be partially a misnomer. It's more like a data fetching protocol

villgax 3y ago

Not once have I seen a graphql blog/usage since like 2018 onwards, just die already

sidcool 3y ago

The main article does not mention in detail why the decision was taken to use GraphQL. But the HN comments are a treasure trove right here.