4 ms·

Leaking data is one of the issues with all those CRUD framework tutorials. It is nice marketing: look how easy it is to output entities with our framework! Th

by thdrdt 6y ago

Leaking data is one of the issues with all those CRUD framework tutorials. It is nice marketing: look how easy it is to output entities with our framework! The problem is: you almost never want to output complete entities. You want to output data based on the user's role/rights and on the context. I wish frameworks would put more focus on this.

silviogutierrez 6y ago
This is a major problem with REST / API / SPA-based applications. When you build around the concept of resources, you either get lazy or forget to build individual endpoints for each object. That is, it's not just "/api/people/" that shows everything. You'd need: "/api/people/matched/" with a few fields, and /api/people/before-match/" etc. One end point per use case. Almost like one endpoint per... screen. Suddenly, the old school MVC starts to make more sense. Your view dictates what data you need only for that view. Across resources. And it factors in entitlements. Often, people think this means no AJAX or SPA. That's not quite right, they are orthogonal. What you really want is a data-on-the-wire protocol but with RPC calls that are still domain based rather than resource based. Everything should be its own endpoint. Django Forms and Formsets, and the Rails and Laravel equivalents are much closer to this than REST (even if they can be RESTful themselves). Think "UpdateProfile" instead of "UserResource". The UpdateProfile may have a lot more fields, some which don't even belong on the User database table. Lot of rambling here, but I've been meaning to consolidate this in a blog post. Some related thoughts here: https://news.ycombinator.com/item?id=21875331 https://news.ycombinator.com/item?id=21875331
- arkanciscan 6y ago
  In REST you return all the data because it's better to have some geek reading about which girls rejected him than to have to create a ticket with the backend team to get them to add it later #tea
- derangedHorse 6y ago
  I agree with you up to the point of RPC calls. In general it doesn't matter what specific technology is used, as long as each data source is structured around the use-case and not around backend details like how entity models are actually stored. A lot of these ideas are captured well by Robert Martin ("Uncle Bob") in his book Clean Architecture
  - silviogutierrez 6y ago
    I guess I just use RPC as a catch all for "server call that could involve resources, actions, effects, etc." So even a REST PUT is an RPC in this world of mine.
  - sixdimensional 6y ago
    I mentioned the Onion Model above, I saw your comment about Clean Architecture, which is a refinement of the Onion Model. I think we might already have all the architectural solutions we need, we just haven't put together how to express the architectures we want in the new technologies we have. Some combination of the onion model/clean architecture, CQRS, APIs, data federation like GraphQL and microservices is probably how it can all go together. I know I am mixing implementations and patterns together in this sentence, but am just trying to express patterns and examples of things that have pieces of those patterns implemented.
- sixdimensional 6y ago
  Actually, there is a simple and good way to think about this. We often espouse the "onion model of applications" describing the responsibilities of different layers of applications. Just apply that here - some API endpoints should be backend "system APIs" offering as much access to entities as possible - think "data layer APIs". When you want to expose data to consumers for use in a UI, these are "experience APIs". These are APIs that might be built as part of the backend that directly interacts with the frontend. Mulesoft talks about this a lot [1]. I am a user of it at work, not necessarily a big fan, but I think their architecture idea is applicable anywhere and helpful for thinking about layers of APIs in a large system. It's similar to the onion model, just applied to APIs. Now, if you are building a smaller system this might all seem overkill. For a small system it might be true, unless you believe it will scale and need to be architected like this from the beginning. I think scale of the end result should always be factored into architecture decisions, because not one size fits all. I think personally there is a model out there that hasn't been expressed fully/fully realized, that integrates ideas of APIs, GraphQL, CQRS, microservices and the onion model. [1] https://blogs.mulesoft.com/dev/api-dev/what-is-api-led-connectivity/ https://blogs.mulesoft.com/dev/api-dev/what-is-api-led-conne...
  - silviogutierrez 6y ago
    I'll have to give this a read. Many of these technologies (GraphQL) seem like layers on top of layers. Really, I just want a page with 3 fields specific to that page. They may come from 3 different tables. GraphQL's original solution to this was to batch the request to 3 different resources, but the underlying problem was still there. Of course, you can make ad-hoc GraphQL equivalents to a random REST ad-hoc /api/get-info-for-my-profile-page/. The problem becomes especially noticeable when you want to update nested resources. Say, a profile's subscription status. Instead, in "MVC" world, I would just bump up the subscription status to be a first class field on that view. So no profile->subscription->status. Just "subscription_status". Hope that makes sense. So far, I've reverted to using Django Forms (across tables) and breaking them down to JSON to use them in a SPA. My endpoints are still very much domain specific, but I can use XHR/fetch to make the page interactive.

by thdrdt 6y ago

silviogutierrez 6y ago

This is a major problem with REST / API / SPA-based applications. When you build around the concept of resources, you either get lazy or forget to build individual endpoints for each object. That is, it's not just "/api/people/" that shows everything. You'd need: "/api/people/matched/" with a few fields, and /api/people/before-match/" etc. One end point per use case. Almost like one endpoint per... screen. Suddenly, the old school MVC starts to make more sense. Your view dictates what data you need only for that view. Across resources. And it factors in entitlements. Often, people think this means no AJAX or SPA. That's not quite right, they are orthogonal. What you really want is a data-on-the-wire protocol but with RPC calls that are still domain based rather than resource based. Everything should be its own endpoint. Django Forms and Formsets, and the Rails and Laravel equivalents are much closer to this than REST (even if they can be RESTful themselves). Think "UpdateProfile" instead of "UserResource". The UpdateProfile may have a lot more fields, some which don't even belong on the User database table. Lot of rambling here, but I've been meaning to consolidate this in a blog post. Some related thoughts here: https://news.ycombinator.com/item?id=21875331 https://news.ycombinator.com/item?id=21875331

arkanciscan 6y ago
In REST you return all the data because it's better to have some geek reading about which girls rejected him than to have to create a ticket with the backend team to get them to add it later #tea
derangedHorse 6y ago
I agree with you up to the point of RPC calls. In general it doesn't matter what specific technology is used, as long as each data source is structured around the use-case and not around backend details like how entity models are actually stored. A lot of these ideas are captured well by Robert Martin ("Uncle Bob") in his book Clean Architecture
- silviogutierrez 6y ago
  I guess I just use RPC as a catch all for "server call that could involve resources, actions, effects, etc." So even a REST PUT is an RPC in this world of mine.
- sixdimensional 6y ago
  I mentioned the Onion Model above, I saw your comment about Clean Architecture, which is a refinement of the Onion Model. I think we might already have all the architectural solutions we need, we just haven't put together how to express the architectures we want in the new technologies we have. Some combination of the onion model/clean architecture, CQRS, APIs, data federation like GraphQL and microservices is probably how it can all go together. I know I am mixing implementations and patterns together in this sentence, but am just trying to express patterns and examples of things that have pieces of those patterns implemented.
sixdimensional 6y ago
Actually, there is a simple and good way to think about this. We often espouse the "onion model of applications" describing the responsibilities of different layers of applications. Just apply that here - some API endpoints should be backend "system APIs" offering as much access to entities as possible - think "data layer APIs". When you want to expose data to consumers for use in a UI, these are "experience APIs". These are APIs that might be built as part of the backend that directly interacts with the frontend. Mulesoft talks about this a lot [1]. I am a user of it at work, not necessarily a big fan, but I think their architecture idea is applicable anywhere and helpful for thinking about layers of APIs in a large system. It's similar to the onion model, just applied to APIs. Now, if you are building a smaller system this might all seem overkill. For a small system it might be true, unless you believe it will scale and need to be architected like this from the beginning. I think scale of the end result should always be factored into architecture decisions, because not one size fits all. I think personally there is a model out there that hasn't been expressed fully/fully realized, that integrates ideas of APIs, GraphQL, CQRS, microservices and the onion model. [1] https://blogs.mulesoft.com/dev/api-dev/what-is-api-led-connectivity/ https://blogs.mulesoft.com/dev/api-dev/what-is-api-led-conne...
- silviogutierrez 6y ago
  I'll have to give this a read. Many of these technologies (GraphQL) seem like layers on top of layers. Really, I just want a page with 3 fields specific to that page. They may come from 3 different tables. GraphQL's original solution to this was to batch the request to 3 different resources, but the underlying problem was still there. Of course, you can make ad-hoc GraphQL equivalents to a random REST ad-hoc /api/get-info-for-my-profile-page/. The problem becomes especially noticeable when you want to update nested resources. Say, a profile's subscription status. Instead, in "MVC" world, I would just bump up the subscription status to be a first class field on that view. So no profile->subscription->status. Just "subscription_status". Hope that makes sense. So far, I've reverted to using Django Forms (across tables) and breaking them down to JSON to use them in a SPA. My endpoints are still very much domain specific, but I can use XHR/fetch to make the page interactive.