4 ms·

I love new technology, and this is really cool. I don't want to derail the discussion too much, but I've noticed this trend of "making web dev easier with just

I love new technology, and this is really cool. I don't want to derail the discussion too much, but I've noticed this trend of "making web dev easier with just Python" - e.g. htmx, pynecone, streamlit, etc. The common theme seems to be magically hiding JS from the developer. Can any python devs who use these tools explain the appeal? Do you just have an aversion to JS? And how do you reconcile that with the fact that, at the end of the day, your code is running in the browser as JS, CSS, and HTML? As soon as you need to debug something, you'll be a fish out of water. To me that sounds like a "worst of both worlds" kind of situation. Is JS really that bad?

hkhanna 4y ago
I use htmx, and I have a good amount of experience in the JS world. The main issue with JS frameworks like React is that it's painful to integrate into things like Django-rendered templates. It can definitely be done, but the tooling just isn't there yet, although it's way ahead of where it was even 5 years ago. You can go the SPA route and use something like Django Rest Framework with React, but then you need to build API endpoints for basic plumbing like authentication that often doesn't warrant an API. And you lose the use of a lot of the batteries-included features of frameworks like Django. If there were an ergonomic way to integrate React components into specific areas of the DOM tree rendered by Django templates and a sensible way to pass information into props from Django, I'd love to use a mature framework like React over things like htmx.
- robomartin 4y ago
  I've been away from direct work with Python + Django (P/D) for a couple of years. One of my issues with P/D has always been that deployment on anything other than one of the specialized services is an absolute pain in the ass. In the PHP world, you can get a simplest of servers from any host, upload the files and you are up and running. With P/D, well, it's a lot of work. I have developed this opinion that, while useful, Django's SQLite and runserver approach for getting up and running quickly actually is detrimental. To run a real site you have to break all of that, deal with static content, etc., etc. I could also argue that the default directory structure, while simple, it is also detrimental when moving to production. In other words, sometimes I feel it would be far better if P/D initial setup was closer to a canonical installation that could be run on any server with something as close as possible to a simple FTP file transfer (just like PHP). One can always make things more complex (multiple servers, load balancers, etc.). However, I truly believe the P/D ecosystem is seriously missing the point when not delivering a 5-minute install that actually is a basic production-ready install on any server. Have things changed at all in this regard over the last couple of years?
  - hkhanna 4y ago
    That's true, and PHP certainly has ease of deployment over Django. I generally deploy Django applications on services like Heroku or Render, but it can get expensive since you can't really re-use instances for multiple applications like you could with a VPS and multiple PHP applications. But, at least to me, ease of deployment is less of a concern once it's going. And since I stick to one hosting provider and only have a handful of applications I maintain, its not something that crosses my mind until I spin up a new Django application which happens only rarely.
    - chatmasta 4y ago
      I often joke that Docker wouldn't exist if it had been easier to deploy Python code, since it was DotCloud who invented Docker as part of solving that problem. I was hoping Vercel would be their spiritual successor, back when they were known as Zeit and allowed running arbitrary containers at the edge, but alas they pivoted to focus only on the JS platform. That said, IMO Next.js is the closest thing to modern day PHP in terms of filesystem-based routing and "write code on a page, render it at the same URL." For Python, we run our services in containers using some custom images, and build the code from a monorepo using a combination of Pants and Poetry. For smaller or more focused projects, I think Google Cloud Run is quite nice, since it supports the Buildpack Spec and is in many ways a successor to Google AppEngine.
- chatmasta 4y ago
  Something many people miss is that JSX and React are two separate technologies. JSX is a fantastic templating language in its own right. Have there been any libraries that attempt to integrate JSX more natively into Django templates? It seems like that might be an approach that could solve the ergonomics problem you describe.
enoch2090 4y ago
These are just frameworks that allows you to make an easy interface for your mostly-Python-based code. The problem is never that “I only want Python”, but “I want a web app but I don’t know much JS”.
woolion 4y ago
It's not so much about having another language to learn, it is about sharing code. Suppose you need a meaningful data structure to encode something very domain-specific (for example a dynamic filter you could build with arbitrary boolean operations). On the front-end side, it will be like a user-facing DSL that you can build with UI elements. On the backend a recursive algebraic data structure you need to parse and that should go through standard algorithms to obtain (most likely deterministic) results. So what happens if you have two stacks? You'll need to code a number of these parsing and processing algorithms twice, and each inconsistency will be a bug. Every time we had the need for something like that in a project, it was done in the backend easily and correctly, but the frontend was lagging and missing features. So even if the backend language is transpiled to run in a browser, it does change the game to be able to run the correct algorithms directly.
- chatmasta 4y ago
  My co-founder and I originally divided our backend/frontend work between each other, and between Python/TypeScript. We found the sweet spot for solving this code sharing problem was GraphQL with code generation on both sides (resolvers from types on the Python side, and types from schema on the TypeScipt side). Most backend services are in Python, using Strawberry for strongly-typed GraphQL resolvers, and client code is in Next.js using TypeScript, Apollo GraphQL, and graphql-codegen for autogenerated types from the GraphQL schema. (We also use Postgraphile as a sort of naive base layer for basic operations, and later added schema stitching to have one GraphQL API in front of multiple services, but that's a longer story [0]). In our company we generally divide the work between frontend and backend developers, and we've found that we usually complete the backend implementation for most features before the frontend anyway, so the GraphQL schema itself becomes part of the spec. But more importantly, GraphQL becomes a unified type layer shared between languages. Backend and frontend developers can each work with the most native features of their respective languages, rather than some subset of each that's supported by the other. The code generation is the secret ingredient, because it lets us draw a straight line of automation from Python types to TypeScript types, with no need to manually write any middleware. Usually frontend and backend is implemented by a different person, but in the cases where one person is doing both, they can literally save a Python file and wait for a chain of hot reloading to have those types available in the Next.js app. It's a pretty awesome setup and I'd recommend looking into building something like it if you're working on a greenfield project and intend to grow your team of developers. But it takes a lot of effort to setup. We arrived here incrementally, originally starting with just Postgraphile and later adding the Strawberry resolvers and schema stitching. If you're an individual, it's almost certainly an unjustifiable amount of work to setup. And it might even be too complicated for an org of multiple people, if they typically have the same full stack developer implement the front and backend of a feature. But for us it's been a great experience overall. And in general, I absolutely love the combination of GraphQL and type-driven code generation, and I would recommend that even individual developers find a way to leverage that aspect of it. [0] https://www.splitgraph.com/blog/graphql-schema-stitching https://www.splitgraph.com/blog/graphql-schema-stitching
ShakataGaNai 4y ago
Is it that bad? No. But that doesn't mean I want to do it. I'm a more casual python dev. I make small tools, solve unique/corner issues. Nothing fancy. It makes my life vastly easier for many of these tools to be webbased, typically because I want to hand them to someone else to use so I don't need to think/worry about it. Context switching between languages is very very expensive for me. I used to write a lot of nodejs and Python but I'd find myself either forgetting or adding semi-colons, confusing which for-loop style for which language. Eventually I came to the conclusion it wasn't worth the mental gymnastics to switch and gave up node (aka Javscript). So can I write it? Yes, but I don't want to. These new entirely python based web apps are really interesting to me. They don't have to be super advanced because I don't need that. It greatly simplifies the process and the time required. Also, almost no one writes "just javascript" any more. It's htmlx, react, yada yada.... and they've all got some cool new build system. Seems like every time I look, there is a new build system people are using. Which I don't want to learn or deal with.
- edmundsauto 4y ago
  This describes me to a T. I think if GP saw the js I write, they would understand why I don’t want to write it. But it works, I can get fairly straightforward stuff done quickly, and I’m the only jerk who has to maintain it.
jollyllama 4y ago
>Is JS really that bad? No, but npm is. If you shelve your vue/react/etc project for six months, then go to build it on a new machine, you'll find all kinds of things are broken. Contrast with python and one or two smaller js libraries, which will not require any updates or fixes for much longer.
- chatmasta 4y ago
  > If you shelve your vue/react/etc project for six months, then go to build it on a new machine, you'll find all kinds of things are broken. This is not true in my experience (I literally did exactly what you describe recently). I use Yarn, with pinned dependencies and yarn.lock checked into the repo. Everything works exactly as expected after 6 months, because the hashes of all the packages match each other. You just need to make sure to run `yarn install --immutable` when you come back after 6 months, to make sure you don't update the lockfile (and pinning dependencies to their exact versions helps avoid installing a different version within the same semver range). Now, whether everything will "just work" after upgrading one of the dependencies is a different story. But if you don't touch package.json or yarn.lock, then everything should work fine. > Contrast with python I'm not sure Python's dependency management is a great hill to die on. But Poetry is pretty good. In our project's monorepo we've settled on a combination of Pants and Poetry. But it's far from easy to setup, and I'm not sure the default behavior has the six months of stability you describe, in comparison to Yarn (but I'm not the Python guy, so I don't know the details. I've just noticed a few occasions where installing packages unexpectedly modified the lockfile; take that with a grain of salt, though.) What is your Python package manager setup (just curious)?
  - jollyllama 4y ago
    >What is your Python package manager setup For this project, literally just pip with a requirements.txt. Poetry is great though.

hkhanna 4y ago

I use htmx, and I have a good amount of experience in the JS world. The main issue with JS frameworks like React is that it's painful to integrate into things like Django-rendered templates. It can definitely be done, but the tooling just isn't there yet, although it's way ahead of where it was even 5 years ago. You can go the SPA route and use something like Django Rest Framework with React, but then you need to build API endpoints for basic plumbing like authentication that often doesn't warrant an API. And you lose the use of a lot of the batteries-included features of frameworks like Django. If there were an ergonomic way to integrate React components into specific areas of the DOM tree rendered by Django templates and a sensible way to pass information into props from Django, I'd love to use a mature framework like React over things like htmx.

robomartin 4y ago
I've been away from direct work with Python + Django (P/D) for a couple of years. One of my issues with P/D has always been that deployment on anything other than one of the specialized services is an absolute pain in the ass. In the PHP world, you can get a simplest of servers from any host, upload the files and you are up and running. With P/D, well, it's a lot of work. I have developed this opinion that, while useful, Django's SQLite and runserver approach for getting up and running quickly actually is detrimental. To run a real site you have to break all of that, deal with static content, etc., etc. I could also argue that the default directory structure, while simple, it is also detrimental when moving to production. In other words, sometimes I feel it would be far better if P/D initial setup was closer to a canonical installation that could be run on any server with something as close as possible to a simple FTP file transfer (just like PHP). One can always make things more complex (multiple servers, load balancers, etc.). However, I truly believe the P/D ecosystem is seriously missing the point when not delivering a 5-minute install that actually is a basic production-ready install on any server. Have things changed at all in this regard over the last couple of years?
- hkhanna 4y ago
  That's true, and PHP certainly has ease of deployment over Django. I generally deploy Django applications on services like Heroku or Render, but it can get expensive since you can't really re-use instances for multiple applications like you could with a VPS and multiple PHP applications. But, at least to me, ease of deployment is less of a concern once it's going. And since I stick to one hosting provider and only have a handful of applications I maintain, its not something that crosses my mind until I spin up a new Django application which happens only rarely.
  - chatmasta 4y ago
    I often joke that Docker wouldn't exist if it had been easier to deploy Python code, since it was DotCloud who invented Docker as part of solving that problem. I was hoping Vercel would be their spiritual successor, back when they were known as Zeit and allowed running arbitrary containers at the edge, but alas they pivoted to focus only on the JS platform. That said, IMO Next.js is the closest thing to modern day PHP in terms of filesystem-based routing and "write code on a page, render it at the same URL." For Python, we run our services in containers using some custom images, and build the code from a monorepo using a combination of Pants and Poetry. For smaller or more focused projects, I think Google Cloud Run is quite nice, since it supports the Buildpack Spec and is in many ways a successor to Google AppEngine.
chatmasta 4y ago
Something many people miss is that JSX and React are two separate technologies. JSX is a fantastic templating language in its own right. Have there been any libraries that attempt to integrate JSX more natively into Django templates? It seems like that might be an approach that could solve the ergonomics problem you describe.

enoch2090 4y ago

These are just frameworks that allows you to make an easy interface for your mostly-Python-based code. The problem is never that “I only want Python”, but “I want a web app but I don’t know much JS”.

woolion 4y ago

It's not so much about having another language to learn, it is about sharing code. Suppose you need a meaningful data structure to encode something very domain-specific (for example a dynamic filter you could build with arbitrary boolean operations). On the front-end side, it will be like a user-facing DSL that you can build with UI elements. On the backend a recursive algebraic data structure you need to parse and that should go through standard algorithms to obtain (most likely deterministic) results. So what happens if you have two stacks? You'll need to code a number of these parsing and processing algorithms twice, and each inconsistency will be a bug. Every time we had the need for something like that in a project, it was done in the backend easily and correctly, but the frontend was lagging and missing features. So even if the backend language is transpiled to run in a browser, it does change the game to be able to run the correct algorithms directly.

chatmasta 4y ago
My co-founder and I originally divided our backend/frontend work between each other, and between Python/TypeScript. We found the sweet spot for solving this code sharing problem was GraphQL with code generation on both sides (resolvers from types on the Python side, and types from schema on the TypeScipt side). Most backend services are in Python, using Strawberry for strongly-typed GraphQL resolvers, and client code is in Next.js using TypeScript, Apollo GraphQL, and graphql-codegen for autogenerated types from the GraphQL schema. (We also use Postgraphile as a sort of naive base layer for basic operations, and later added schema stitching to have one GraphQL API in front of multiple services, but that's a longer story [0]). In our company we generally divide the work between frontend and backend developers, and we've found that we usually complete the backend implementation for most features before the frontend anyway, so the GraphQL schema itself becomes part of the spec. But more importantly, GraphQL becomes a unified type layer shared between languages. Backend and frontend developers can each work with the most native features of their respective languages, rather than some subset of each that's supported by the other. The code generation is the secret ingredient, because it lets us draw a straight line of automation from Python types to TypeScript types, with no need to manually write any middleware. Usually frontend and backend is implemented by a different person, but in the cases where one person is doing both, they can literally save a Python file and wait for a chain of hot reloading to have those types available in the Next.js app. It's a pretty awesome setup and I'd recommend looking into building something like it if you're working on a greenfield project and intend to grow your team of developers. But it takes a lot of effort to setup. We arrived here incrementally, originally starting with just Postgraphile and later adding the Strawberry resolvers and schema stitching. If you're an individual, it's almost certainly an unjustifiable amount of work to setup. And it might even be too complicated for an org of multiple people, if they typically have the same full stack developer implement the front and backend of a feature. But for us it's been a great experience overall. And in general, I absolutely love the combination of GraphQL and type-driven code generation, and I would recommend that even individual developers find a way to leverage that aspect of it. [0] https://www.splitgraph.com/blog/graphql-schema-stitching https://www.splitgraph.com/blog/graphql-schema-stitching

ShakataGaNai 4y ago

Is it that bad? No. But that doesn't mean I want to do it. I'm a more casual python dev. I make small tools, solve unique/corner issues. Nothing fancy. It makes my life vastly easier for many of these tools to be webbased, typically because I want to hand them to someone else to use so I don't need to think/worry about it. Context switching between languages is very very expensive for me. I used to write a lot of nodejs and Python but I'd find myself either forgetting or adding semi-colons, confusing which for-loop style for which language. Eventually I came to the conclusion it wasn't worth the mental gymnastics to switch and gave up node (aka Javscript). So can I write it? Yes, but I don't want to. These new entirely python based web apps are really interesting to me. They don't have to be super advanced because I don't need that. It greatly simplifies the process and the time required. Also, almost no one writes "just javascript" any more. It's htmlx, react, yada yada.... and they've all got some cool new build system. Seems like every time I look, there is a new build system people are using. Which I don't want to learn or deal with.

edmundsauto 4y ago
This describes me to a T. I think if GP saw the js I write, they would understand why I don’t want to write it. But it works, I can get fairly straightforward stuff done quickly, and I’m the only jerk who has to maintain it.

jollyllama 4y ago

>Is JS really that bad? No, but npm is. If you shelve your vue/react/etc project for six months, then go to build it on a new machine, you'll find all kinds of things are broken. Contrast with python and one or two smaller js libraries, which will not require any updates or fixes for much longer.

chatmasta 4y ago
> If you shelve your vue/react/etc project for six months, then go to build it on a new machine, you'll find all kinds of things are broken. This is not true in my experience (I literally did exactly what you describe recently). I use Yarn, with pinned dependencies and yarn.lock checked into the repo. Everything works exactly as expected after 6 months, because the hashes of all the packages match each other. You just need to make sure to run `yarn install --immutable` when you come back after 6 months, to make sure you don't update the lockfile (and pinning dependencies to their exact versions helps avoid installing a different version within the same semver range). Now, whether everything will "just work" after upgrading one of the dependencies is a different story. But if you don't touch package.json or yarn.lock, then everything should work fine. > Contrast with python I'm not sure Python's dependency management is a great hill to die on. But Poetry is pretty good. In our project's monorepo we've settled on a combination of Pants and Poetry. But it's far from easy to setup, and I'm not sure the default behavior has the six months of stability you describe, in comparison to Yarn (but I'm not the Python guy, so I don't know the details. I've just noticed a few occasions where installing packages unexpectedly modified the lockfile; take that with a grain of salt, though.) What is your Python package manager setup (just curious)?
- jollyllama 4y ago
  >What is your Python package manager setup For this project, literally just pip with a requirements.txt. Poetry is great though.