4 ms·

Hello, author here. Happy to answer any questions about the post if you have them.

by czue 3y ago

williamdclt 3y ago
I've built a "project generator" that worked pretty much exactly the same way (using plop.js rather than cookiecutter), generating various parts of various stacks (Django/NestJS/React/Elastic beanstalk/React Native...). It's simple enough, as you say it's mostly templating, but the hardest part I found is maintaining dependencies. I needed to keep all dependencies up-to-date, which was a big amount of manual work. For something like React-native, rather than templating I was generating a project using create-react-native-app then applying patch files. How do you manage this? Also, how's the Pegasus devx? Another annoying part of templating is that I was never really writing Python or Javascript, I was writing Jinja, so syntax highlighting or autoformatting didn't really work well or at all.
- czue 3y ago
  Yeah, dependencies are a pain. I'll do security patches ~immediately, and have gotten into a ~quarterly rhythm of just making a big push to upgrade everything all at once. Usually it's pretty painless, but sometimes it opens up a big rabbit hole. But I've found the more I stay on top of them, the easier it is. Big version jumps are always hard/scary. Re: devx... it kinda sucks, but I've made it work. Honestly, that could be a whole follow up blog post. But basically I do dev in a generated project, and then have some scripts that use git patches to apply the changes back into Pegasus itself. After that I have to add the cookiecutter/templating logic. I learned early on to do as little dev as possible in the actual generator repo, because - as you said - nothing works.
- MoreQARespect 3y ago
  >It's simple enough, as you say it's mostly templating, but the hardest part I found is maintaining dependencies. I needed to keep all dependencies up-to-date, which was a big amount of manual work. For something like React-native, rather than templating I was generating a project using create-react-native-app then applying patch files. How do you manage this? I did something like this before by building integration tests against the templater project and performed various smoke tests on them. The integration tests I would schedule once or twice a week on the latest dependencies. It was somewhat complicated by the fact that often plugin B to framework A required a maximum of version 4.1 for framework A, so at some point you had to make a decision: * Wait for plugin B to catch up to framework A before upgrading framework A. * Drop support for plugin B. It really helped having the tests run periodically to get quick notification when something broke, coz then you could quickly adjust the dependencies (e.g. change >=3.5 to >=3.5, <4.1).
japhyr 3y ago
Hi Cory, this was a fantastic writeup! Your explanation of the value of generating a codebase for people instead of having them use a project template is compelling; it articulates that aversion I've always felt towards large templated projects. The section about testing is particularly interesting to me. I'm working on django-simple-deploy [0], a project that automates people's initial deployments of a Django project. When people first hear about this project, they assume it's just for beginners. But when a lot of people started migrating away from Heroku, we saw how much work it is even for experienced developers to read through a new platform's docs and configure their project correctly for deployment. simple_deploy does all that configuration in one pass, and then all of the platform-specific configuration is contained in one commit. A fun question I've shared when talking about this project is "How do you test a standalone management command, whose goal is to act on a variety of Django projects?" In this project there's no settings file, no models, no urls, no views, etc. The project also needs to support multiple dependency management systems, multiple target platforms, and since it works on the user's local system it needs to be cross platform as well. To deal with this efficiently, the unit test suite copies a sample project to a temp directory. It then builds a venv, and runs a commit. It runs simple_deploy against the project, and then does as many tests as it can against the configuration that was done for that project. Then, instead of starting over, it uses git to reset the sample project to the original state. This lets it run ~90 tests in about 10 seconds. Even more interestingly, that temp project is still available when the test suite finishes. pytest takes care of deleting it at some point, but not immediately. So, debugging can be really efficient. If you run `pytest -x`, you can open that test project and poke around to see what didn't work. The setup uses an editable install, so you can modify simple_deploy, reset the test project manually, and run the command again (outside of the test suite). pytest is an amazing tool. Just yesterday I wrote a plugin that lets you run `pytest -x --open-test-project`. When the test suite exits, it pops open a new terminal window at the test project, with an active virtual environment and the output of `git status` and `git log`. This turns the test suite into a development tool. I'm curious if you use your test suite for development in any way, or if it's just for catching bugs and regressions? Also, with that tree of configuration options, have you integrated randomization into your tests at all? That seems like it might be a good way to hit branches you haven't tried before, but then again the majority of the 33M branches are probably not meaningful to test. - [0] https://django-simple-deploy.readthedocs.io/en/latest/ https://django-simple-deploy.readthedocs.io/en/latest/
RangerScience 3y ago
Hi hi! Nice work :) Thinking over some of the other conversation here, about the difficulties in the devx what with the templates being invalid python - Could you pull some shenanigans with https://docs.python.org/2/library/inspect.html#inspect.getsourcelines https://docs.python.org/2/library/inspect.html#inspect.getso... and siblings? I'm thinking - at least in some places - you might be able to have lists of functions/modules/etc to get copied, rather than lists of templates to get rendered. Might even be able to do some code-as-strings manipulation during the copy. The entire point would be so that the code from Pegasus that ends up in the generated project is valid Python code from the get-go, rather than only one it's rendered.
- czue 3y ago
  Interesting idea. I could see that being a viable option, though not necessarily any easier to maintain than the current situation.

Hello, author here. Happy to answer any questions about the post if you have them.

by czue 3y ago

Hello, author here. Happy to answer any questions about the post if you have them.

I've built a "project generator" that worked pretty much exactly the same way (using plop.js rather than cookiecutter), generating various parts of various stacks (Django/NestJS/React/Elastic beanstalk/React Native...). It's simple enough, as you say it's mostly templating, but the hardest part I found is maintaining dependencies. I needed to keep all dependencies up-to-date, which was a big amount of manual work. For something like React-native, rather than templating I was generating a project using create-react-native-app then applying patch files. How do you manage this? Also, how's the Pegasus devx? Another annoying part of templating is that I was never really writing Python or Javascript, I was writing Jinja, so syntax highlighting or autoformatting didn't really work well or at all.

czue 3y ago
Yeah, dependencies are a pain. I'll do security patches ~immediately, and have gotten into a ~quarterly rhythm of just making a big push to upgrade everything all at once. Usually it's pretty painless, but sometimes it opens up a big rabbit hole. But I've found the more I stay on top of them, the easier it is. Big version jumps are always hard/scary. Re: devx... it kinda sucks, but I've made it work. Honestly, that could be a whole follow up blog post. But basically I do dev in a generated project, and then have some scripts that use git patches to apply the changes back into Pegasus itself. After that I have to add the cookiecutter/templating logic. I learned early on to do as little dev as possible in the actual generator repo, because - as you said - nothing works.
MoreQARespect 3y ago
>It's simple enough, as you say it's mostly templating, but the hardest part I found is maintaining dependencies. I needed to keep all dependencies up-to-date, which was a big amount of manual work. For something like React-native, rather than templating I was generating a project using create-react-native-app then applying patch files. How do you manage this? I did something like this before by building integration tests against the templater project and performed various smoke tests on them. The integration tests I would schedule once or twice a week on the latest dependencies. It was somewhat complicated by the fact that often plugin B to framework A required a maximum of version 4.1 for framework A, so at some point you had to make a decision: * Wait for plugin B to catch up to framework A before upgrading framework A. * Drop support for plugin B. It really helped having the tests run periodically to get quick notification when something broke, coz then you could quickly adjust the dependencies (e.g. change >=3.5 to >=3.5, <4.1).

japhyr 3y ago

Hi Cory, this was a fantastic writeup! Your explanation of the value of generating a codebase for people instead of having them use a project template is compelling; it articulates that aversion I've always felt towards large templated projects. The section about testing is particularly interesting to me. I'm working on django-simple-deploy [0], a project that automates people's initial deployments of a Django project. When people first hear about this project, they assume it's just for beginners. But when a lot of people started migrating away from Heroku, we saw how much work it is even for experienced developers to read through a new platform's docs and configure their project correctly for deployment. simple_deploy does all that configuration in one pass, and then all of the platform-specific configuration is contained in one commit. A fun question I've shared when talking about this project is "How do you test a standalone management command, whose goal is to act on a variety of Django projects?" In this project there's no settings file, no models, no urls, no views, etc. The project also needs to support multiple dependency management systems, multiple target platforms, and since it works on the user's local system it needs to be cross platform as well. To deal with this efficiently, the unit test suite copies a sample project to a temp directory. It then builds a venv, and runs a commit. It runs simple_deploy against the project, and then does as many tests as it can against the configuration that was done for that project. Then, instead of starting over, it uses git to reset the sample project to the original state. This lets it run ~90 tests in about 10 seconds. Even more interestingly, that temp project is still available when the test suite finishes. pytest takes care of deleting it at some point, but not immediately. So, debugging can be really efficient. If you run `pytest -x`, you can open that test project and poke around to see what didn't work. The setup uses an editable install, so you can modify simple_deploy, reset the test project manually, and run the command again (outside of the test suite). pytest is an amazing tool. Just yesterday I wrote a plugin that lets you run `pytest -x --open-test-project`. When the test suite exits, it pops open a new terminal window at the test project, with an active virtual environment and the output of `git status` and `git log`. This turns the test suite into a development tool. I'm curious if you use your test suite for development in any way, or if it's just for catching bugs and regressions? Also, with that tree of configuration options, have you integrated randomization into your tests at all? That seems like it might be a good way to hit branches you haven't tried before, but then again the majority of the 33M branches are probably not meaningful to test. - [0] https://django-simple-deploy.readthedocs.io/en/latest/ https://django-simple-deploy.readthedocs.io/en/latest/

RangerScience 3y ago

Hi hi! Nice work :) Thinking over some of the other conversation here, about the difficulties in the devx what with the templates being invalid python - Could you pull some shenanigans with https://docs.python.org/2/library/inspect.html#inspect.getsourcelines https://docs.python.org/2/library/inspect.html#inspect.getso... and siblings? I'm thinking - at least in some places - you might be able to have lists of functions/modules/etc to get copied, rather than lists of templates to get rendered. Might even be able to do some code-as-strings manipulation during the copy. The entire point would be so that the code from Pegasus that ends up in the generated project is valid Python code from the get-go, rather than only one it's rendered.

czue 3y ago
Interesting idea. I could see that being a viable option, though not necessarily any easier to maintain than the current situation.