7 ms·

> In the end you should be aiming to design your application to depend on Flask as little as possible. I'm 3 years in to a project where I've taken the approac

by aidos 11y ago

> In the end you should be aiming to design your application to depend on Flask as little as possible. I'm 3 years in to a project where I've taken the approach of having my "app" as its own package. Flask is then a thin wrapper that instantiates the app and then exposes the views to talk to it. I handle validation / sessions / config etc within Flask. It's extremely flexible. I have some stuff that runs off as worker processes. No problem - the core is the same, the wrapper around it is different. I also have a boot.py that I can call so I'm working directly with the app in ipython (again, it's a wrapper that starts the app). I'm currently reworking the structure for capacity planning, which is mostly just a matter of splitting the Flask app into 2 apps for handling different parts of the work. Because my core app is decoupled from the web app it's all really easy to do.

odonnellryan 11y ago
Great points! I've worked with Flask for a while, as I've gotten to be a better developer. I used to have a lot of my logic in the application itself. Now, I ensure my logic lives in its own modules. Flask is really just either a way to serve my templates or API.
voltagex_ 11y ago
Maybe I'm just not thinking clearly today but how do you turn Flask into a wrapper? I thought you had to put enough logic in to a class that inherits from Flask to return views and such.
- aidos 11y ago
  I think I need to explain a bit better. I have a few different layers. From the outside in: Flask/Web Deal with sessions, authentication, caching, templating, logging etc here. Fires up an instance of the App and keeps it around for the duration of the life of the Web instance. I actually have an API in here that looks like what you'd expect in most other web apps. It does the validation / error handling etc. In terms of manipluating the data, I'm not a total purist - the API is allowed to load up the SQLA models and make changes. If there's a more complex operation going on though, it will live inside the next layer down. App Deal with remote services / data models / db connection. I guess on the outside you might call this a "service layer". A surprisingly large amount of this is data persistance management. The model I use is to put the data access stuff just off to the side at the top level of the app. Nothing in here knows anything about things like web frameworks. This is important. In other places in the system, this same App is used by RQ (worker instances taking on work via a redis queue). There's another web app that uses it for administrative tasks. I can boot it up to work with it in ipython (I do this a lot). Oh, and I don't manage the db life-cycle here. Whichever layer sits above this determines when it's ok to commit transactions. If something goes wrong here we raise app specific expections - the layer above can handle those and relay them as appropriate. Core In the very core I have "pure" algorithmic stuff (for context - http://www.countfire.com http://www.countfire.com is effectively an image recognition app). Really this is just one component that the App happens to be using. If the App wants to talk to this, it needs to create the data structures that the core defines. At various borders I often have mappings from the objects in the current local world to versions for the created to talk to the layer below (ps I like namedtuples). It's a lot more complex than all this, each of these layers has more sub layers etc. And really, what I've called the Core is a sublayer of the App (in my case there's a well definied algorithmic chunk that I think of as the core so it's in its own package - it stands alone as its own library). There's no real way of knowing where you'll draw lines between the layers when you start but I have a few rules. Any bit of code should be given the bare minimum of data required to do its work. At borders map local data to data for the next layer. Make sure the stuff going into the current layer is really there because it needs to be there. Be a pragmatist - things will change, you'll pull stuff out into submodules. By following the other rules hopefully refactoring won't be too painful.
BinaryIdiot 11y ago
Honestly this is how all development should be done. Treat the web server as yet another interface into your application to keep it as slim as possible. Need to change web servers, support thrift, avro, etc? Just write another interface that hooks into your application. Easy! There are plenty of examples in every language where a web framework turns into a web service + caching + session management + ORM + whatever else you can think of then you use all of those features and now you're inflexible, can't easily add new interfaces without seriously rewriting the old and good luck if you need to move away from that web framework for whatever reason.
- whalesalad 11y ago
  YES! Your username should be binaryfullofwisdom.
  - BinaryIdiot 11y ago
    I will take that under advisement :)
m_mueller 11y ago
I like this approach. How do you handle routes? Do you just initialize a wrapper app that is composed of your own application, wrap your views in Flask views objects and then call `add_url_rule` on them in the wrapper setup code? If I understand this correctly, how do you define something like a redirect or abort within your own views?
- aidos 11y ago
  I've tried to explain my structure a bit better in a reply to voltagex_. For your specific question - all routing is done from within the top / Flask layer (as is redirecting / aborting). My App layer does not have any Flask stuff in, at all, ever.
rahimnathwani 11y ago
"Flask is then a thin wrapper that instantiates the app and then exposes the views to talk to it." Why did you decide to go this way? Is it because the user-facing parts of the app (the bits where Flask helps) are only a small part? I mean, if I'm developing a CRUD app with small amounts of supporting non-interactive code, I can't see a reason not to use Django (and put the other stuff in manage.py commands). On the other hand, if I were to write something like an analytics system, which had a web UI for users to inspect and play with results, I can see that I'd feel constrained by Django when writing the non-web parts. As someone who loved getting started with Flask, but then switched to Django after getting stuck without a good structure, I'd really like to understand whether the choice of Django vs. Flask should be made based on individual preferences (comfort of an imposed structure vs. freedom without it) or based on the type of project (mainly web, or mainly non-web)?
- whalesalad 11y ago
  It's tremendously freeing to have your application live as what amounts to vanilla python (or ruby, or php, or clojure, or insert-your-favorite-language-here). Once you've abstracted your core business logic from the HTTP layer, a lot of problems become easier to solve. If you want to replace Flask entirely it becomes very simple. If you want to replace REST entirely with a CQRS-style system, it's much easier. Testing gets easier. Wanna play with your app without firing up a web server? Build it so that you can use it from an ipython shell. Vanilla is the holy grail. You mention manage.py commands but why not if __name__ == '__main__': do the thing. Or the multitude of tools like invoke (https://github.com/pyinvoke/invoke https://github.com/pyinvoke/invoke) and click (http://click.pocoo.org/5/ http://click.pocoo.org/5/). I digress... Django is great for a monolith. Stack Django REST Framework on top of it and you're even bigger and nastier. I would never use it again because the amount of time spent bending it to my will often exceeds the time I'd spend building my own system. Fair warning, I have been using Django in production since version 0.96 and have built many companies on it. After ten years of building web apps, using every MVC under the sun, and working with every size/shape team under the sun you really start to appreciate simple things that compose well together. They're easier to reason about. They're easier to replace. They're easier to pull out and turn into a standalone library or shared service. Are you hitting the wall with one tiny part of your app? Replace it with something written in an entirely different language and communicate via JSON, or over a message bus. Not everything can be mapped to a WSGI request/response. I am so glad aidos has the top comment on this post because his/her style is what we should all be aiming for from the get-go. I'd definitely suggest reading up on the SOLID –https://en.wikipedia.org/wiki/SOLID_(object-oriented_design) https://en.wikipedia.org/wiki/SOLID_(object-oriented_design) – principles. Furthermore, the adapter concept is often explained with scary Java code samples, but the core of it is wonderful. Essentially you should consider your API as a vanilla system in your language of choice and you build adapters which then allow for external services to communicate. One adapter could be HTTP (via Flask perhaps) for a front-end website. One adapter could be HTTP but one that only speaks JSON, which you might serve under api.yourdomain.com. Another adapter might be via a queue system which allows for arbitrary commands from an external source to get written to a durable queue, so that they can be replayed later. I took some melatonin so at this point I feel like I am just babbling and about to fall asleep. tl;dr build things without frameworks, use your language well, be vanilla, compose tiny things to build big things.
edwinnathaniel 11y ago
This type of development style (http://martinfowler.com/eaaCatalog/serviceLayer.html http://martinfowler.com/eaaCatalog/serviceLayer.html) is quite well-known in the Java world yet unfortunately people always think that the technique is overkill (in reality, it's really really not overkill, it's only overkill if you force it to be).
- merb 11y ago
  Thanks to javax Most things could be switched out easily
vok5 11y ago
I'm curious, is this an open-source project? I would love to have a look into how you built this. If not, well, no worries.
- aidos 11y ago
  No, sorry - it's very much commercial code (it's our startup). Though it's 90% unicorns and dragons :-) Edit http://www.countfire.com http://www.countfire.com
nemmons 11y ago
This is probably a pretty stupid question, but can you point me in the direction of some recommended reading or documentation regarding how i might go about using a web framework as a wrapper for a separate app? I'm just getting started in python web dev, coming from PHP-MVC-framework-land and this idea is fascinating to me but i'm having a hard time conceptualizing how you would do it. edit: to be clear, i've read Fowler and understand the idea of a multi-layered application i'm just having a hard time understanding how to achieve it technically.
- aidos 11y ago
  Not a stupid question, especially since I just made it up as I went along - so there's not much to go on. Will try to put together a little mini example on github (but that will require me actually doing a bit of dissecting). I've also looked through the Fowler stuff (as well as almost everything else on the subject!). One thing I struggled with a lot was how to handle the DB layer. Fowler has that as the layer right in the middle of the system but I really didn't like that model. All the web frameworks are set up so the db layer is very entangled with the framework and it's quite hard to escape. Let me put something up on github. Send me an email (in my profile) and then I can let you know once I have something to look at.

by aidos 11y ago

odonnellryan 11y ago

Great points! I've worked with Flask for a while, as I've gotten to be a better developer. I used to have a lot of my logic in the application itself. Now, I ensure my logic lives in its own modules. Flask is really just either a way to serve my templates or API.

voltagex_ 11y ago

Maybe I'm just not thinking clearly today but how do you turn Flask into a wrapper? I thought you had to put enough logic in to a class that inherits from Flask to return views and such.

aidos 11y ago
I think I need to explain a bit better. I have a few different layers. From the outside in: Flask/Web Deal with sessions, authentication, caching, templating, logging etc here. Fires up an instance of the App and keeps it around for the duration of the life of the Web instance. I actually have an API in here that looks like what you'd expect in most other web apps. It does the validation / error handling etc. In terms of manipluating the data, I'm not a total purist - the API is allowed to load up the SQLA models and make changes. If there's a more complex operation going on though, it will live inside the next layer down. App Deal with remote services / data models / db connection. I guess on the outside you might call this a "service layer". A surprisingly large amount of this is data persistance management. The model I use is to put the data access stuff just off to the side at the top level of the app. Nothing in here knows anything about things like web frameworks. This is important. In other places in the system, this same App is used by RQ (worker instances taking on work via a redis queue). There's another web app that uses it for administrative tasks. I can boot it up to work with it in ipython (I do this a lot). Oh, and I don't manage the db life-cycle here. Whichever layer sits above this determines when it's ok to commit transactions. If something goes wrong here we raise app specific expections - the layer above can handle those and relay them as appropriate. Core In the very core I have "pure" algorithmic stuff (for context - http://www.countfire.com http://www.countfire.com is effectively an image recognition app). Really this is just one component that the App happens to be using. If the App wants to talk to this, it needs to create the data structures that the core defines. At various borders I often have mappings from the objects in the current local world to versions for the created to talk to the layer below (ps I like namedtuples). It's a lot more complex than all this, each of these layers has more sub layers etc. And really, what I've called the Core is a sublayer of the App (in my case there's a well definied algorithmic chunk that I think of as the core so it's in its own package - it stands alone as its own library). There's no real way of knowing where you'll draw lines between the layers when you start but I have a few rules. Any bit of code should be given the bare minimum of data required to do its work. At borders map local data to data for the next layer. Make sure the stuff going into the current layer is really there because it needs to be there. Be a pragmatist - things will change, you'll pull stuff out into submodules. By following the other rules hopefully refactoring won't be too painful.

BinaryIdiot 11y ago

Honestly this is how all development should be done. Treat the web server as yet another interface into your application to keep it as slim as possible. Need to change web servers, support thrift, avro, etc? Just write another interface that hooks into your application. Easy! There are plenty of examples in every language where a web framework turns into a web service + caching + session management + ORM + whatever else you can think of then you use all of those features and now you're inflexible, can't easily add new interfaces without seriously rewriting the old and good luck if you need to move away from that web framework for whatever reason.

whalesalad 11y ago
YES! Your username should be binaryfullofwisdom.
- BinaryIdiot 11y ago
  I will take that under advisement :)

m_mueller 11y ago

I like this approach. How do you handle routes? Do you just initialize a wrapper app that is composed of your own application, wrap your views in Flask views objects and then call `add_url_rule` on them in the wrapper setup code? If I understand this correctly, how do you define something like a redirect or abort within your own views?

aidos 11y ago
I've tried to explain my structure a bit better in a reply to voltagex_. For your specific question - all routing is done from within the top / Flask layer (as is redirecting / aborting). My App layer does not have any Flask stuff in, at all, ever.

rahimnathwani 11y ago

"Flask is then a thin wrapper that instantiates the app and then exposes the views to talk to it." Why did you decide to go this way? Is it because the user-facing parts of the app (the bits where Flask helps) are only a small part? I mean, if I'm developing a CRUD app with small amounts of supporting non-interactive code, I can't see a reason not to use Django (and put the other stuff in manage.py commands). On the other hand, if I were to write something like an analytics system, which had a web UI for users to inspect and play with results, I can see that I'd feel constrained by Django when writing the non-web parts. As someone who loved getting started with Flask, but then switched to Django after getting stuck without a good structure, I'd really like to understand whether the choice of Django vs. Flask should be made based on individual preferences (comfort of an imposed structure vs. freedom without it) or based on the type of project (mainly web, or mainly non-web)?

whalesalad 11y ago
It's tremendously freeing to have your application live as what amounts to vanilla python (or ruby, or php, or clojure, or insert-your-favorite-language-here). Once you've abstracted your core business logic from the HTTP layer, a lot of problems become easier to solve. If you want to replace Flask entirely it becomes very simple. If you want to replace REST entirely with a CQRS-style system, it's much easier. Testing gets easier. Wanna play with your app without firing up a web server? Build it so that you can use it from an ipython shell. Vanilla is the holy grail. You mention manage.py commands but why not if __name__ == '__main__': do the thing. Or the multitude of tools like invoke (https://github.com/pyinvoke/invoke https://github.com/pyinvoke/invoke) and click (http://click.pocoo.org/5/ http://click.pocoo.org/5/). I digress... Django is great for a monolith. Stack Django REST Framework on top of it and you're even bigger and nastier. I would never use it again because the amount of time spent bending it to my will often exceeds the time I'd spend building my own system. Fair warning, I have been using Django in production since version 0.96 and have built many companies on it. After ten years of building web apps, using every MVC under the sun, and working with every size/shape team under the sun you really start to appreciate simple things that compose well together. They're easier to reason about. They're easier to replace. They're easier to pull out and turn into a standalone library or shared service. Are you hitting the wall with one tiny part of your app? Replace it with something written in an entirely different language and communicate via JSON, or over a message bus. Not everything can be mapped to a WSGI request/response. I am so glad aidos has the top comment on this post because his/her style is what we should all be aiming for from the get-go. I'd definitely suggest reading up on the SOLID –https://en.wikipedia.org/wiki/SOLID_(object-oriented_design) https://en.wikipedia.org/wiki/SOLID_(object-oriented_design) – principles. Furthermore, the adapter concept is often explained with scary Java code samples, but the core of it is wonderful. Essentially you should consider your API as a vanilla system in your language of choice and you build adapters which then allow for external services to communicate. One adapter could be HTTP (via Flask perhaps) for a front-end website. One adapter could be HTTP but one that only speaks JSON, which you might serve under api.yourdomain.com. Another adapter might be via a queue system which allows for arbitrary commands from an external source to get written to a durable queue, so that they can be replayed later. I took some melatonin so at this point I feel like I am just babbling and about to fall asleep. tl;dr build things without frameworks, use your language well, be vanilla, compose tiny things to build big things.

edwinnathaniel 11y ago

This type of development style (http://martinfowler.com/eaaCatalog/serviceLayer.html http://martinfowler.com/eaaCatalog/serviceLayer.html) is quite well-known in the Java world yet unfortunately people always think that the technique is overkill (in reality, it's really really not overkill, it's only overkill if you force it to be).

merb 11y ago
Thanks to javax Most things could be switched out easily

vok5 11y ago

I'm curious, is this an open-source project? I would love to have a look into how you built this. If not, well, no worries.

aidos 11y ago
No, sorry - it's very much commercial code (it's our startup). Though it's 90% unicorns and dragons :-) Edit http://www.countfire.com http://www.countfire.com

nemmons 11y ago

This is probably a pretty stupid question, but can you point me in the direction of some recommended reading or documentation regarding how i might go about using a web framework as a wrapper for a separate app? I'm just getting started in python web dev, coming from PHP-MVC-framework-land and this idea is fascinating to me but i'm having a hard time conceptualizing how you would do it. edit: to be clear, i've read Fowler and understand the idea of a multi-layered application i'm just having a hard time understanding how to achieve it technically.

aidos 11y ago
Not a stupid question, especially since I just made it up as I went along - so there's not much to go on. Will try to put together a little mini example on github (but that will require me actually doing a bit of dissecting). I've also looked through the Fowler stuff (as well as almost everything else on the subject!). One thing I struggled with a lot was how to handle the DB layer. Fowler has that as the layer right in the middle of the system but I really didn't like that model. All the web frameworks are set up so the db layer is very entangled with the framework and it's quite hard to escape. Let me put something up on github. Send me an email (in my profile) and then I can let you know once I have something to look at.