9 ms·
Lessons learned defying Joel Spolsky with Django
- tachion 13y agoI wonder, is there a video of the talk available? The slides, unfortunately, alone are rarely very informative.
- iconfinder 13y agoWe will post video version soon (Monday) on http://blog.iconfinder.com http://blog.iconfinder.com
- lstamour 13y agoVideo now live: http://vimeo.com/65057265 http://vimeo.com/65057265
- emperorcezar 13y agoWhile I love seeing slides, this deck obviously could use the audio or transcript along with it.
- rattray 13y agoAgreed, I for one was felt feeling rather in the dark for the last few slides especially...
- iconfinder 13y agoWe are posting a version with audio on our blog on Monday: http://blog.iconfinder.com http://blog.iconfinder.com
- abs_farah 13y agoI look forward to it! Only audio, no video?
- iconfinder 13y agoWe will combine audio with the slides :-)
- lstamour 13y agoVideo now live: http://vimeo.com/65057265 http://vimeo.com/65057265
- iconfinder 13y agoHere it is: http://blog.iconfinder.com/staying-sane-while-defying-joel-spolsky-with-django/ http://blog.iconfinder.com/staying-sane-while-defying-joel-s...
- deleted 13y ago[deleted]
- deleted 13y ago[deleted]
- zzzeek 13y agoDon't use just one ORM and then declare "ORM's are stupid". The "object = None" / "object_id = None" issue illustrated here is certainly not a mistake every ORM makes.
- InclinedPlane 13y agoDo you have an example of an ORM which is not in some fundamental way "stupid"? I haven't found one yet, but I'd love to know one existed somewhere.
- zzzeek 13y agoUntil we've attained AI, all software can be said to be "stupid". Google is stupid. Relational databases are stupid, SQL is stupid. None of it works without human intelligence actively directing it to do work for us. Why single out ORMs?
- PommeDeTerre 13y agoYou're contradicting yourself. Your first comment indicates that it's wrong to label ORMs as "stupid". Yet in this latest comment, you've labeled all software (which includes ORMs, of course) as "stupid". Like the other commenter requested, can you provide an example of an ORM that isn't "stupid"? Your initial comment makes it sound like you've worked with at least one that isn't. We're curious to learn about which these may be.
- zzzeek 13y agoThe word "stupid" here is ambiguous, so can be interpreted as, "stupid", "it's stupid to use them" (I disagree), or "stupid", "they don't necessarily understand your intent" (this applies to all software), "stupid", "simple issues are made more complicated than they should be, to an unreasonable extent" (this is a problem that varies to a significant degree based on the ORM in use and cannot be generalized based on the experience of just one ORM). Without the benefit of the speaker's words here it's hard to tell, though the only backing evidence given in these slides for "ORMs are stupid" is a trivial issue that not every ORM has, so that's IMO not a good argument. View my profile for further detail.
- nnq 13y agoBumping into ORM limitations + moving to Jinja for templates --> one word: Flask ...really, what advantage does Django provide at this point in this project anymore?
- megaman821 13y agoThere are still a lot of Django apps that work fine in this scenario. Plus I don't think the slides imply that they are just blindly replacing ORM calls with raw SQL, just that they have profiled and replaced the hot spots. Also Flask is a micro-framework and Django is a full-stack framework. Flask can be used as the backbone of a full-stack framework but figuring out a good project structure and finding out what third-party apps to use can be daunting for a person without Flask experience. If you really want to get people to switch, package up a Flask-based framework with the features of Django.
- pyriku 13y agoMiddlewares, context processors, forms, (class based) views, and tons of third party applications. The CTO of a startup where some friends are working thought the same you did, and 2 years ago rewrote everything to Flask. Now they're going back to Django. Django is much more that an ORM and templates.
- jokull 13y agoDjango middlewares is a poorly designed system. Flask has all of the things you mentioned except for forms. For that you can use Flask-WTF, but these libraries are becoming outdated anyway because they’re HTML based and don’t work so well for JSON validation.
- lstamour 13y agoI found I could follow along with the slides, though some of the icons and messages around third-party tools were lost on me. I'd definitely appreciate a video. Edit: 9 minutes ago, Nick posted to Twitter: "we have the recording - just need the sound cleaned up. Expect it early next week ;)"
- lstamour 13y agoVideo now live: http://vimeo.com/65057265 http://vimeo.com/65057265 [Slides in sync with audio]
- speg 13y agoDoes anyone know what tool was used to profile the django app? Looks cool.
- naithemilkman 13y agoIf you're talking about the slide that says '87.91% of time spent rendering' then yes, I would like to know too. Annoying that it doesnt have slide numbers to refer to.
- cdavid 13y agoit looks like kcachegrind or pycallgraph (https://github.com/gak/pycallgraph https://github.com/gak/pycallgraph).
- bdarnell 13y agoLooks like gprof2dot: https://code.google.com/p/jrfonseca/wiki/Gprof2Dot https://code.google.com/p/jrfonseca/wiki/Gprof2Dot
- bsaul 13y agoDid someone understand the last slides about the differences between transactions in tasks with and without celery ? I'm using celery, and i have been using django in the past, and I really didn't get the point.
- mbell 13y agoHe's trying to avoid the situation where you enqueue a background job inside a transaction and the worker gets started on that job before the transaction is committed. If the job needs to hit the database for any reason your likely to get errors as the new data hasn't been committed yet. It looks like the work around he used is to cache the job queue locally and only flush it to the real job queue after the database commit, so your guaranteed whatever data may be needed for the job has been committed to the database.
- ashchristopher 13y agoBasically if you modify data in a transaction, add a task to the queue that relies on that data, and the worker pulls the task off the queue BEFORE the transaction is committed, then the task tries to access data that doesn't exist in the database yet.
- spitfire 13y agoHe's trying to fix/avoid a race condition caused by data created within a transaction not being committed yet.
- jroseattle 13y agoI was expecting this slidedeck to be a bit more focused on defying Spolsky, and whether or not that was a good decision. FWIW, I've never bought into Spolsky's vision that re-writing code is poor strategy. Steve Jobs never thought twice about ripping something apart and starting over. If anything, code re-write can be an advantageous position -- you often have a greater understanding of the problems you're intending to solve. When well-executed, it can take the form of heavy refactoring, even when switching languages/platforms.
- LockeWatts 13y agoI'm not really sure why Steve Jobs is your model for code design.
- widdershins 13y agoBecause he oversaw the creation of 3 of the most successful operating systems of all time? I know he didn't design them, but he was involved in managing the projects. Just playing devil's advocate.
- zerr 13y agoWell, it doesn't mean he evaluated at the code level and decided that rewriting is better than refactoring...
- jroseattle 13y agoApple has/had a history of re-writing code over the years. The first iPods and their iterations were code re-writes, IIRC. Many of the onboard applications were completely re-written for the iPad.
- Daishiman 13y agoRewriting for new hardware is totally different from rewriting business logic. In the first case there are sometimes no other choices left. Also, the code that usually goes into hardware microcode and firmwares is much more tightly coupled.
- level09 13y agoI found these tools mentioned by the author really helpful, I compiled a list of them: http://jinja.pocoo.org/ http://jinja.pocoo.org/ https://www.getsentry.com/welcome/ https://www.getsentry.com/welcome/ http://graphite.wikidot.com/start http://graphite.wikidot.com/start https://opbeat.com/ https://opbeat.com/ is there a way though to use/test opbeat ?
- iconfinder 13y agoJust send them a message via twitter or email. I'm sure they will help you get started.
- Uchikoma 13y agoIconfinder. They are profitable? They have business requirements? Joel wrote about the problems of a rewrite when you need to earn money, implement money making features, heavy competition while at the same time maintaining two plattform where one is a moving target. I don't think Iconfinder fits in any of these constraints.
- Uchikoma 13y ago+100 for using "leaking abstraction" in a slide deck "defying Joel Spolsky"
- Kiro 13y agoI didn't understand why ORMs are stupid. Can someone enlighten me?
- kybernetyk 13y agoI guess his point is that they sometimes create huge ugly and inefficient SQL queries. (Reminds me of the 90s sentiment of C compilers creating ugly assembly.)
- macspoofing 13y agoORMs pretend to give you full abstraction, but in reality you have to be aware of the underlying SQL layer when you build your object model.
- antihero 13y agoI find that if you are aware of the SQL underneath, they are good for saving time. Do you really want to write SQL to retrieve data and code to populate an object for every damn thing in your system?
- macspoofing 13y agoI agree. Problems happen if you treat ORMs as a total blackbox.
- cwbrandsma 13y agoThe real issue with ORMs isn't the ORM itself, but over reliance on the ORM to do everything the right way and not validating the ORM is doing things the right way. ORMs are also often heavily leaned on by people who don't understand SQL and relational databases well enough and just want a data dumping ground (would have been better off with a document database). But, if properly validated, and knowing when to NOT use the ORM, a good ORM can help you get a lot of work done very efficiently. But I've also seen improperly used ORMs turn into MASSIVE time sinks where devs spends days just configuring the stupid thing (hello Hibernate/nhibernate).
- 13y ago
- scott_meade 13y agoORMs may be "stupid", yet Basecamp manages to handle 400-500 request per second with Rails and MySQL just fine. https://twitter.com/dhh/status/287221705443774465 https://twitter.com/dhh/status/287221705443774465
- claudiusd 13y agoI don't think Spolsky's argument applies to startups... His argument basically boils down to "You think the code in front of you is a mess, but the reality is you're just having trouble reading somebody else's code which is probably good enough". But what if you wrote the code yourself? In that case, it's probably just a mess.
- gingerlime 13y ago> 20ms with Jinja2 without auto-escaping could this performance improvement back-fire if you end up with a security issue? I'm not saying that it definitely would. If you know what you're doing / trust your data sources or sanitize them elsewhere, you should be fine. I'd be careful turning off such a feature completely though...
- Daishiman 13y agoOf course. This is merely a tradeoff between performance and developer time. 99% of projects will never have HTML autoescaping as a performance pain point. Then again, you're going to need tens of hours to review all templates to make sure you're escaping everything. If your hardware budget is greated than what it costs to audit the code, it's the proper decision.
- hasenj 13y agoYou should give SQLAlchemy a try: http://lucumr.pocoo.org/2011/7/19/sqlachemy-and-you/ http://lucumr.pocoo.org/2011/7/19/sqlachemy-and-you/ It gives you more control and requires you to be more explicit about your queries and relationships.
- beambot 13y agoIs there a (publicly available) video that accompanies this slide deck?
- lstamour 13y agoVideo now live: http://vimeo.com/65057265 http://vimeo.com/65057265
- dmishe 13y agoInteresting point on celery and transaction, though I've never seen that happen in real life, interesting.
- iconfinder 13y agoWe have posted a video version of the talk (slides + audio): http://blog.iconfinder.com/staying-sane-while-defying-joel-spolsky-with-django/ http://blog.iconfinder.com/staying-sane-while-defying-joel-s...