9 ms·
Lessons learned defying Joel Spolsky with Django
- tachion 14y agoI wonder, is there a video of the talk available? The slides, unfortunately, alone are rarely very informative.
- iconfinder 14y agoWe will post video version soon (Monday) on http://blog.iconfinder.com http://blog.iconfinder.com
- lstamour 14y agoVideo now live: http://vimeo.com/65057265 http://vimeo.com/65057265
- emperorcezar 14y agoWhile I love seeing slides, this deck obviously could use the audio or transcript along with it.
- rattray 14y agoAgreed, I for one was felt feeling rather in the dark for the last few slides especially...
- iconfinder 14y agoWe are posting a version with audio on our blog on Monday: http://blog.iconfinder.com http://blog.iconfinder.com
- abs_farah 14y agoI look forward to it! Only audio, no video?
- iconfinder 14y agoWe will combine audio with the slides :-)
- lstamour 14y agoVideo now live: http://vimeo.com/65057265 http://vimeo.com/65057265
- iconfinder 14y agoHere it is: http://blog.iconfinder.com/staying-sane-while-defying-joel-spolsky-with-django/ http://blog.iconfinder.com/staying-sane-while-defying-joel-s...
- deleted 14y ago[deleted]
- deleted 14y ago[deleted]
- zzzeek 14y agoDon't use just one ORM and then declare "ORM's are stupid". The "object = None" / "object_id = None" issue illustrated here is certainly not a mistake every ORM makes.
- InclinedPlane 14y agoDo you have an example of an ORM which is not in some fundamental way "stupid"? I haven't found one yet, but I'd love to know one existed somewhere.
- zzzeek 14y agoUntil we've attained AI, all software can be said to be "stupid". Google is stupid. Relational databases are stupid, SQL is stupid. None of it works without human intelligence actively directing it to do work for us. Why single out ORMs?
- PommeDeTerre 14y agoYou're contradicting yourself. Your first comment indicates that it's wrong to label ORMs as "stupid". Yet in this latest comment, you've labeled all software (which includes ORMs, of course) as "stupid". Like the other commenter requested, can you provide an example of an ORM that isn't "stupid"? Your initial comment makes it sound like you've worked with at least one that isn't. We're curious to learn about which these may be.
- zzzeek 14y agoThe word "stupid" here is ambiguous, so can be interpreted as, "stupid", "it's stupid to use them" (I disagree), or "stupid", "they don't necessarily understand your intent" (this applies to all software), "stupid", "simple issues are made more complicated than they should be, to an unreasonable extent" (this is a problem that varies to a significant degree based on the ORM in use and cannot be generalized based on the experience of just one ORM). Without the benefit of the speaker's words here it's hard to tell, though the only backing evidence given in these slides for "ORMs are stupid" is a trivial issue that not every ORM has, so that's IMO not a good argument. View my profile for further detail.
- nnq 14y agoBumping into ORM limitations + moving to Jinja for templates --> one word: Flask ...really, what advantage does Django provide at this point in this project anymore?
- megaman821 14y agoThere are still a lot of Django apps that work fine in this scenario. Plus I don't think the slides imply that they are just blindly replacing ORM calls with raw SQL, just that they have profiled and replaced the hot spots. Also Flask is a micro-framework and Django is a full-stack framework. Flask can be used as the backbone of a full-stack framework but figuring out a good project structure and finding out what third-party apps to use can be daunting for a person without Flask experience. If you really want to get people to switch, package up a Flask-based framework with the features of Django.
- pyriku 14y agoMiddlewares, context processors, forms, (class based) views, and tons of third party applications. The CTO of a startup where some friends are working thought the same you did, and 2 years ago rewrote everything to Flask. Now they're going back to Django. Django is much more that an ORM and templates.
- jokull 14y agoDjango middlewares is a poorly designed system. Flask has all of the things you mentioned except for forms. For that you can use Flask-WTF, but these libraries are becoming outdated anyway because they’re HTML based and don’t work so well for JSON validation.
- lstamour 14y agoI found I could follow along with the slides, though some of the icons and messages around third-party tools were lost on me. I'd definitely appreciate a video. Edit: 9 minutes ago, Nick posted to Twitter: "we have the recording - just need the sound cleaned up. Expect it early next week ;)"
- lstamour 14y agoVideo now live: http://vimeo.com/65057265 http://vimeo.com/65057265 [Slides in sync with audio]
- speg 14y agoDoes anyone know what tool was used to profile the django app? Looks cool.
- naithemilkman 14y agoIf you're talking about the slide that says '87.91% of time spent rendering' then yes, I would like to know too. Annoying that it doesnt have slide numbers to refer to.
- cdavid 14y agoit looks like kcachegrind or pycallgraph (https://github.com/gak/pycallgraph https://github.com/gak/pycallgraph).
- bdarnell 14y agoLooks like gprof2dot: https://code.google.com/p/jrfonseca/wiki/Gprof2Dot https://code.google.com/p/jrfonseca/wiki/Gprof2Dot
- bsaul 14y agoDid someone understand the last slides about the differences between transactions in tasks with and without celery ? I'm using celery, and i have been using django in the past, and I really didn't get the point.
- mbell 14y agoHe's trying to avoid the situation where you enqueue a background job inside a transaction and the worker gets started on that job before the transaction is committed. If the job needs to hit the database for any reason your likely to get errors as the new data hasn't been committed yet. It looks like the work around he used is to cache the job queue locally and only flush it to the real job queue after the database commit, so your guaranteed whatever data may be needed for the job has been committed to the database.
- ashchristopher 14y agoBasically if you modify data in a transaction, add a task to the queue that relies on that data, and the worker pulls the task off the queue BEFORE the transaction is committed, then the task tries to access data that doesn't exist in the database yet.
- spitfire 14y agoHe's trying to fix/avoid a race condition caused by data created within a transaction not being committed yet.
- jroseattle 14y agoI was expecting this slidedeck to be a bit more focused on defying Spolsky, and whether or not that was a good decision. FWIW, I've never bought into Spolsky's vision that re-writing code is poor strategy. Steve Jobs never thought twice about ripping something apart and starting over. If anything, code re-write can be an advantageous position -- you often have a greater understanding of the problems you're intending to solve. When well-executed, it can take the form of heavy refactoring, even when switching languages/platforms.
- LockeWatts 14y agoI'm not really sure why Steve Jobs is your model for code design.
- widdershins 14y agoBecause he oversaw the creation of 3 of the most successful operating systems of all time? I know he didn't design them, but he was involved in managing the projects. Just playing devil's advocate.
- zerr 14y agoWell, it doesn't mean he evaluated at the code level and decided that rewriting is better than refactoring...
- jroseattle 14y agoApple has/had a history of re-writing code over the years. The first iPods and their iterations were code re-writes, IIRC. Many of the onboard applications were completely re-written for the iPad.
- Daishiman 14y agoRewriting for new hardware is totally different from rewriting business logic. In the first case there are sometimes no other choices left. Also, the code that usually goes into hardware microcode and firmwares is much more tightly coupled.
- level09 14y agoI found these tools mentioned by the author really helpful, I compiled a list of them: http://jinja.pocoo.org/ http://jinja.pocoo.org/ https://www.getsentry.com/welcome/ https://www.getsentry.com/welcome/ http://graphite.wikidot.com/start http://graphite.wikidot.com/start https://opbeat.com/ https://opbeat.com/ is there a way though to use/test opbeat ?
- iconfinder 14y agoJust send them a message via twitter or email. I'm sure they will help you get started.
- Uchikoma 14y agoIconfinder. They are profitable? They have business requirements? Joel wrote about the problems of a rewrite when you need to earn money, implement money making features, heavy competition while at the same time maintaining two plattform where one is a moving target. I don't think Iconfinder fits in any of these constraints.
- Uchikoma 14y ago+100 for using "leaking abstraction" in a slide deck "defying Joel Spolsky"
- Kiro 14y agoI didn't understand why ORMs are stupid. Can someone enlighten me?
- kybernetyk 14y agoI guess his point is that they sometimes create huge ugly and inefficient SQL queries. (Reminds me of the 90s sentiment of C compilers creating ugly assembly.)
- macspoofing 14y agoORMs pretend to give you full abstraction, but in reality you have to be aware of the underlying SQL layer when you build your object model.
- antihero 14y agoI find that if you are aware of the SQL underneath, they are good for saving time. Do you really want to write SQL to retrieve data and code to populate an object for every damn thing in your system?
- macspoofing 14y agoI agree. Problems happen if you treat ORMs as a total blackbox.
- cwbrandsma 14y agoThe real issue with ORMs isn't the ORM itself, but over reliance on the ORM to do everything the right way and not validating the ORM is doing things the right way. ORMs are also often heavily leaned on by people who don't understand SQL and relational databases well enough and just want a data dumping ground (would have been better off with a document database). But, if properly validated, and knowing when to NOT use the ORM, a good ORM can help you get a lot of work done very efficiently. But I've also seen improperly used ORMs turn into MASSIVE time sinks where devs spends days just configuring the stupid thing (hello Hibernate/nhibernate).
- 14y ago
- scott_meade 14y agoORMs may be "stupid", yet Basecamp manages to handle 400-500 request per second with Rails and MySQL just fine. https://twitter.com/dhh/status/287221705443774465 https://twitter.com/dhh/status/287221705443774465
- claudiusd 14y agoI don't think Spolsky's argument applies to startups... His argument basically boils down to "You think the code in front of you is a mess, but the reality is you're just having trouble reading somebody else's code which is probably good enough". But what if you wrote the code yourself? In that case, it's probably just a mess.
- gingerlime 14y ago> 20ms with Jinja2 without auto-escaping could this performance improvement back-fire if you end up with a security issue? I'm not saying that it definitely would. If you know what you're doing / trust your data sources or sanitize them elsewhere, you should be fine. I'd be careful turning off such a feature completely though...
- Daishiman 14y agoOf course. This is merely a tradeoff between performance and developer time. 99% of projects will never have HTML autoescaping as a performance pain point. Then again, you're going to need tens of hours to review all templates to make sure you're escaping everything. If your hardware budget is greated than what it costs to audit the code, it's the proper decision.
- hasenj 14y agoYou should give SQLAlchemy a try: http://lucumr.pocoo.org/2011/7/19/sqlachemy-and-you/ http://lucumr.pocoo.org/2011/7/19/sqlachemy-and-you/ It gives you more control and requires you to be more explicit about your queries and relationships.
- beambot 14y agoIs there a (publicly available) video that accompanies this slide deck?
- lstamour 14y agoVideo now live: http://vimeo.com/65057265 http://vimeo.com/65057265
- dmishe 14y agoInteresting point on celery and transaction, though I've never seen that happen in real life, interesting.
- iconfinder 14y agoWe have posted a video version of the talk (slides + audio): http://blog.iconfinder.com/staying-sane-while-defying-joel-spolsky-with-django/ http://blog.iconfinder.com/staying-sane-while-defying-joel-s...