7 ms·
You should worry. If you don't want your server or your app to run slow, you should worry about these things. Can you imagine going into a programming interview
by btilly 14y ago
You should worry. If you don't want your server or your app to run slow, you should worry about these things. Can you imagine going into a programming interview and saying something like this?
You sound like someone who doesn't really understand performance very well. Lists and strings and arrays as implemented in any scripting language can implement virtually any algorithm with only a constant factor performance and memory penalty. (Try to come up with a counter-example to that statement, I dare you.) Switching from language to language comes with similar factors. If you discover that you've got a problem that requires you to worry about data structures at that level, worry about it then. Possibly you should use a more efficient language.
But for most apps and websites, those constant factors are really not a big deal. Basic algorithm mistakes are a different story.
For the exceptional cases, you should build working code first, then use a profiler to figure out where your problems are. You should only worry about performance after identifying actual bottlenecks.
If I interview with a company that uses a scripting language and does not understand all of that, that's a pretty good sign that I don't want to work there. Because their opinions on performance are clearly misguided, and I don't want to have to work with what they thought would be "optimized code".
- zanny 14y agoCache coherency matters, and making sure a large data structure fits in memory is important too. If you can't fit it, at least make it nicely match up to page boundaries. If you don't understandt he implementation of your choice array / set / map / tuple data structure that you are throwing around all over the place, you could be missing plenty of performance just from TLB misses or some other hardware voodoo.
- Evbn 14y agoThis matters if you are Google, serving gigabytes of data per user for free. This matters less for sites whose compute budget is 10% of their human budget, where trying to cut from 2x to 1x on compute isn't worth the human cost.
- justincormack 14y agoI imagine Google's staff budget is hugely higher than their compute budget. But when budgets get big you want to cut them anyway ...
- lmm 14y agoPoor logic on their part if so. Slowing your programmers down 2% to cut 10% off your hardware costs is usually a bad idea, and that's still true as your budgets for both grow, as long as the ratio stays the same. (I don't think Google are stupid, I think their ratio probably has shifted to the point where their hardware budget is comparable to their salary budget. Which makes them a very unusual case)
- btilly 14y agoYes, all of those make a difference. Sometimes a big one. But in general, it is best to just try to be reasonably sane and not worry about it until you have a proven problem. A surprising fraction of the time, a "reasonably sane" approach is not that far off from optimal. And setting the bar there helps development speed. Which, of course, is what you care about when you choose to use a scripting language. If you care more about computer time than developer time, then using a more efficient language makes sense.
- chubot 14y agoYeah I always hear people saying how Python is slow. But in 10 years of Python programming, I see 2 repeated performance anti-patterns: 1) Writing quadratic loops and not realizing it, then it blows up on a biggish data set. This is quite easy to do in Python because people don't realize that "in" on a list and .remove() and so forth do a linear search. 2) Writing Python like Java, where you have tons of indirection, which is slow. People somehow feel naked when they only have to type 10 characters rather than banging out a bunch of boilerplate for a mundane task. When you're just using basic data structures in an idiomatic way, Python is fast. I think there were some dictionary-heavy benchmarks I saw awhile ago where Python is faster than Go, because its dictionaries are so highly tuned.
- btilly 14y agoThe person I was responding to sounds likely to make mistake #2.
- paganel 14y ago> Writing Python like Java, where you have tons of indirection, which is slow I've actually just introduced one of my younger colleagues (and pretty new to Python, but knowledgeable in C# and Java) to this 8-year old article: http://dirtsimple.org/2004/12/python-is-not-java.html http://dirtsimple.org/2004/12/python-is-not-java.html I remember reading that article back in the day, I had only begun to do stuff in Python for a year or so, and everything seemed so well explained but here I am now instead still spreading the word :) At least some part of it I hope it's done for good, such as "XML is not the answer", which actually brings back "horrific" memories of Zope3.
- fauigerzigerk 14y agoNo, I think worrying about data structures should come before worrying about language efficiency. But coming up with a counter example to your challenge is hardly possible because you worded it in a way that is sufficiently vague to dispute any suggested counter-example by saying "virtually any algorithm". But how about substring search? Using Python's string.find instead of something like a suffix tree certainly carries more than a constant time penalty (Obviously for memory complexity it's the other way around in this case). The same goes for blindly using a standard Python list when you plan to insert lots of items in the middle. Not knowing the complexity of operations on standard data structures inevitably leads to "basic algorithm mistakes". How would you choose a good algorithm without knowing the datastrucure it's operating on?