4 ms·

Optimizations worth doing right away

5 points by sanj 19y ago

This submission: http://news.ycombinator.com/item?id=130567 ended up hitting on a couple of things that ARE worth doing right away; things which are worth at least considering, and perhaps implementing, early. I'd appreciate insights from the community about those "early" optimization which seem worthwhile.

sanj 19y ago
Avoid obvious n^2 or n*m loops
- ardit33 19y ago
  wrong. If you know your loops is going to go thru max a hundred of times, then just don't optimizing, as it would be a n overkill. if you know your loop might iterate up to thousands, then it is time to seek some better algorithms.
sanj 19y ago
If you're intending to partition database tables across servers (aka "shard") by users: 1. Make sure every table has a user_id field 2. Make sure every record gets that field populated 3. Don't forget to index it Watch out for queries that don't use the field as part of their 'where' clause.
sanj 19y ago
Reduce linear lookups to hashed ones if they happen regularly.
edw519 19y ago
Structure your data (both on your data base and in your programs) AS IF you already had billions of records. Do this from the start. It takes no more time than throwing something together now. It will definitely save you lots of time later. Also, MINIMIZE the amount of data that travels between client and server, either way, ever. If you don't now, you won't notice the hit in your test environment, but you will notice it later. The problem then may be a lot of restructuring and rewriting.
- ardit33 19y ago
  what if my app will never reach a billion records? This seems an overkill. I think when your program, you should program thinking in a this way: What's the max usage groth I can realisticly predict of happening on the next year? If in a year you have the good problem as too many people are siging up for your service, there is many ways to keep that groth and performance happening. Concetrating on having millions of users, when you haven't written a line of code yet, is just an dellusion delaying your progress to even get any users.
  - edw519 19y ago
    "Concetrating on having millions of users" I never said that. I said to structure your data AS IF you had billions of records. Are you going to normalize your data? How much? Are you going to index it? How? How are you going to structure your iterations, branches, and internal data structures? Put any thought into any of it? With rare exceptions, what works well for billions of records also works well for hundreds of records. The converse, unfortunately, is painfully untrue. Think ahead and you won't have to worry about dealing with it.
radu_floricica 19y ago
Sorry... conventional wisdom seems to be right here. No early optimisations is best. If a software grows like a tree, then only some branches, no, some leaves will require optimisation. You never know from the start the route to those leaves because of the complexity of the software. Trying to optimise early is either like taking all possible routes but stopping after a short while on each. What I'm trying to say is optimisation is always a matter of _depth_, not of breadth. Finding the right direction and going all the way gives order of magnitudes better results then following best practices throughout the software.

5 points by sanj 19y ago

sanj 19y ago

Avoid obvious n^2 or n*m loops

ardit33 19y ago
wrong. If you know your loops is going to go thru max a hundred of times, then just don't optimizing, as it would be a n overkill. if you know your loop might iterate up to thousands, then it is time to seek some better algorithms.

sanj 19y ago

If you're intending to partition database tables across servers (aka "shard") by users: 1. Make sure every table has a user_id field 2. Make sure every record gets that field populated 3. Don't forget to index it Watch out for queries that don't use the field as part of their 'where' clause.

sanj 19y ago

Reduce linear lookups to hashed ones if they happen regularly.

edw519 19y ago

Structure your data (both on your data base and in your programs) AS IF you already had billions of records. Do this from the start. It takes no more time than throwing something together now. It will definitely save you lots of time later. Also, MINIMIZE the amount of data that travels between client and server, either way, ever. If you don't now, you won't notice the hit in your test environment, but you will notice it later. The problem then may be a lot of restructuring and rewriting.

ardit33 19y ago
what if my app will never reach a billion records? This seems an overkill. I think when your program, you should program thinking in a this way: What's the max usage groth I can realisticly predict of happening on the next year? If in a year you have the good problem as too many people are siging up for your service, there is many ways to keep that groth and performance happening. Concetrating on having millions of users, when you haven't written a line of code yet, is just an dellusion delaying your progress to even get any users.
- edw519 19y ago
  "Concetrating on having millions of users" I never said that. I said to structure your data AS IF you had billions of records. Are you going to normalize your data? How much? Are you going to index it? How? How are you going to structure your iterations, branches, and internal data structures? Put any thought into any of it? With rare exceptions, what works well for billions of records also works well for hundreds of records. The converse, unfortunately, is painfully untrue. Think ahead and you won't have to worry about dealing with it.

radu_floricica 19y ago

Sorry... conventional wisdom seems to be right here. No early optimisations is best. If a software grows like a tree, then only some branches, no, some leaves will require optimisation. You never know from the start the route to those leaves because of the complexity of the software. Trying to optimise early is either like taking all possible routes but stopping after a short while on each. What I'm trying to say is optimisation is always a matter of _depth_, not of breadth. Finding the right direction and going all the way gives order of magnitudes better results then following best practices throughout the software.