8 ms·

The most difficult thing is going to be getting to 10K active users :) These days RAM is cheap and SSD storage is also widely available. For a very long time,

• sixpenrose16 12y ago
  That's a good overview. I would also not forget about the type of application. Like you said, resources are pretty cheap these days, maybe leveraging Amazon cloud and integrating with other applications could life the heavy weight that is normally required. If someone can get to 10k active users, they should also note on possible scaling issues down the line.
• ShinyCyril 12y ago
  > Rails should take care of most of this for you I often found myself falling into the "I'm not using PHP, so I don't have to worry about any security holes" trap. CSRF is something you really need to watch out for if you are constructing forms manually!
  • oafitupa 12y ago
    And while you are bashing PHP, actual professionals use it all the time and don't fall for those noob mistakes. Learn some Symfony2.
    • hackerboos 12y ago
      Giving OP benefit of the doubt I think s/he meant to say "When I'm not using a framework."
    • ShinyCyril 12y ago
      No bashing intended. I started off writing bad applications in plain PHP that were full of security holes, and moved on to writing better applications in Python with the help of frameworks - at the time I didn't realise there were helper frameworks for PHP too and thought Python was infallible.
  • briandear 12y ago
    No need to construct forms manually, Rails form helpers take care of authenticity tokens automatically.
    • ShinyCyril 12y ago
      I use Python myself and had this problem until I discovered WTForms.
• porker 12y ago
  > * Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space. +1 You can't log too much. The user who claims an important email never arrived - does your system say it was sent? This bug 3 users have reported yet no one can reproduce - what were they doing at the time and what else was going on? No, I'm not at that stage yet (of effectively being able to rewind application state in the log files to see what was going on), but for debugging issues in production it's exceedingly useful.
  • edmack 12y ago
    Getting loads of core services out into third parties is really wonderful for logging. E.g. if email sending happens in Mandrill, then you never need to write decent logging calls for that and you have a reliable source of truth!
    • tlack 12y ago
      Except you won't know if your server ever sent it to Mandrill. :) Always be extremely verbose with logging!
      • reymus 12y ago
        I have always heard the opposite, that too much logging is as bad as no logging. I see the point of having the logs to be able to find out what happened, but what happens when there' s so much logging that the information needed is just buried into huge amount of noise?
        • msielski 12y ago
          This is true, without the right tools. I am moving to logstash with kibana to do this, and it's looking very promising. See http://www.elasticsearch.org/videos/kibana-logstash/ http://www.elasticsearch.org/videos/kibana-logstash/
• icehawk219 12y ago
  > Backups - Take regular backups of all user data. It's also VERY important that you actually try restoring the data as well, as it's quite possible that backups are not occurring properly. The part about testing your backups is huge. I can't count how many projects I've been on that had problems where we needed to restore and we looked only to find any number of problems. Oh, backups actually stopped last month when we ran out of space, oops the backups only backed up these 3 db's and not the one you want, things like that. I'd also stress the importance of off-site backups. If you're using AWS for everything and your account is compromised can they delete your backups (assuming they have full, 100% unlimited, admin access to AWS)? Which is also why if you're using stuff like AWS, Heroku, or any other third party provider (hosted Mongo, hosted ElasticSearch, Stripe, NewRelic, etc.) it's very important to ensure those passwords are secured and only the people absolutely necessary have access. Also, when offered, two-factor authentication should always be used.
  • Rapzid 12y ago
    And don't use keys on your console admin accounts.
  • bw00d 12y ago
    This is great, thanks.
• danmanstx 12y ago
  not sure if this goes without saying, but a huge one you missed is some sort of VCS. Use what you know and be sure to commit early and often. It can seem like a hassle to begin with, but once it saves you once you'll be very thankful your using it.
• ing33k 12y ago
  good list. I just want to add more voice to the unit testing part. It's natural to feel that testing will slow down the development process , but once you got some unit tests for the core parts of the app, it feels awesome and the developers will have a lot of confidence in the product.
  • briandear 12y ago
    TDD speeds up development, especially when adding features. Without full test coverage, you're relying upon manual regression testing to catch bugs. Once you have actual users, that could be catastrophic to deploy broken code without even realizing it.
• JoshTriplett 12y ago
  > * Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space. Depending on the service you're building, you can log too much. Consider the privacy and security implications of the existence of those logs; anything you log can be subpoenaed, but logs that don't exist cannot be. Consider anonymizing your logs from day 1, and only turning on non-anonymous logging upon a report from a user. Alternatively, give users a "report a problem" button, and save their last N minutes of otherwise-ephemeral logs only when they hit that button. You absolutely want to log enough to help you debug the service, but do you really need to archive old logs, or should you delete them entirely?
• ybrs 12y ago
  also i think one should take care of Reliability into account as well, if you have only one app server it will fail - maybe eventually - and you'll find yourself trying to ssh into the server at 2am, have at least a few app servers load balanced, have 2 db servers replicating with auto failover etc..
• landr0id 12y ago
  > Logging, logging, logging - I can't stress on this enough. When things break, logging is crucial in piecing together what happened. Use log rotation to archive old logs so they don't hog the disk space. How do most people manage activity logs? Currently what we have set up is the user id (if the user is logged in), IP address, URL they hit, user agent, and timestamp are all inserted into an activity logs table. For one particular site with an API that's being polled the size of the DB grew pretty large.
  • daigoba66 12y ago
    Logging every hit will always require a lot of space. But there are some tricks you can use to "compress" it: hash long strings like the URL and user agent and store the hash as binary instead of a string. A 100+ byte string can compress to just 16 or 32 bytes depending the hash your pick. Store the hash lookup in a separate table.
    • mandeepj 12y ago
      what is the benefit of your approach? The lookup table will still have data growth issues
      • tonyarkles 12y ago
        Presumably there will be many more hits than URLs, i.e. the URL table will grow much slower than the Hits table.
  • chrissnell 12y ago
    Off-machine logging. There are commercial services (we're using Papertrail but there are tons of them), roll-your-own solutions (Elasticsearch-Logstash-Kibana), and simple solutions (syslog). For an easy and simple solution, spin up a second instance and send logs to it via rsyslog over a private network interface. Most mature frameworks provide a method to send logs over syslog. It's UDP and very lightweight. Another plus: if you are compromised, you have another server with your logs and that server isn't running your vulnerable app.
• pbreit 12y ago
  Nailed it with first sentence: it takes 10k users to run an app with 10k users! And that will be, by far, the hardest part. The technical stuff will be pretty much trivial. Any decently constructed app on any decent framework (Rails, etc) on any decent host (AWS, DO) would be able to handle a 10k user app (probably maxing out at 1% online at same time) without breaking a sweat. And you will have plenty of time to build out the tech because it will probably take you many months to get to even a few thousand users (depending on what kind of app it is, of course).
  • bw00d 12y ago
    Thanks
• Gurkenmaster 12y ago
  > These days RAM is cheap Ironicially it's a completly different story on the consumer market. I bought my two 4GB sticks about 2 years ago and now they cost twice as much. 4GB can cost you $40 which is not cheap at all.
  • jacquesm 12y ago
    > 4GB can cost you $40 which is not cheap at all. That's totally negligible on the cost of the rest of the machine. Imagine $160 will get you 16G!! That's an absolutely enormous amount of memory, most power users would be more than satisfied with that. Not all that long ago that amount of RAM would cost more than a brand new mid sized car.
• bw00d 12y ago
  Thank you for your input.