3 ms·

Yeah, I've been following that work for quite a while, it's really impressive! I'm a big fan of Cassandra in general (in fact, I wrote an example application t

by ericflo 16y ago

Yeah, I've been following that work for quite a while, it's really impressive! I'm a big fan of Cassandra in general (in fact, I wrote an example application to help teach beginners about it: https://github.com/ericflo/twissandra https://github.com/ericflo/twissandra). The decision to go with PostgreSQL over Cassandra (or another distributed system like Riak or HBase) was simply because it gave us the most flexibility to change our product quickly while we operate at low scale. And if I'm honest with myself, we're operating at very low scale right now. In the future as we scale up, Cassandra's distributed counters will be one of the first places we look.

saurik 16y ago
Are you willing to say how many simultaneous users you are handling at this point? (I obviously totally understand if you aren't.)
- ericflo 16y ago
  Unfortunately I can't give out any exact numbers :(
  - saurik 16y ago
    I realize you may also not want to answer this, and that's also alright, but I figure I may as well ask, as I really am curious (and maybe someone else will answer from the perspective of their company): why not? Whenever anyone has asked me for a stat on Cydia, whether it be active devices over some time range, daily revenue, costs related to <insert-subproject-here>, or what have you, I tend to break out a quick SQL query and provide an exact answer (assuming the question is of something I an measure: not all are); is this stupid of me? I've noticed a ton of companies refuse to disclose numbers, and I've always assumed that the result will be that anyone listening will just assume "ok, so almost none then" unless you give them a good answer and back up how you calculated it, but does this actually "hurt the cause"? (I do not have the benefits of the years if startup experience that you have access to by being a part of Y Combinator, so I try to get advice whenever I can ;P.)
    - axod 16y ago
      The top 20 public groups topped out at about 250 users last night, add on a bit for the other public groups and a little for private groups. Maybe 400 concurrent users max so far? Just my guess, but I'd be surprised if it's orders of magnitude different.
foobarbazetc 16y ago
Just FYI: we run a very large web service with purely PostgreSQL, and have no need for sharding or anything of the sort. Scale up first. Use redis or memcached to store 'overview' stats. If you lose those stats, recalc from Pg.
- ericflo 16y ago
  That's exactly the approach we're taking. We're not going to spend any time prematurely worrying about various scaling things until we can forecast that it's legitimately going to be a problem.
  - foobarbazetc 16y ago
    Great to hear. :) While we're here, here are some PG tips: 1. Make sure your PG install is fully optimized. I've seen countless postgresql.conf's at major companies with no opts what so ever. Take a look at this book: http://www.2ndquadrant.com/books/postgresql-9-0-high-performance/ http://www.2ndquadrant.com/books/postgresql-9-0-high-perform... . 2. Make sure you're using PG 9+. 3. Use pgbouncer for connection pooling and try to write your app so you can use 'pool_mode = transaction'. 4. Please don't run PG virtualized.
    - ericflo 16y ago
      We're good on all fronts except 4, and the transaction pool mode :)

by ericflo 16y ago

saurik 16y ago

Are you willing to say how many simultaneous users you are handling at this point? (I obviously totally understand if you aren't.)

ericflo 16y ago
Unfortunately I can't give out any exact numbers :(
- saurik 16y ago
  I realize you may also not want to answer this, and that's also alright, but I figure I may as well ask, as I really am curious (and maybe someone else will answer from the perspective of their company): why not? Whenever anyone has asked me for a stat on Cydia, whether it be active devices over some time range, daily revenue, costs related to <insert-subproject-here>, or what have you, I tend to break out a quick SQL query and provide an exact answer (assuming the question is of something I an measure: not all are); is this stupid of me? I've noticed a ton of companies refuse to disclose numbers, and I've always assumed that the result will be that anyone listening will just assume "ok, so almost none then" unless you give them a good answer and back up how you calculated it, but does this actually "hurt the cause"? (I do not have the benefits of the years if startup experience that you have access to by being a part of Y Combinator, so I try to get advice whenever I can ;P.)
  - axod 16y ago
    The top 20 public groups topped out at about 250 users last night, add on a bit for the other public groups and a little for private groups. Maybe 400 concurrent users max so far? Just my guess, but I'd be surprised if it's orders of magnitude different.

foobarbazetc 16y ago

Just FYI: we run a very large web service with purely PostgreSQL, and have no need for sharding or anything of the sort. Scale up first. Use redis or memcached to store 'overview' stats. If you lose those stats, recalc from Pg.

ericflo 16y ago
That's exactly the approach we're taking. We're not going to spend any time prematurely worrying about various scaling things until we can forecast that it's legitimately going to be a problem.
- foobarbazetc 16y ago
  Great to hear. :) While we're here, here are some PG tips: 1. Make sure your PG install is fully optimized. I've seen countless postgresql.conf's at major companies with no opts what so ever. Take a look at this book: http://www.2ndquadrant.com/books/postgresql-9-0-high-performance/ http://www.2ndquadrant.com/books/postgresql-9-0-high-perform... . 2. Make sure you're using PG 9+. 3. Use pgbouncer for connection pooling and try to write your app so you can use 'pool_mode = transaction'. 4. Please don't run PG virtualized.
  - ericflo 16y ago
    We're good on all fronts except 4, and the transaction pool mode :)