7 ms·

This is relevant only for multimedia apps. I have fintech systems in production with 100k+ users with complex Gov app for entire country that runs on commodity

This is relevant only for multimedia apps. I have fintech systems in production with 100k+ users with complex Gov app for entire country that runs on commodity hardware (majority of work done by 1 backend server , 1 database server and all reporting by 1 reporting backend server using the same db). Based on our grafana metrics it can survive x10 number of users without upgrade of any kind. It runs on linux and dot net core and Sql Server. Most of the software is not multimedia in nature and those numbers are off the charts for such systems.

rlander 7y ago
I’ll bite. What’s a “fintech system” with “complex Gov app”?
- jermaustin1 7y ago
  Looking at his CV [1], it looks like he's done a lot of work inside the Serbiam Treasury. 1: https://gist.github.com/majkinetor/877d5174ba322fbb808cc47a8ded67ad https://gist.github.com/majkinetor/877d5174ba322fbb808cc47a8...
grezql 7y ago
SQL Server is a different beast. You get alot of performance enhancement out of box. Yes its costs money you save tremendous amount of tweaking time and headaches.
- viggity 7y ago
  or you can just use AzureSQL and essentially just pay what it costs for the box, because it is platform as a service. Its far cheaper (and easier to maintain) than what it'd cost to buy a SQL Server license and run it on a VM.
  - dillonmckay 7y ago
    A 32 core SQL Server license is $700.
    - mtVessel 7y ago
      This page[1] seems to indicate licensing SQL Server (Standard Ed) for 32 cores would cost $59,472 ($3,717 per 2 cores). Where do you get $700 from? [1] https://www.microsoft.com/en-us/sql-server/sql-server-2017-pricing https://www.microsoft.com/en-us/sql-server/sql-server-2017-p...
      - dillonmckay 7y ago
        https://indigosoftwarecompany.com/products/microsoft-sql-server-2017-standard-24-core-10-cal-64-bit-comp https://indigosoftwarecompany.com/products/microsoft-sql-ser...
        james_s_tayler 7y ago
        This site isn't Microsoft?
        vonseel 7y ago
        Those prices are too good to be true, reminds me of the sites that sell windows or office licenses to personal users for like 10 bucks.
        jabart 7y ago
        That is a 10 cal license, so you can only have 10 users using it, and that is 10 assigned users. Don't quote me on that but you have to use core licenses for web apps if you have internet users.
NicoJuicy 7y ago
Any tests on progress? I'm going on Postgress because of Bson support and no-sql with sql duplicate fields for search. Using .net core also. In general: I agree, there's rarely a case for really using the cloud. Page loads of my E-commerce project are 8ms for basket, I'm wondering what should kill it first on a big load, even without caching. Probably the database, not sure yet.
Kiro 7y ago
Yeah, it really depends on the kind of app. My app with a couple of hundred thousand users runs on a single $5 Digital Ocean droplet with standard PHP and MySQL.
- wackget 7y ago
  Wow, what's your app?
- leeoniya 7y ago
  > My app with a couple of hundred thousand users per year? per month? per day? simultaneously? doing what? it matters. i ask this as someone who runs a $40/mo Linode with a debian/nginx/node/mysql stack that's definitely 20x over-provisioned for an e-commerce site with 10k daily visitors, 15 simultaneous backend users (reporting, order-entry, CRM, analytics) and 0 caching tricks. i could easily run the site on any 5 year old laptop with an SSD and 8GB RAM. normalize/de-normalize when needed, understand and hand-write efficient SQL queries (ditch ORMs), choose small/fast libs carefully (or write your own), and you can easily serve 100k users per day on a single cheap VPS with no orchestration/replication/hz-scaling bullshit. definitely can't say the same about 200k simultaneous users - that would need proper hardware, but can still be a single server. Monoliths Are the Future: https://news.ycombinator.com/item?id=22193383 https://news.ycombinator.com/item?id=22193383
  - neillyons 7y ago
    What is the e-commerce site?
    - Demiurge 7y ago
      Are you offering a free stress test? :)
    - leeoniya 7y ago
      i'm working on a blog post about the tech choices & stack. maybe it'll make it to the front page in a month or two - that'll be a good stress test!
karambir 7y ago
Yeah, for normal web app, we can easily have 100x users for each step mentioned in article. For our company, we had more than 500k users with 1 small nginx + 1 medium appServer(with autoscaling, though never needed it) + 1 small cache server and RDS till now. We just added a aws managed load balancer into the setup and think it might be overkill. For a client with NewsFeed needs, I used a dedicated server(64GB, 2TB space) to run nginx, app, cache, huge elasticsearch and postgres. It was great(and cheap) option for an MVP and let them validate the product for few months with >10k users. It was awesome to learn a few years ago, how much compute power we don't use.
bob1029 7y ago
Thank you for this post. I read "10 Users: Split out the Database Layer" and about had an aneurysm. I also work with fintech systems built upon .NET Core and have similar experiences regarding scaling of these solutions. You can get an incredible amount of throughput from a single box if you are careful with the technologies you use. A single .NET Core web API process using Kestrel and whatever RDBMS (even SQLite) can absolutely devour requests in the <1 megabyte range. I would feel confident putting the largest customer we can imagine x10 on a single 16~32 core server for the solution we provide today. Obviously, if you are pushing any form of multimedia this starts to break down rapidly.
- thinkmassive 7y ago
  It seems like they recommend splitting out the database at the start because using a managed service is much easier than properly managing your own production database.
  - throwaway5752 7y ago
    I can't speak for high performance/near or realtime system people but I would not trust a managed database service for those needs. My experience is that the managed offerings lag behind upstream versions substantially and usually are economical because they are multitenant. So you have a bit less predictability in io wait / cpu queue, lose host kernel level tunings (page sizes or hugepages, share memory allocation, etc), and - not naming names - some managed db services are so behind they lack critical query planner profiling features. That's not even going into application workload specific tuning for various nosql stores. This is a nice article but its audience is people that haven't scaled up a system but are trying to cope with success. It's not great generalized scaling advice.
    - bob1029 7y ago
      All I will say is that our latency getting a business entity in or out of a SQLite database (running on top of NVMe flash) is on the order of tens to hundreds of microseconds. There will never be a hosted/cloud offering that can even remotely approach this without installing some "bring the cloud to you" appliance in your datacenter.
kernoble 7y ago
Hey, really specific question regarding your deployment. A teammate of mine reported difficulties with the .net SqlServer database driver establishing connection from a linux client (container instance based on the public .net core image) to a SqlServer instance (on windows) . Are you familiar with this problem? I think moving our systems over to .NET Core on linux is the future, but this one experience has somewhat soured the idea for some decision makers and the team managing our db.
- mikem170 7y ago
  I bumped into that at the office - connecting from a linux server to a microsoft sql server. We were able to connect after installing unixodbc and microsoft's odbc driver for sql server on linux. Note that a local mssql account is required, you can't connect with microsoft domain authentication from a linux box. We haven't finished testing and tweaking, but so far it looks like it's working.
  - kernoble 7y ago
    Right, the auth from non-windows was a separate organizational hurdle we had to overcome, there was real hesitancy about not using a GMSA windows user to auth with SqlServer. When you say "local mssql account is required" do you mean a user or some other entity on the linux client? Or do you mean an mssql user that the client with authenticate as?
    - mikem170 7y ago
      an mssql user account
      - kernoble 7y ago
        Two more questions (if you see this, I know it's been a few days), are you running SQL Server on Linux or Windows? And are you using encrypted connections?
tempestn 7y ago
Even though the absolute numbers may well not apply to other types of apps, the general concepts of how to scale do. We have 100k+ users on effectively a single box too (actually two for redundancy, ease of upgrades, etc., but one can handle it), but this is a great overview of how to think about scaling beyond that, however many users that's done at. Honestly when I was reading the article I read that 1/10/100 as more of a unitless degree of scale than actual numbers of humans.
holoduke 7y ago
Yes exactly. I have a an app with about 1m concurrent users at peak hours. For years it is running perfectly fine on a single 8 core machine. Linux - > Nginx -> memcache -> nodejs -> filesystem. Yes filesystem. All content is fetched from files. Another process is populating all files. I pay 40 dollars/m for this self managed machine.
- mwcampbell 7y ago
  What do you do when that one machine goes offline? Say there's a power outage, or a networking issue, or you have to reboot with a patched kernel. Now your 1M concurrent users can't use your application. This is why we go to the trouble to use multiple machines, ideally in multiple availability zones if we're using one of the big three cloud providers.
  - jjeaff 7y ago
    That could be solved by mirroring the system to another $40/mo machine in another data center. Redundancy was around long before cloud compute was a thing.
  - zrobotics 7y ago
    What is wrong in this case w/ multiple VPS's? For instance, I run a shipping system that our entire operation depends on. I can't afford downtime either, as that would shutdown our entire shipping department. However, 1 server running on-prem, 1 remote in shared hosting and 2 failover servers on DigitalOcean costs ~$120/month. What would a managed cloud like AWS or Azure provide besides higher expenses? Last I checked, AWS would be ~$550/month based on their calculator. IT departments managed redundancy long before cloud was a thing, and it's certainly possible to achieve acceptable uptime without the managed services. I realize that I still rely on DO for their uptime, and I'm pricing renting rackspace for that reason. However, uptime is currently within acceptable levels (ie I haven't had to explain to the CEO why all his employees are standing around). We had tried 2 SaaS vendors before our current on-prem solution, and both had had downtime that cost us money. Both were hosted in AWS, so clearly that isn't the panacea for ultimate reliability. If our on-prem server goes down, I get a call. However, I got calls when our SaaS vendor went down as well; sitting on line with their tech support wasn't any great comfort. The same graceful failover processes are accessible on machines you control, there's no reason that you have to run on one machine if you avoid the cloud. The biggest hurdle for me to implement this was database concurrency with multiple servers, but a cloud solution wouldn't have done anything to solve that problem. [edit:] typos and miss-typed AWS pricing due to mobile keyboard.
arsenico 7y ago
It also depends on the nature of the fintech app. Imagine running a 100K + users on a stock trading app for day traders, where latency matters, and a lot of streaming happens.

rlander 7y ago

I’ll bite. What’s a “fintech system” with “complex Gov app”?

jermaustin1 7y ago
Looking at his CV [1], it looks like he's done a lot of work inside the Serbiam Treasury. 1: https://gist.github.com/majkinetor/877d5174ba322fbb808cc47a8ded67ad https://gist.github.com/majkinetor/877d5174ba322fbb808cc47a8...

grezql 7y ago

SQL Server is a different beast. You get alot of performance enhancement out of box. Yes its costs money you save tremendous amount of tweaking time and headaches.

viggity 7y ago
or you can just use AzureSQL and essentially just pay what it costs for the box, because it is platform as a service. Its far cheaper (and easier to maintain) than what it'd cost to buy a SQL Server license and run it on a VM.
- dillonmckay 7y ago
  A 32 core SQL Server license is $700.
  - mtVessel 7y ago
    This page[1] seems to indicate licensing SQL Server (Standard Ed) for 32 cores would cost $59,472 ($3,717 per 2 cores). Where do you get $700 from? [1] https://www.microsoft.com/en-us/sql-server/sql-server-2017-pricing https://www.microsoft.com/en-us/sql-server/sql-server-2017-p...
    - dillonmckay 7y ago
      https://indigosoftwarecompany.com/products/microsoft-sql-server-2017-standard-24-core-10-cal-64-bit-comp https://indigosoftwarecompany.com/products/microsoft-sql-ser...
      - james_s_tayler 7y ago
        This site isn't Microsoft?
      - vonseel 7y ago
        Those prices are too good to be true, reminds me of the sites that sell windows or office licenses to personal users for like 10 bucks.
      - jabart 7y ago
        That is a 10 cal license, so you can only have 10 users using it, and that is 10 assigned users. Don't quote me on that but you have to use core licenses for web apps if you have internet users.

NicoJuicy 7y ago

Any tests on progress? I'm going on Postgress because of Bson support and no-sql with sql duplicate fields for search. Using .net core also. In general: I agree, there's rarely a case for really using the cloud. Page loads of my E-commerce project are 8ms for basket, I'm wondering what should kill it first on a big load, even without caching. Probably the database, not sure yet.

Kiro 7y ago

Yeah, it really depends on the kind of app. My app with a couple of hundred thousand users runs on a single $5 Digital Ocean droplet with standard PHP and MySQL.

wackget 7y ago
Wow, what's your app?
leeoniya 7y ago
> My app with a couple of hundred thousand users per year? per month? per day? simultaneously? doing what? it matters. i ask this as someone who runs a $40/mo Linode with a debian/nginx/node/mysql stack that's definitely 20x over-provisioned for an e-commerce site with 10k daily visitors, 15 simultaneous backend users (reporting, order-entry, CRM, analytics) and 0 caching tricks. i could easily run the site on any 5 year old laptop with an SSD and 8GB RAM. normalize/de-normalize when needed, understand and hand-write efficient SQL queries (ditch ORMs), choose small/fast libs carefully (or write your own), and you can easily serve 100k users per day on a single cheap VPS with no orchestration/replication/hz-scaling bullshit. definitely can't say the same about 200k simultaneous users - that would need proper hardware, but can still be a single server. Monoliths Are the Future: https://news.ycombinator.com/item?id=22193383 https://news.ycombinator.com/item?id=22193383
- neillyons 7y ago
  What is the e-commerce site?
  - Demiurge 7y ago
    Are you offering a free stress test? :)
  - leeoniya 7y ago
    i'm working on a blog post about the tech choices & stack. maybe it'll make it to the front page in a month or two - that'll be a good stress test!

karambir 7y ago

Yeah, for normal web app, we can easily have 100x users for each step mentioned in article. For our company, we had more than 500k users with 1 small nginx + 1 medium appServer(with autoscaling, though never needed it) + 1 small cache server and RDS till now. We just added a aws managed load balancer into the setup and think it might be overkill. For a client with NewsFeed needs, I used a dedicated server(64GB, 2TB space) to run nginx, app, cache, huge elasticsearch and postgres. It was great(and cheap) option for an MVP and let them validate the product for few months with >10k users. It was awesome to learn a few years ago, how much compute power we don't use.

bob1029 7y ago

Thank you for this post. I read "10 Users: Split out the Database Layer" and about had an aneurysm. I also work with fintech systems built upon .NET Core and have similar experiences regarding scaling of these solutions. You can get an incredible amount of throughput from a single box if you are careful with the technologies you use. A single .NET Core web API process using Kestrel and whatever RDBMS (even SQLite) can absolutely devour requests in the <1 megabyte range. I would feel confident putting the largest customer we can imagine x10 on a single 16~32 core server for the solution we provide today. Obviously, if you are pushing any form of multimedia this starts to break down rapidly.

thinkmassive 7y ago
It seems like they recommend splitting out the database at the start because using a managed service is much easier than properly managing your own production database.
- throwaway5752 7y ago
  I can't speak for high performance/near or realtime system people but I would not trust a managed database service for those needs. My experience is that the managed offerings lag behind upstream versions substantially and usually are economical because they are multitenant. So you have a bit less predictability in io wait / cpu queue, lose host kernel level tunings (page sizes or hugepages, share memory allocation, etc), and - not naming names - some managed db services are so behind they lack critical query planner profiling features. That's not even going into application workload specific tuning for various nosql stores. This is a nice article but its audience is people that haven't scaled up a system but are trying to cope with success. It's not great generalized scaling advice.
  - bob1029 7y ago
    All I will say is that our latency getting a business entity in or out of a SQLite database (running on top of NVMe flash) is on the order of tens to hundreds of microseconds. There will never be a hosted/cloud offering that can even remotely approach this without installing some "bring the cloud to you" appliance in your datacenter.

kernoble 7y ago

Hey, really specific question regarding your deployment. A teammate of mine reported difficulties with the .net SqlServer database driver establishing connection from a linux client (container instance based on the public .net core image) to a SqlServer instance (on windows) . Are you familiar with this problem? I think moving our systems over to .NET Core on linux is the future, but this one experience has somewhat soured the idea for some decision makers and the team managing our db.

mikem170 7y ago
I bumped into that at the office - connecting from a linux server to a microsoft sql server. We were able to connect after installing unixodbc and microsoft's odbc driver for sql server on linux. Note that a local mssql account is required, you can't connect with microsoft domain authentication from a linux box. We haven't finished testing and tweaking, but so far it looks like it's working.
- kernoble 7y ago
  Right, the auth from non-windows was a separate organizational hurdle we had to overcome, there was real hesitancy about not using a GMSA windows user to auth with SqlServer. When you say "local mssql account is required" do you mean a user or some other entity on the linux client? Or do you mean an mssql user that the client with authenticate as?
  - mikem170 7y ago
    an mssql user account
    - kernoble 7y ago
      Two more questions (if you see this, I know it's been a few days), are you running SQL Server on Linux or Windows? And are you using encrypted connections?

tempestn 7y ago

Even though the absolute numbers may well not apply to other types of apps, the general concepts of how to scale do. We have 100k+ users on effectively a single box too (actually two for redundancy, ease of upgrades, etc., but one can handle it), but this is a great overview of how to think about scaling beyond that, however many users that's done at. Honestly when I was reading the article I read that 1/10/100 as more of a unitless degree of scale than actual numbers of humans.

holoduke 7y ago

Yes exactly. I have a an app with about 1m concurrent users at peak hours. For years it is running perfectly fine on a single 8 core machine. Linux - > Nginx -> memcache -> nodejs -> filesystem. Yes filesystem. All content is fetched from files. Another process is populating all files. I pay 40 dollars/m for this self managed machine.

mwcampbell 7y ago
What do you do when that one machine goes offline? Say there's a power outage, or a networking issue, or you have to reboot with a patched kernel. Now your 1M concurrent users can't use your application. This is why we go to the trouble to use multiple machines, ideally in multiple availability zones if we're using one of the big three cloud providers.
- jjeaff 7y ago
  That could be solved by mirroring the system to another $40/mo machine in another data center. Redundancy was around long before cloud compute was a thing.
- zrobotics 7y ago
  What is wrong in this case w/ multiple VPS's? For instance, I run a shipping system that our entire operation depends on. I can't afford downtime either, as that would shutdown our entire shipping department. However, 1 server running on-prem, 1 remote in shared hosting and 2 failover servers on DigitalOcean costs ~$120/month. What would a managed cloud like AWS or Azure provide besides higher expenses? Last I checked, AWS would be ~$550/month based on their calculator. IT departments managed redundancy long before cloud was a thing, and it's certainly possible to achieve acceptable uptime without the managed services. I realize that I still rely on DO for their uptime, and I'm pricing renting rackspace for that reason. However, uptime is currently within acceptable levels (ie I haven't had to explain to the CEO why all his employees are standing around). We had tried 2 SaaS vendors before our current on-prem solution, and both had had downtime that cost us money. Both were hosted in AWS, so clearly that isn't the panacea for ultimate reliability. If our on-prem server goes down, I get a call. However, I got calls when our SaaS vendor went down as well; sitting on line with their tech support wasn't any great comfort. The same graceful failover processes are accessible on machines you control, there's no reason that you have to run on one machine if you avoid the cloud. The biggest hurdle for me to implement this was database concurrency with multiple servers, but a cloud solution wouldn't have done anything to solve that problem. [edit:] typos and miss-typed AWS pricing due to mobile keyboard.

arsenico 7y ago

It also depends on the nature of the fintech app. Imagine running a 100K + users on a stock trading app for day traders, where latency matters, and a lot of streaming happens.