3 ms·

Out of curiosity, you're paying for all the infrastructure out of pocket? And tangentially, what is your setup? Presumably browsers are among the more resource

by nateb2022 3y ago

Out of curiosity, you're paying for all the infrastructure out of pocket? And tangentially, what is your setup? Presumably browsers are among the more resource hungry applications you could run in the cloud, are you spawning each browser in a container in some large server or are you spinning up/shutting down small instances on demand?

keepamovin 3y ago
Hey nateb2022 thanks for the questions: Out of curiosity, you're paying for all the infrastructure out of pocket? Yeha, it's a loss leader for marketing our product. We're going to impose some more usage caps for the signupless version, and restrict to a 2-week free tier for the signed up one, in a little while. After that it will be 6/month or 60/yearly. You can sign up today at https://browse.cloudtabs.net https://browse.cloudtabs.net and subscribe now to get a 4-week free tier. :) And tangentially, what is your setup? Ok, we don't use a load balancer because that would be a single point of choke/failure and this is basically a real-time streaming application with audio/video. Because of that each client has direct WS/WebRTC connections to their home server. In future, we'll enable the ability for folks to shift their home server around, but for now it's locked in from signup. We just have some geodistributed servers and run BrowserBox instances on their with some simple scaling software that we're not making the source available for. The code for BrowserBox is at: https://github.com/BrowserBox https://github.com/BrowserBox and it used to be AGPL but too many companies (yep, actual companies, US companies even) abused that to do closed source mods without licensing, so we made it commercial and will add a license key soon. Presumably browsers are among the more resource hungry applications you could run in the cloud? Reaonsable assumption but not always true. At rest, they use basically nothing. Chrome runs headless and with 0 tabs opens consumes essentially zero memory and 0 cpu. It's all about what the client is browsing. Usin' video? More CPU. Browsing HN? Basically nothing. It's very dynamic but not as hungry as you may think. BrowserBox is free for non-commercial individual/government/public school use (at least right now!!) so you can just DL it and play around to see that it doesn't use much. The BrowserBox node services don't do any heavy lifting, don't use any databases and are all designed around the libuv/io_uring async task loop to make them as efficient as possible. They all sit a 0% CPU and 3-4 MB of memory each when idle. The most expensive thing we do is use pulseaudio, which regardless of what happens consumes about 1%-3% of 1 core at rest. are you spawning each browser in a container in some large server or are you spinning up/shutting down small instances on demand? It's faster to just run big servers and than spin up tiny ones. VPS Instance startup time is in the minutes typically, and longer with a custom image. While you could keep a pool to buffer demand, this is wasteful and more complex than just having constantly running big servers that already have BB installed, then starting the app as a new user on that server when anyone needs one. Simple, fast, effective and very easy to manage. For a small number of users, if you really want to squeeze costs as low as possible, using on demand instances 1 for each browser and just eating the startup time is the way to go. BrowserBox is containerized but we don't use containers for deployment. We don't need the 'complex simplicity' of Docker, as BrowserBox is already written with running as a multiuser Unix app/daemon in mind from the start. If there was a cloud service that could just take the BrowserBox docker image and provision that at infinite scale and do it on PAYG that would be ideal, but AFAIK there is no such solution. I think K8s has issues with WS/WebRTC and persistent connections and we need a hostname per instance anyway, so there's complexity. We don't have any K8S experts so we didn't explore it beyond that. We do explore ways to do what we do better tho. If you know a solution there I'd be happy to chat to learn more how you think it could help? :) Reach out if you like: cris@dosyago.com
- nateb2022 3y ago
  This is extremely insightful, thank you for taking the time!
  - keepamovin 3y ago
    You’re welcome. It’s always welcome to see technical questions delivered respectfully. Props to you!

keepamovin 3y ago

Hey nateb2022 thanks for the questions: Out of curiosity, you're paying for all the infrastructure out of pocket? Yeha, it's a loss leader for marketing our product. We're going to impose some more usage caps for the signupless version, and restrict to a 2-week free tier for the signed up one, in a little while. After that it will be 6/month or 60/yearly. You can sign up today at https://browse.cloudtabs.net https://browse.cloudtabs.net and subscribe now to get a 4-week free tier. :) And tangentially, what is your setup? Ok, we don't use a load balancer because that would be a single point of choke/failure and this is basically a real-time streaming application with audio/video. Because of that each client has direct WS/WebRTC connections to their home server. In future, we'll enable the ability for folks to shift their home server around, but for now it's locked in from signup. We just have some geodistributed servers and run BrowserBox instances on their with some simple scaling software that we're not making the source available for. The code for BrowserBox is at: https://github.com/BrowserBox https://github.com/BrowserBox and it used to be AGPL but too many companies (yep, actual companies, US companies even) abused that to do closed source mods without licensing, so we made it commercial and will add a license key soon. Presumably browsers are among the more resource hungry applications you could run in the cloud? Reaonsable assumption but not always true. At rest, they use basically nothing. Chrome runs headless and with 0 tabs opens consumes essentially zero memory and 0 cpu. It's all about what the client is browsing. Usin' video? More CPU. Browsing HN? Basically nothing. It's very dynamic but not as hungry as you may think. BrowserBox is free for non-commercial individual/government/public school use (at least right now!!) so you can just DL it and play around to see that it doesn't use much. The BrowserBox node services don't do any heavy lifting, don't use any databases and are all designed around the libuv/io_uring async task loop to make them as efficient as possible. They all sit a 0% CPU and 3-4 MB of memory each when idle. The most expensive thing we do is use pulseaudio, which regardless of what happens consumes about 1%-3% of 1 core at rest. are you spawning each browser in a container in some large server or are you spinning up/shutting down small instances on demand? It's faster to just run big servers and than spin up tiny ones. VPS Instance startup time is in the minutes typically, and longer with a custom image. While you could keep a pool to buffer demand, this is wasteful and more complex than just having constantly running big servers that already have BB installed, then starting the app as a new user on that server when anyone needs one. Simple, fast, effective and very easy to manage. For a small number of users, if you really want to squeeze costs as low as possible, using on demand instances 1 for each browser and just eating the startup time is the way to go. BrowserBox is containerized but we don't use containers for deployment. We don't need the 'complex simplicity' of Docker, as BrowserBox is already written with running as a multiuser Unix app/daemon in mind from the start. If there was a cloud service that could just take the BrowserBox docker image and provision that at infinite scale and do it on PAYG that would be ideal, but AFAIK there is no such solution. I think K8s has issues with WS/WebRTC and persistent connections and we need a hostname per instance anyway, so there's complexity. We don't have any K8S experts so we didn't explore it beyond that. We do explore ways to do what we do better tho. If you know a solution there I'd be happy to chat to learn more how you think it could help? :) Reach out if you like: cris@dosyago.com

nateb2022 3y ago
This is extremely insightful, thank you for taking the time!
- keepamovin 3y ago
  You’re welcome. It’s always welcome to see technical questions delivered respectfully. Props to you!