7 ms·

The alternative to using last-mile transit is for everyone to have a virtual server somewhere. Even this seems to be a bit off since 90% of all people only ever

by sqrt17 15y ago

The alternative to using last-mile transit is for everyone to have a virtual server somewhere. Even this seems to be a bit off since 90% of all people only ever consume content and can't be bothered with anything like creating their own content.

DanWaterworth 15y ago
Even if just a small minority of people had VPSs that were on some P2P network it would be better than the current system of monopolies.
kragen 15y ago
I think that's a good idea. It might not even have to be virtual, given servers the size of the Raspberry Pi: http://www.geek.com/articles/chips/raspberry-pi-25-pc-goes-into-alpha-production-20110728/ http://www.geek.com/articles/chips/raspberry-pi-25-pc-goes-i...
- DanWaterworth 15y ago
  It would be cheaper to buy big servers with enormous amounts of RAM and use visualization than to have thousands of raspberry pis. Unless you mean each person has a raspberry pi on there home network, in which case you're still using expensive last-mile bandwidth.
  - kragen 15y ago
    It may or may not be cheaper. I was envisioning ten thousand Raspberry Pis in your local telephone central office.
  - lsc 15y ago
    I have been following this math /very closely/ - and I am fairly certain that if someone wanted to build a board like this in quantity, with more ram, of course, and if I could get my hands on a suitable power supply so I would have one psu for every twenty of these servers, it would likely come out pretty close to the cost of a big box. The problem is that all boards I can get for a reasonable price in the quantities I can afford have a bunch of crap on them I don't need; video and the like, which pushes the cost up higher than I'd like. I've been talking with a friend of mine who does ARM stuff about maybe seeing if we could get venture funding, as I think we could get into reasonable quantities in the low-six figure range. But, like I said, as far as I can tell, right now the cost difference is 'close' - even if I could get a custom board with lots of ram and no video crap, for most things it wouldn't be dramatically better than virtualizing a massive supermicro box, and considering that design time would mean that the thing would be kinda old by the time we were done, there probably isn't a lot of profit to be had. But, if trends continue, this may change; and I can tell you right now, customers are willing to pay a premium for dedicated hardware, and for good reason. (Yes, you have the management problem of thousands of servers, but most of the things that don't have PXE have some way to boot from serial, and I can deal with that. Having managed thousands of virtuals and tens of thousands of hardware nodes, I'm not convinced that virtuals are all that much easier to manage, once you have a good system in place.)
    - DanWaterworth 15y ago
      What your not taking into account is that thousands of small servers will likely fail faster than dozens of big servers. I think you'll find that this and the fact that you'd have to maintain much more hardware (the servers themselves and switches) tips the balance.
      - lsc 15y ago
        I've managed clusters of 60K plus large servers. I mean, I didn't manage them by myself; my primary job was automating hardware failure detection, but I took my turn on the pager. My personal focus was to see to it that bad hardware didn't go back into production; as someone who shared pager, that was... offensive. But really, hardware failure is less of an issue than people seem to say. I mean, sure, it happens. But not nearly as often as software issues that bring down the box, unless you count bad hard drives; and those happen all the time. Even with 60K servers, I quickly learned that if I found a lot of bad servers that were bad in similar ways, I needed to be careful before having them sent back to the vendor. More than once I embarrassed myself blaming software problems on the hardware. For the first 3-5 years of it's life, if the computer was properly assembled, very likely the only things that will need replacement are the disks. My thought would be to treat the things as hard drives. two options: For the lowest tier of support, if your system fails, I leave it and provision a new one. Leave the dead ones in the rack until it's time to upgrade. The second option is to put the boards on some sort of hot-swap power backplane, and when the thing fails, pop it out and replace it like a hard drive. The question would be "how expensive is it to make the hot-swap system, and does the manufacturer give me a warranty that makes it worth my time to swap 'em out and send 'em back." I suspect that if I can get 'em to give me a discount to not have a warranty, my best bet would be to set it up so I could hot swap whatever i used for storage, but if anything but storage failed, I'd just leave the system in place. sure, maybe I'd want to offer 'premium' support with mirrored storage where I'd swap your storage if it fails, and yeah, in that case, expenses would be higher than a giant server with fewer disks. (by fewer disks, I mean fewer disks per 256mb guest. Obviously, it has more disks total) I/O performance is probably also going to be worse on the giant server with fewer disks, so I can charge more for the 'premium' dedicated hardware with dedicated I/O. I mean, you get into the 'all troubleshooting must be automated' range much faster with those than with physical hardware, but somewhere between a couple hundred and a couple thousand hardware nodes, you are going to get there either way.
- deleted 15y ago
  [deleted]
illume 15y ago
Most people create content. See facebook, and email as proof if you are skeptical.

by sqrt17 15y ago

DanWaterworth 15y ago

Even if just a small minority of people had VPSs that were on some P2P network it would be better than the current system of monopolies.

kragen 15y ago

I think that's a good idea. It might not even have to be virtual, given servers the size of the Raspberry Pi: http://www.geek.com/articles/chips/raspberry-pi-25-pc-goes-into-alpha-production-20110728/ http://www.geek.com/articles/chips/raspberry-pi-25-pc-goes-i...

DanWaterworth 15y ago
It would be cheaper to buy big servers with enormous amounts of RAM and use visualization than to have thousands of raspberry pis. Unless you mean each person has a raspberry pi on there home network, in which case you're still using expensive last-mile bandwidth.
- kragen 15y ago
  It may or may not be cheaper. I was envisioning ten thousand Raspberry Pis in your local telephone central office.
- lsc 15y ago
  I have been following this math /very closely/ - and I am fairly certain that if someone wanted to build a board like this in quantity, with more ram, of course, and if I could get my hands on a suitable power supply so I would have one psu for every twenty of these servers, it would likely come out pretty close to the cost of a big box. The problem is that all boards I can get for a reasonable price in the quantities I can afford have a bunch of crap on them I don't need; video and the like, which pushes the cost up higher than I'd like. I've been talking with a friend of mine who does ARM stuff about maybe seeing if we could get venture funding, as I think we could get into reasonable quantities in the low-six figure range. But, like I said, as far as I can tell, right now the cost difference is 'close' - even if I could get a custom board with lots of ram and no video crap, for most things it wouldn't be dramatically better than virtualizing a massive supermicro box, and considering that design time would mean that the thing would be kinda old by the time we were done, there probably isn't a lot of profit to be had. But, if trends continue, this may change; and I can tell you right now, customers are willing to pay a premium for dedicated hardware, and for good reason. (Yes, you have the management problem of thousands of servers, but most of the things that don't have PXE have some way to boot from serial, and I can deal with that. Having managed thousands of virtuals and tens of thousands of hardware nodes, I'm not convinced that virtuals are all that much easier to manage, once you have a good system in place.)
  - DanWaterworth 15y ago
    What your not taking into account is that thousands of small servers will likely fail faster than dozens of big servers. I think you'll find that this and the fact that you'd have to maintain much more hardware (the servers themselves and switches) tips the balance.
    - lsc 15y ago
      I've managed clusters of 60K plus large servers. I mean, I didn't manage them by myself; my primary job was automating hardware failure detection, but I took my turn on the pager. My personal focus was to see to it that bad hardware didn't go back into production; as someone who shared pager, that was... offensive. But really, hardware failure is less of an issue than people seem to say. I mean, sure, it happens. But not nearly as often as software issues that bring down the box, unless you count bad hard drives; and those happen all the time. Even with 60K servers, I quickly learned that if I found a lot of bad servers that were bad in similar ways, I needed to be careful before having them sent back to the vendor. More than once I embarrassed myself blaming software problems on the hardware. For the first 3-5 years of it's life, if the computer was properly assembled, very likely the only things that will need replacement are the disks. My thought would be to treat the things as hard drives. two options: For the lowest tier of support, if your system fails, I leave it and provision a new one. Leave the dead ones in the rack until it's time to upgrade. The second option is to put the boards on some sort of hot-swap power backplane, and when the thing fails, pop it out and replace it like a hard drive. The question would be "how expensive is it to make the hot-swap system, and does the manufacturer give me a warranty that makes it worth my time to swap 'em out and send 'em back." I suspect that if I can get 'em to give me a discount to not have a warranty, my best bet would be to set it up so I could hot swap whatever i used for storage, but if anything but storage failed, I'd just leave the system in place. sure, maybe I'd want to offer 'premium' support with mirrored storage where I'd swap your storage if it fails, and yeah, in that case, expenses would be higher than a giant server with fewer disks. (by fewer disks, I mean fewer disks per 256mb guest. Obviously, it has more disks total) I/O performance is probably also going to be worse on the giant server with fewer disks, so I can charge more for the 'premium' dedicated hardware with dedicated I/O. I mean, you get into the 'all troubleshooting must be automated' range much faster with those than with physical hardware, but somewhere between a couple hundred and a couple thousand hardware nodes, you are going to get there either way.
deleted 15y ago
[deleted]

illume 15y ago

Most people create content. See facebook, and email as proof if you are skeptical.