9 ms·

I have two conflicting responses to what I am seeing here ... First, this is awesome. Just like I want to live in a world where people are paying picodollars

by rsync 12y ago

I have two conflicting responses to what I am seeing here ... First, this is awesome. Just like I want to live in a world where people are paying picodollars for cloud storage[1], I also want to live in a world where a bunch of mac pro cylinders are racked up in a datacenter. Very cool. Second, this is complete silliness. I'm not going to go down the rabbithole of flops per dollar, but there is no way that you can't build a hackintosh 1U with dual cpus and multiple GPUs and not come out big money ahead. Whatever management overhead gets added by playing the hackintosh cat and mouse is certainly less than building new things out of sheet metal. Let me say one other thing: right around mid 2000 was when certain companies started selling fancy third-party rack chassis gizmos for the Sun e4500, which was the cadillac of datacenter servers at the time. Huge specs on paper, way underpowered for the money they cost ($250k+) and the epitome of Suns brand-value. And there were suddenly new and fancy ways to rack and roll them. This reminds me a lot of that time, and that time didn't last long... [1] Our esteemed competitor, tarsnap.

tinco 12y ago
Obviously, but running OSX on non-Apple hardware is a violation of its EULA. I have contacted a lawyer for this (I wanted to run Hackintosh in the office), the language is very clear. The author of the software has the full power to license its use to you with any restrictions they find necessary no matter how ridiculous. If Apple only sells you the license if you promise not to run it on a thursday, you'll be in violation of their terms if you run it on a thursday.
- nulltype 12y ago
  Even if that were not the case, I doubt the performance of the OS X image pipeline is tuned for non-Apple hardware if it supports it at all.
  - tinco 12y ago
    Non-Apple hardware does not differ in any meaningful way from Apple hardware. The performance of anything in OSX is perfectly tuned for any generic desktop computer. It also supports most hardware straight from the box.
    - deleted 12y ago
      [deleted]
    - KeytarHero 12y ago
      > any generic desktop computer Any generic desktop computer with the same hardware. It sounds like they're using Apple's image pipeline, which I imagine would be designed around the specific graphics hardware in the Mac Pro. Sure it could work on other hardware, but when you know exactly the hardware you're running on you can do a lot of low-level optimizations you couldn't otherwise do.
      - wtallis 12y ago
        Apple supports Intel, AMD, and NVidia GPUs. Their graphics pipeline has over the years supported substantially all of the graphics chips produced by those vendors in the past 10-15 years. Their current full feature set may only be supported on GPUs that admit an OpenCL implementation, but that's still every bit as broad as the generic desktop computer GPU market—about two microarchitecture per vendor. Apple's not getting any optimization benefits from a narrow pool of hardware, for GPUs or anything else. The only benefit they get along the lines of narrow hardware support is that they don't have to deal with all the various motherboard firmware bugs from dozens of OEMs.
  - smackfu 12y ago
    The GPUs aren't made by Apple, and I assume that is what this code is using (because otherwise it's a huge waste of money.)
  - wnevets 12y ago
    which hardware is "Apple"?
    - deleted 12y ago
      [deleted]
    - 7Z7 12y ago
      The small subset of third party hardware that they ship, and optimise for?
- MrBuddyCasino 12y ago
  Depends on where you live. At least in Germany (and I think the whole EU), EULAs are meaningless.
  - Someone 12y ago
    I can't find that for EULA's proper (would surprise me, too, as that would allow anyone to pirate any shrink-wrapped software), but EULAs cannot prohibit selling your license: that was upheld for software bought by download, too: http://curia.europa.eu/jcms/upload/docs/application/pdf/2012-07/cp120094en.pdf http://curia.europa.eu/jcms/upload/docs/application/pdf/2012...: "Where the copyright holder makes available to his customer a copy – tangible or intangible – and at the same time concludes, in return form payment of a fee, a licence agreement granting the customer the right to use that copy for an unlimited period, that rightholder sells the copy to the customer and thus exhausts his exclusive distribution right. Such a transaction involves a transfer of the right of ownership of the copy. Therefore, even if the licence agreement prohibits a further transfer, the rightholder can no longer oppose the resale of that copy" You can even buy the right to download future updates: "Therefore the new acquirer of the user licence, such as a customer of UsedSoft, may, as a lawful acquirer of the corrected and updated copy of the computer program concerned, download that copy from the copyright holder’s website."
    - Ded7xSEoPKYNsDd 12y ago
      Afaik, if the EULA is only shown after the sale has already taken place, it is basically meaningless around here. But IANAL and only relaying what I heard on the internet.
  - tinco 12y ago
    The lawyer I spoke to was Dutch. There is some unclarity about EULA's sold to consumers, I think the general idea of the consumer protection is that the consumer should have seen the EULA before they paid for the product. That said, it says on the box that the software is only for Apple hardware, and I think even only as an upgrade for an existing OSX install. If you're a company, then EULAs are definitely binding, no matter where you are.
- skuhn 12y ago
  This is correct, and a huge impetus for our use of Apple hardware. We simply cannot risk our business on saving some money at the expense of violating Apple's EULA.
  - Alupis 12y ago
    > We simply cannot risk our business on saving some money at the expense of violating Apple's EULA. I believe people are questioning that Apple hardware/software is a requirement of your business (and that it's not "some money", but a lot of money you'd be saving). It's difficult to fathom Apple hardware/software being a hard requirement to operate any business (as-in you can't operate without it). Both Windows and Linux have a plethora of image utilities, audio, etc... Sure, OSX might have some optimized image processing stuff, but couldn't the massive savings be used to scale wider with more generic hardware and still come out ahead?
    - skuhn 12y ago
      Not at the moment. The cost overhead to OS X is minimized by my approach to datacenter design and operation, and the benefit of OS X's image processing software stack is maximized by the imaging engineers on staff at imgix who are making the most of it. The math may not work out this way forever, and when it doesn't, we'll make a change.
  - notsony 12y ago
    Apple gave up on the server market. They don't care about number-crunching or scientific computing. To build a server-side based business around OS X doesn't make sense to me. Did you try writing your image processing code for Linux and CUDA / OpenCL? Is there anything specific about the OS X frameworks which means you can't develop a non-OS X solution?
    - deleted 12y ago
      [deleted]
- tajen 12y ago
  > you'll be in violation if you run it on a Thursday Indeed, there is a JS library opensourced by Microsoft to decode Excel files, named xlsx.js. In the license, it is written... that it cannot run on another OS than Windows. It means even though it's Javascript, the page hosting it cannot be viewed on a Mac or Linux. Long story short, Stuart Knightley created a clean room implementation named js-xlsx to do the same thing, without the lawyer string attached. https://github.com/stephen-hardy/xlsx.js/issues/8 https://github.com/stephen-hardy/xlsx.js/issues/8
  - stu_k 12y ago
    Thanks for the mention but I only created JSZip[0], the library that powers the zip/unzip of js-xlsx[1], but not js-xlsx itself! [0] https://github.com/Stuk/jszip https://github.com/Stuk/jszip [1] https://github.com/SheetJS/js-xlsx https://github.com/SheetJS/js-xlsx
- hornetblack 12y ago
  Although this depends on the country. If Apple says that you can't run your Mac Pro on Thursday, it would probably break some Australia Consumer Law. Although the "Only run on Apple Hardware" would probably be fine.
terhechte 12y ago
Sure, you can build such a hackintosh, but you're behind on OSX updates, you can't update automatically (it may break some of the custom hacks required), you probably can't use auto update. Also, some of the drivers may have additional bugs due to the ever-so-slight hardware change. In the end, your system may freeze more often, or display weird behaviour. All that apart from the EULA issue that was already mentioned. Maybe imgix already tested such hackintosh systems and realized that they're just not stable enough for continous high workloads.
- tinco 12y ago
  terhechte is right about this. To clarify: the reason you can't update automatically is although most hardware is supported just fine, there are many minor adjustments needed, mostly to text files but sometimes to binaries as well to make OSX recognize your hardware. Every time an OSX update comes out there's updates to drivers that overwrite your modified files. An example of the sort of hack I'm talking about would be a graphics driver that says it's for the NVidia model E532D. Your graphics card is an E532E. You looked on the internet, and you found out they are exactly identical except for branding, so you dive in the driver and simply flip a bit to make OSX recognize it.
elif 12y ago
I'm not trying to lure you down the flops per dollar rabbithole... but when i was researching the mac pro before purchasing, putting together a computer with GPUs providing the same flop performance was something like 80-90% of the cost of the mac pro. the D700's are really good. This is just an anecdote based on my memory, so take with a grain of salt, but the comparison is probably not as bad as you are imagining.
- skuhn 12y ago
  This is what I found as well, and allude to in other comments in the thread. Particularly when you look at GPGPUs like the NVidia Tesla, they are generally pretty terrible when priced per-gflop because they live in a niche market that wants to handle finite element analysis and supercomputing tasks. Image processing doesn't require double precision, so we don't need GPUs tuned for it, which means we can use Fire Pro's and similar workstation or server grade cards.
skuhn 12y ago
(I'm the datacenter manager at imgix, and I wrote the article) I've alluded to this elsewhere, but the math doesn't add up to your gut reaction. It's cheaper, but not by a significant enough margin relative to the engineering costs, to go with commodity servers and GPUs. Building things out of sheet metal is actually easier than migrating to Linux, for one big reason: we can pay someone else to do it, because it isn't part of our core competency. In fact, I'm pushing to open source the design of this chassis, in tandem with our design partner (Racklive). Not sure if it will happen, but I'd love to see it.
- x0054 12y ago
  It strikes me that OS X is a solid server platform. Linux is always a more flexible choice, but if OS X works for you, it works for you. That being said, is sticking cylindrical mac pros on their side into square racks really the best solution. There are 2 problems I see with this design: 1: You are placing the Mac Pros on their side, which may lead to premature bearing failure on the main cooling fan. Apple designed the cooling fan to be as silent as possible, which means that they optimized the bearing and the fan to work in vertical orientation. Bearings designed for thrust (vertical) orientation may not work so well if placed horizontally for a long time. 2: You are fitting triangular shaped computers, wrapped into round cases, into square shaped boxes, resulting in significant loss of space density. Considering that Apple is a huge company that owns huge data centers, combined with the fact that it would be simply stupid for a company who makes their own OS to run anything but that OS, and combined with the above mentioned problems with using Mac Pros as server "logs" (because you cant call them blades), I would assume that Apple has internally OSX servers designed in the traditional blade configuration. They may not sell or advertise them, but they MUST have them. Given that you guys are buying a ton of hardware, and are located nearby, and would be actively promoting running Apple hardware, wouldn't it be wise to at least approach Apple and see if they would be kind enough to sell you some of those blade form factor servers they simply must have. I may be completely wrong here, but apple did brag about how Swift is the new language thats so flexible that you can make a Mobile app in it, or a Desktop app, or even a full blown social network. If that's the case, they must have some plans for the server market? No? Any way, in the end it's a cool design, but I would seriously consider at least stacking the Mac Pros vertically to avoid fan issues. You can actually get a tighter form factor that way as well, unless space is not the issue. And if it's not, then hell, what's wrong with just placing a bunch of Pros on an Ikea shelf in a well air-conditioned room :)
ska 12y ago
re your hackintosh idea: I have no idea what the current state of play is, but I looked at this in some detail when they came out - and there was no way you could build a directly comparable machine without spending very similar money. The GPUs would just kill you, far more than any other components.
- skuhn 12y ago
  The GPUs in the Mac Pro are significantly more cost effective per gflop than the equivalent AMD Fire Pro server models.
  - ska 12y ago
    That matches what I found, looking at this a while ago. Not perfect for every application, of course.
- wooger 12y ago
  Apple fans often justify the prices with this argument, but in reality, there is no reason to buy a machine directly comparable in all aspects, just the ones I care about. And that can be had for a fraction of the cost. It's unlikely that this company needs all the hardware features of the Mac Pro - probably just the beefy GPUs. That's combined with the power density problems (and higher monthly costs) of this solution, compared to modern rack or blade servers, making it far worse value. Compare this also to John Siracusa's woes over buying a new Mac: He wants a graphics card powerful enough to game on, and remain useful for a number of years. He wants to be able to get a retina display. He's for now stuck with a 2007 Mac Pro as Apple don't sell a suitable machine.
  - ska 12y ago
    Well sure, if you don't need the machine, you don't need it and that' fine. When I was comparing hardware, you really couldn't buy comparable cost/flop GPUs for any significant savings (and you'd spend more on some similar builds), which was my point. No idea if that's still true. The idea that you could get the same thing for half the money just wasn't true. Your comment about John Siracusa's problem doesn't seem relevent to the OP, although it something to consider if you were buying a machine for home use.
grecy 12y ago
> there is no way that you can't build a hackintosh Have you ever personally run a Hackintosh, full-time for a prolonged period of time? It's anecdotal, but I can assure that once you're used to how OS X and the Apple hardware work together and never, ever, ever crash, using a Hackintosh is an exercise in frustration. I had one of the known-best Hackintosh configurations in existence, and it didn't hold a candle to the MBP I had prior to it in terms of "it just works". Sure, it was cheaper. Guess what I did when that Hackintosh needed replacing? I walked in and dropped the coin on genuine Apple hardware without a second thought. I have never regretted it, and I'll never go back.
- xaduha 12y ago
  > Have you ever personally run a Hackintosh, full-time for a prolonged period of time? I did. But now I'm running Yosemite under KVM, VT-d motherboard, dedicated videocard and USB3 hub. You can get to a point where "it just works".
  - grecy 12y ago
    > You can get to a point where "it just works". For months and months on end of heavy usage without a single restart or issue? EDIT: to expand a little - I was developing/compiling all day long on my ~2008 MBP with it plugged into an external monitor, network, mouse, kb. I'd close the lid and walk home with it, then watch movies, torrent, develop some more, surf etc. Close lid, and repeat for months on end. The only time I ever restarted was for OS updates, I never had a single app even crash in ~2 years of doing that. My hackintosh (and the windows 7&8 HP machines here at work) don't hold a candle to that.
    - xaduha 12y ago
      > OS X and the Apple hardware work together and never, ever, ever crash, using a Hackintosh is an exercise in frustration. Well all I can say that there are no crashes and no causes for frustration on my end. "It just works" for me is - I don't have to think about it, it does not get in a way. As a bonus: configuration of the VM can be put in a VCS, whole virtual disk can be snapshotted and reverted if needed.
s73v3r 12y ago
"but there is no way that you can't build a hackintosh 1U with dual cpus and multiple GPUs and not come out big money ahead." Licensing costs, and legal costs when you get sued for violating the license?
k2enemy 12y ago
I had the exact same thoughts. On one hand I love the idea of racking mac pros and having drawers full of Wera tools to work with, but on the other hand, it just seems like the kind of silly expenditures that come with too much funding.
- skuhn 12y ago
  I've answered the concern about Wera hand tools before, but to reiterate my position: a $6 screwdriver doesn't matter if it helps you do your job. Not having a screwdriver when you need it in a pinch is penny wise and pound foolish. At best you're now out 30 minutes while you drive to Home Depot, potentially during some sort of catastrophe. At worst maybe you simply cannot do the task that you need to do, because it's 2am and you're in Frankfurt. I've worked in a lot of datacenters that didn't stock basic tools to perform tasks, and frankly it sucked. I keep a log of all of the purchases I made for the current datacenter build. Non-server / non-structural expenses account for less than $3000, which is less than the cost of a single server. This includes storage bins, carts, shelves, workbenches, chairs, supplies and tools.
dzhiurgis 12y ago
Am I the only to think that sheet metal is quite cheap to bend?
- skuhn 12y ago
  Indeed, it is. Servers are expensive; everything built for a datacenter is expensive. It's very easy to make a passive chassis for far less than conventional datacenter gear prices. A lot of cost estimates have been thrown around (here and elsewhere). The highest that I've seen is $4000 per unit. That is simply absurd. The initial run of prototypes was far less per unit, and this was a small batch made to iron out the kinks. Economies of scale and design tweaks will drive this down even further. The chassis design is actually quite elegant from a manufacturing standpoint. That's something that I hope will be made evident by follow-up posts that delve into more technical detail.

by rsync 12y ago

tinco 12y ago

Obviously, but running OSX on non-Apple hardware is a violation of its EULA. I have contacted a lawyer for this (I wanted to run Hackintosh in the office), the language is very clear. The author of the software has the full power to license its use to you with any restrictions they find necessary no matter how ridiculous. If Apple only sells you the license if you promise not to run it on a thursday, you'll be in violation of their terms if you run it on a thursday.

nulltype 12y ago
Even if that were not the case, I doubt the performance of the OS X image pipeline is tuned for non-Apple hardware if it supports it at all.
- tinco 12y ago
  Non-Apple hardware does not differ in any meaningful way from Apple hardware. The performance of anything in OSX is perfectly tuned for any generic desktop computer. It also supports most hardware straight from the box.
  - deleted 12y ago
    [deleted]
  - KeytarHero 12y ago
    > any generic desktop computer Any generic desktop computer with the same hardware. It sounds like they're using Apple's image pipeline, which I imagine would be designed around the specific graphics hardware in the Mac Pro. Sure it could work on other hardware, but when you know exactly the hardware you're running on you can do a lot of low-level optimizations you couldn't otherwise do.
    - wtallis 12y ago
      Apple supports Intel, AMD, and NVidia GPUs. Their graphics pipeline has over the years supported substantially all of the graphics chips produced by those vendors in the past 10-15 years. Their current full feature set may only be supported on GPUs that admit an OpenCL implementation, but that's still every bit as broad as the generic desktop computer GPU market—about two microarchitecture per vendor. Apple's not getting any optimization benefits from a narrow pool of hardware, for GPUs or anything else. The only benefit they get along the lines of narrow hardware support is that they don't have to deal with all the various motherboard firmware bugs from dozens of OEMs.
- smackfu 12y ago
  The GPUs aren't made by Apple, and I assume that is what this code is using (because otherwise it's a huge waste of money.)
- wnevets 12y ago
  which hardware is "Apple"?
  - deleted 12y ago
    [deleted]
  - 7Z7 12y ago
    The small subset of third party hardware that they ship, and optimise for?
MrBuddyCasino 12y ago
Depends on where you live. At least in Germany (and I think the whole EU), EULAs are meaningless.
- Someone 12y ago
  I can't find that for EULA's proper (would surprise me, too, as that would allow anyone to pirate any shrink-wrapped software), but EULAs cannot prohibit selling your license: that was upheld for software bought by download, too: http://curia.europa.eu/jcms/upload/docs/application/pdf/2012-07/cp120094en.pdf http://curia.europa.eu/jcms/upload/docs/application/pdf/2012...: "Where the copyright holder makes available to his customer a copy – tangible or intangible – and at the same time concludes, in return form payment of a fee, a licence agreement granting the customer the right to use that copy for an unlimited period, that rightholder sells the copy to the customer and thus exhausts his exclusive distribution right. Such a transaction involves a transfer of the right of ownership of the copy. Therefore, even if the licence agreement prohibits a further transfer, the rightholder can no longer oppose the resale of that copy" You can even buy the right to download future updates: "Therefore the new acquirer of the user licence, such as a customer of UsedSoft, may, as a lawful acquirer of the corrected and updated copy of the computer program concerned, download that copy from the copyright holder’s website."
  - Ded7xSEoPKYNsDd 12y ago
    Afaik, if the EULA is only shown after the sale has already taken place, it is basically meaningless around here. But IANAL and only relaying what I heard on the internet.
- tinco 12y ago
  The lawyer I spoke to was Dutch. There is some unclarity about EULA's sold to consumers, I think the general idea of the consumer protection is that the consumer should have seen the EULA before they paid for the product. That said, it says on the box that the software is only for Apple hardware, and I think even only as an upgrade for an existing OSX install. If you're a company, then EULAs are definitely binding, no matter where you are.
skuhn 12y ago
This is correct, and a huge impetus for our use of Apple hardware. We simply cannot risk our business on saving some money at the expense of violating Apple's EULA.
- Alupis 12y ago
  > We simply cannot risk our business on saving some money at the expense of violating Apple's EULA. I believe people are questioning that Apple hardware/software is a requirement of your business (and that it's not "some money", but a lot of money you'd be saving). It's difficult to fathom Apple hardware/software being a hard requirement to operate any business (as-in you can't operate without it). Both Windows and Linux have a plethora of image utilities, audio, etc... Sure, OSX might have some optimized image processing stuff, but couldn't the massive savings be used to scale wider with more generic hardware and still come out ahead?
  - skuhn 12y ago
    Not at the moment. The cost overhead to OS X is minimized by my approach to datacenter design and operation, and the benefit of OS X's image processing software stack is maximized by the imaging engineers on staff at imgix who are making the most of it. The math may not work out this way forever, and when it doesn't, we'll make a change.
- notsony 12y ago
  Apple gave up on the server market. They don't care about number-crunching or scientific computing. To build a server-side based business around OS X doesn't make sense to me. Did you try writing your image processing code for Linux and CUDA / OpenCL? Is there anything specific about the OS X frameworks which means you can't develop a non-OS X solution?
  - deleted 12y ago
    [deleted]
tajen 12y ago
> you'll be in violation if you run it on a Thursday Indeed, there is a JS library opensourced by Microsoft to decode Excel files, named xlsx.js. In the license, it is written... that it cannot run on another OS than Windows. It means even though it's Javascript, the page hosting it cannot be viewed on a Mac or Linux. Long story short, Stuart Knightley created a clean room implementation named js-xlsx to do the same thing, without the lawyer string attached. https://github.com/stephen-hardy/xlsx.js/issues/8 https://github.com/stephen-hardy/xlsx.js/issues/8
- stu_k 12y ago
  Thanks for the mention but I only created JSZip[0], the library that powers the zip/unzip of js-xlsx[1], but not js-xlsx itself! [0] https://github.com/Stuk/jszip https://github.com/Stuk/jszip [1] https://github.com/SheetJS/js-xlsx https://github.com/SheetJS/js-xlsx
hornetblack 12y ago
Although this depends on the country. If Apple says that you can't run your Mac Pro on Thursday, it would probably break some Australia Consumer Law. Although the "Only run on Apple Hardware" would probably be fine.

terhechte 12y ago

Sure, you can build such a hackintosh, but you're behind on OSX updates, you can't update automatically (it may break some of the custom hacks required), you probably can't use auto update. Also, some of the drivers may have additional bugs due to the ever-so-slight hardware change. In the end, your system may freeze more often, or display weird behaviour. All that apart from the EULA issue that was already mentioned. Maybe imgix already tested such hackintosh systems and realized that they're just not stable enough for continous high workloads.

tinco 12y ago
terhechte is right about this. To clarify: the reason you can't update automatically is although most hardware is supported just fine, there are many minor adjustments needed, mostly to text files but sometimes to binaries as well to make OSX recognize your hardware. Every time an OSX update comes out there's updates to drivers that overwrite your modified files. An example of the sort of hack I'm talking about would be a graphics driver that says it's for the NVidia model E532D. Your graphics card is an E532E. You looked on the internet, and you found out they are exactly identical except for branding, so you dive in the driver and simply flip a bit to make OSX recognize it.

elif 12y ago

I'm not trying to lure you down the flops per dollar rabbithole... but when i was researching the mac pro before purchasing, putting together a computer with GPUs providing the same flop performance was something like 80-90% of the cost of the mac pro. the D700's are really good. This is just an anecdote based on my memory, so take with a grain of salt, but the comparison is probably not as bad as you are imagining.

skuhn 12y ago
This is what I found as well, and allude to in other comments in the thread. Particularly when you look at GPGPUs like the NVidia Tesla, they are generally pretty terrible when priced per-gflop because they live in a niche market that wants to handle finite element analysis and supercomputing tasks. Image processing doesn't require double precision, so we don't need GPUs tuned for it, which means we can use Fire Pro's and similar workstation or server grade cards.

skuhn 12y ago

(I'm the datacenter manager at imgix, and I wrote the article) I've alluded to this elsewhere, but the math doesn't add up to your gut reaction. It's cheaper, but not by a significant enough margin relative to the engineering costs, to go with commodity servers and GPUs. Building things out of sheet metal is actually easier than migrating to Linux, for one big reason: we can pay someone else to do it, because it isn't part of our core competency. In fact, I'm pushing to open source the design of this chassis, in tandem with our design partner (Racklive). Not sure if it will happen, but I'd love to see it.

x0054 12y ago
It strikes me that OS X is a solid server platform. Linux is always a more flexible choice, but if OS X works for you, it works for you. That being said, is sticking cylindrical mac pros on their side into square racks really the best solution. There are 2 problems I see with this design: 1: You are placing the Mac Pros on their side, which may lead to premature bearing failure on the main cooling fan. Apple designed the cooling fan to be as silent as possible, which means that they optimized the bearing and the fan to work in vertical orientation. Bearings designed for thrust (vertical) orientation may not work so well if placed horizontally for a long time. 2: You are fitting triangular shaped computers, wrapped into round cases, into square shaped boxes, resulting in significant loss of space density. Considering that Apple is a huge company that owns huge data centers, combined with the fact that it would be simply stupid for a company who makes their own OS to run anything but that OS, and combined with the above mentioned problems with using Mac Pros as server "logs" (because you cant call them blades), I would assume that Apple has internally OSX servers designed in the traditional blade configuration. They may not sell or advertise them, but they MUST have them. Given that you guys are buying a ton of hardware, and are located nearby, and would be actively promoting running Apple hardware, wouldn't it be wise to at least approach Apple and see if they would be kind enough to sell you some of those blade form factor servers they simply must have. I may be completely wrong here, but apple did brag about how Swift is the new language thats so flexible that you can make a Mobile app in it, or a Desktop app, or even a full blown social network. If that's the case, they must have some plans for the server market? No? Any way, in the end it's a cool design, but I would seriously consider at least stacking the Mac Pros vertically to avoid fan issues. You can actually get a tighter form factor that way as well, unless space is not the issue. And if it's not, then hell, what's wrong with just placing a bunch of Pros on an Ikea shelf in a well air-conditioned room :)

ska 12y ago

re your hackintosh idea: I have no idea what the current state of play is, but I looked at this in some detail when they came out - and there was no way you could build a directly comparable machine without spending very similar money. The GPUs would just kill you, far more than any other components.

skuhn 12y ago
The GPUs in the Mac Pro are significantly more cost effective per gflop than the equivalent AMD Fire Pro server models.
- ska 12y ago
  That matches what I found, looking at this a while ago. Not perfect for every application, of course.
wooger 12y ago
Apple fans often justify the prices with this argument, but in reality, there is no reason to buy a machine directly comparable in all aspects, just the ones I care about. And that can be had for a fraction of the cost. It's unlikely that this company needs all the hardware features of the Mac Pro - probably just the beefy GPUs. That's combined with the power density problems (and higher monthly costs) of this solution, compared to modern rack or blade servers, making it far worse value. Compare this also to John Siracusa's woes over buying a new Mac: He wants a graphics card powerful enough to game on, and remain useful for a number of years. He wants to be able to get a retina display. He's for now stuck with a 2007 Mac Pro as Apple don't sell a suitable machine.
- ska 12y ago
  Well sure, if you don't need the machine, you don't need it and that' fine. When I was comparing hardware, you really couldn't buy comparable cost/flop GPUs for any significant savings (and you'd spend more on some similar builds), which was my point. No idea if that's still true. The idea that you could get the same thing for half the money just wasn't true. Your comment about John Siracusa's problem doesn't seem relevent to the OP, although it something to consider if you were buying a machine for home use.

grecy 12y ago

> there is no way that you can't build a hackintosh Have you ever personally run a Hackintosh, full-time for a prolonged period of time? It's anecdotal, but I can assure that once you're used to how OS X and the Apple hardware work together and never, ever, ever crash, using a Hackintosh is an exercise in frustration. I had one of the known-best Hackintosh configurations in existence, and it didn't hold a candle to the MBP I had prior to it in terms of "it just works". Sure, it was cheaper. Guess what I did when that Hackintosh needed replacing? I walked in and dropped the coin on genuine Apple hardware without a second thought. I have never regretted it, and I'll never go back.

xaduha 12y ago
> Have you ever personally run a Hackintosh, full-time for a prolonged period of time? I did. But now I'm running Yosemite under KVM, VT-d motherboard, dedicated videocard and USB3 hub. You can get to a point where "it just works".
- grecy 12y ago
  > You can get to a point where "it just works". For months and months on end of heavy usage without a single restart or issue? EDIT: to expand a little - I was developing/compiling all day long on my ~2008 MBP with it plugged into an external monitor, network, mouse, kb. I'd close the lid and walk home with it, then watch movies, torrent, develop some more, surf etc. Close lid, and repeat for months on end. The only time I ever restarted was for OS updates, I never had a single app even crash in ~2 years of doing that. My hackintosh (and the windows 7&8 HP machines here at work) don't hold a candle to that.
  - xaduha 12y ago
    > OS X and the Apple hardware work together and never, ever, ever crash, using a Hackintosh is an exercise in frustration. Well all I can say that there are no crashes and no causes for frustration on my end. "It just works" for me is - I don't have to think about it, it does not get in a way. As a bonus: configuration of the VM can be put in a VCS, whole virtual disk can be snapshotted and reverted if needed.

s73v3r 12y ago

"but there is no way that you can't build a hackintosh 1U with dual cpus and multiple GPUs and not come out big money ahead." Licensing costs, and legal costs when you get sued for violating the license?

k2enemy 12y ago

I had the exact same thoughts. On one hand I love the idea of racking mac pros and having drawers full of Wera tools to work with, but on the other hand, it just seems like the kind of silly expenditures that come with too much funding.

skuhn 12y ago
I've answered the concern about Wera hand tools before, but to reiterate my position: a $6 screwdriver doesn't matter if it helps you do your job. Not having a screwdriver when you need it in a pinch is penny wise and pound foolish. At best you're now out 30 minutes while you drive to Home Depot, potentially during some sort of catastrophe. At worst maybe you simply cannot do the task that you need to do, because it's 2am and you're in Frankfurt. I've worked in a lot of datacenters that didn't stock basic tools to perform tasks, and frankly it sucked. I keep a log of all of the purchases I made for the current datacenter build. Non-server / non-structural expenses account for less than $3000, which is less than the cost of a single server. This includes storage bins, carts, shelves, workbenches, chairs, supplies and tools.

dzhiurgis 12y ago

Am I the only to think that sheet metal is quite cheap to bend?

skuhn 12y ago
Indeed, it is. Servers are expensive; everything built for a datacenter is expensive. It's very easy to make a passive chassis for far less than conventional datacenter gear prices. A lot of cost estimates have been thrown around (here and elsewhere). The highest that I've seen is $4000 per unit. That is simply absurd. The initial run of prototypes was far less per unit, and this was a small batch made to iron out the kinks. Economies of scale and design tweaks will drive this down even further. The chassis design is actually quite elegant from a manufacturing standpoint. That's something that I hope will be made evident by follow-up posts that delve into more technical detail.