7 ms·

OS X 10.8 vs. Ubuntu On Apple Hardware, Benchmarked

68 points by quicklycode 14y ago

nicholassmith 14y ago
Like the article title, there is no clear winner but it does look like certain operations have been optimised on each platform in response to, I'd guess, expected user needs. Which isn't surprising, but it is interesting seeing what specifically each OS designer thinks is the expected use that does require time spent optimising. Also, quite an in-depth piece of coverage, a lot of benchmarking doesn't take real world into account and just runs 3DMarks or similar.
- stephencanon 14y ago
  If there are cases important to you as a user, file bug reports about the performance. OS developers can (and will) guess about what scenarios are most performance-critical, but feedback from developers and users ("doing X is important to me, and is slow on your system" / "why is Y slower on your system than on your competitors" / etc) is enormously valuable and quite welcome. The squeaky wheel gets the grease. Speaking as an OS developer, poor performance absolutely is a bug, and your bug report will be taken seriously.
  - nicholassmith 14y ago
    I don't think it's specifically poor performance, but places that have attracted more optimisation attention. An operating system is a big old chunk of work, and not every element can be honed to a fine edge. It's good that OS developers do actually respond and focus on areas users have flagged up as well, builds better systems.
mcormier 14y ago
15 pages for this article seems quite excessive
- jpk 14y ago
  Mildly OT, but... Agreed. Unfortunately, nearly every hardware review like this does that. I almost invariably skip to the last page to read the conclusion first, and only poke through the middle pages if I'm curious why the reviewer drew the conclusion they did. This one is particularly bad, though, because the pages aren't labeled, and there's no table of contents. Which makes "poke through the middle pages" for the purpose of finding a specific test related to the conclusion hard. We all know it's to bump up the number of page views and therefore ad revenue, but I wonder: For a review like this, how many ad impressions do you need to break even on the reviewer's time, hardware cost (if any), and other overhead? Is ridiculous pagination a necessary evil for these sorts of reviews to exist? Or is it solely to pad margins?
- lathamcity 14y ago
  The text loads quickly enough, but the pictures really kill it. I clicked page 2 and got one line of text and a bunch of white, and decided to make that the extent of my user experience.
davidjgraph 14y ago
What a waste of time, OS X has a usable, stable graphical interface, Ubuntu doesn't ( a lot of the lack of usability comes from the lack of quality of third-party apps ). The only use for Ubuntu is as a server, which means testing things for speed that are server related. And only on a Mini. And yes, I've tried both Ubuntu 12.04 and OS X 10.8 for several days as my only work-based machine to road test both properly.
- shell0x 14y ago
  I will migrate my Macbook to Ubuntu soon :)The latest Ubuntu is ok with a few modifications.
- mhd 14y ago
  If only it were stable, given the skeuomorphic crimes of recent releases and the whole full-screen/unified-expose&virtual-desktop debacle. Not that Ubuntu/Unity fares better in that department, but right now, all the major players aren't really doing a swell job with their desktops.
- calinet6 14y ago
  You're going to get flack for that opinion, but you have a point. The truth is, Unity sucks. A lot. Even after you get used to it. It was a poor decision by a set of developers out of touch with reality or their user-base, and it's only beginning to recover and move in the right direction slowly. Everyone I know is migrating to Mint and Xfce/KDE4/Cinnamon (Gnome 3 fork), and with good reason. OSX is still way ahead of the curve, but Linux (not necessarily only Ubuntu) still has a chance. The idea of Linux is still great: we're hackers, so if you feel there's a problem, help fix it. Not easy, but the fact that you can is pretty cool.
  - davidjgraph 14y ago
    Oh yeah, the Ubuntu fanboys will be busy. But seriously, I need my main work machine to just work (TM). I need audio to work without a 4 second lag on skype (I know that's skype's fault). I need Eclipse to default to fonts that I don't notice, rather than spending 20 minutes tweaking. I need Teamviewer to work correctly when I have 2 monitors on Ubuntu. I know that's off-topic, but I really think everyone/thing Ubuntu related should be focusing on the fact that the desktop doesn't cut it, the OA is off-topic if you like. When you write software, you write it first, properly, then worry about optimization when it's functional (with a few exceptions). If it doesn't do its job, who cares that it can unzip twice as fast.
frownie 14y ago
No clear winner... I'd say there is a winner : Linux. Isn't it interesting that a software developped by a team of hacker (I'd say a chaotic, huge and ever growing) can make a monster like Ubuntu and get performances that don't pale against another OS which is designed by a very organised, super funded, super focused team of carefully selected programmers... Ok, I assume a rather romatic view of linux development, but I'd bet I'm not that far from reality :-)
- SIULHT 14y ago
  Your argument is that Linux is the winner because despite all odds, it doesn't suck shit as much as you'd expect it to?
  - jcbrand 14y ago
    How does comparing very favourably against OSX imply that it merely "sucks shit" less?
    - mitchty 14y ago
      Probably referring to the All Software Sucks camp of philosophy.
  - bgilroy26 14y ago
    It's a pretty solid argument if the premises are correct: The article benchmarks the performance of Ubuntu and OSX, and finds them to be in a dead heat. Ubuntu was developed at a lower investment on Canonical's part (OR with a larger team of less skilled developers, I can't tell which is more important to frownie). Therefore Ubuntu represents a greater achievement relative to the resources invested than OSX does.
    - orlandu63 14y ago
      Let's not forget that GNU/Linux is intended to run on any damn machine you throw at it, while the latest version of OSX is only intended to run on Apple hardware that is no more than 3 years old. It's amazing that Ubuntu is even competitive with OSX given these circumstances.
stephencanon 14y ago
Some of these tests are clearly testing the compiler more than the OS. Looking over the scimark tests, for example, the FFT, LU, and SOR numbers are (at least) an order of magnitude too low to be using optimized libraries that are part of the system. For these benchmarks, we're seeing the compiler quality of compiler codegen on a reference C implementation (OS tasks like malloc/free efficiency may have a small effect, but it should be essentially negligible). No one interested in performance would actually use that code on OS X or Ubuntu; you would use a tuned library.
indiecore 14y ago
I'm kind of wondering why you would use a Mac without using OSX. The only reason to have that computer is to run the OS that ONLY runs on that computer. Otherwise you can get a better computer for less money that is more flexible.
- antirez 14y ago
  Apple laptops are (IMHO the best, but let's say among) the best in the market. So it is no surprise that Linux users may want one but without osx.
  - whalesalad 14y ago
    Yep. Apple notebook notebook hardware cannot be beat. Zero contest from anyone else.
    - CoffeeDregs 14y ago
      Apple [now] uses the same components as everyone else, so I'd be surprised if Apple had significantly different performance. Their construction/build-quality is clearly exceptional, though I'll be curious to see if my [incoming] Thinkpad X1 Carbon comes close.
      - whalesalad 14y ago
        I was primarily pointing out the construction and build quality. An Apple chassis cannot be beat =)
  - gbrindisi 14y ago
    Do you run Linux on a macbook? If yes how is it performing?
    - chimeracoder 14y ago
      In case you're not being rhetorical/sarcastic: Linux (or any other OS) runs terribly on Apple hardware, because the hardware is intentionally designed to run OS X and only OS X. If you try to run, say, Ubuntu on a Macbook, you'll notice that you won't be able to do basic things like adjust the fan speed (it'll be turned up to the max the entire time) and the battery life will thus be horrible, not to mention that the computer will also be insanely hot. This is because of the way an alternate OS is installed on Apple hardware - it's actually running on top of a low-level layer that prevents the OS from having proper access to the hardware. The only time I've ever heard of people running into driver issues these days on Linux for basic work is on Apple hardware, and that's the reason. So even if you like Apple hardware, you still wouldn't be getting your money's worth running any other OS on it. If you want to run Linux, go for almost any other laptop on the market - you'll get better priced hardware, and the performance will be better.
darkstalker 14y ago
The ioquake3-based Urban Terror game was also much faster with Apple's graphics drivers for the Intel HD Graphics than under Ubuntu Linux with Mesa. They're comparing apples to oranges. Mesa are like default graphic drivers, with very little or no hardware acceleration.
- zobzu 14y ago
  that's incorrect. mesa is the 3d graphic library on top of the native intel hd graphic drivers. the comparison actually make sense. you're not going to intel's osx drivers running in linux and you're not going to get mesa and intel's linux drivers running in osx, you realize that right? right? Oh right it's just trendy to hit on Phoronix for no reason. Well, they're testing the same games in the different environments to decide which environment is the fastest. That's very much apple to apple. I'd like to hear your test that's so much more accurate between Linux and Apple gaming with the same hardware. PLEASE ENLIGHTEN US.
  - numbsafari 14y ago
    You seem to be really angry about this. I mean, shockingly so. Your comment just really jumped out at me and caught me by surprise. So much so that I felt like it might be good for someone to point it out to you. I'd like to prescribe an afternoon of UrT. Take about 45 minutes to frag the hell out of a Steve Jobs avatar and post back in the morning. :)
    - tedunangst 14y ago
      Anger aside, he's right. mesa is used even if the intel driver is used too. mesa includes a software implementation, but it's also the gateway to the hardware drivers.
      - ajross 14y ago
        Better stated: the Intel graphics driver is part of the Mesa/DRI2 framework. Mesa's core provides the OpenGL front end (API implementation, shader compilers, high level optimizers). Intel provides a low level component that takes that shader stream and turns it into programs and commands for the hardware, which is managed by a DRI driver in the kernel.
cydonian_monk 14y ago
I haven't yet benchmarked any flavour of Linux running natively on my current-gen Ivy Bridge 13" MacBook Pro, but I have compared Windows 7 to both System 10.7 and 10.8. Consistently Win7 out performs OS-X on "raw" math-heavy benchmarks. I've only uploaded my results form Geekbench-2, but the other benchmarking utils show similar numbers: Geekbench-2 Scores and [Results]: OS-X 10.8: 8743 [1] Windows 7: 10092 [2] Getting a Linux install to run via Bootcamp on this MBP is one of my next projects (most likely Gentoo), and I'm curious to see how well its performance stacks up against the other two. That is taking some time as... well, some things just don't work yet. (Thankfully I don't have to deal with the hybrid GPU setup that's in the Retina MBP.) 1: http://browser.primatelabs.com/geekbench2/902437 2: http://browser.primatelabs.com/geekbench2/810264
donniezazen 14y ago
Power consumption has been such a big problem on Linux. Their was PCIe-ASPM bug before and now new power bugs. Their is no end to it.
unicornporn 14y ago
I was just about to try Xubuntu on my MBP, but after seeing those power consumption stats I don't think I can justify it. Sad but true: Linux worked better in the old school Desktop era when batteries, power consumption, sleep/hibernation and wireless networks weren't there to mess things up.
malkia 14y ago
Things like TLB might also affect your performance, at the cost of memory fragmentation. For example bigger page size (1, 2 or 4MB) would need only two 2 page directory/table lookups, while 4096 pages would need 3. It's just one of the examples. For example 4096bytes vs 4MB page size: http://en.wikipedia.org/wiki/File:X86_Paging_4K.svg http://en.wikipedia.org/wiki/File:X86_Paging_4K.svg http://en.wikipedia.org/wiki/File:X86_Paging_4M.svg http://en.wikipedia.org/wiki/File:X86_Paging_4M.svg
navpatel 14y ago
All these benchmarks are great and all... but in terms of using Linux on a Macbook Pro, the shitty support for the multitouch trackpad/touchpad alone makes linux barely usable. Unless this has changed recently, in which case someone please correct me
Someone 14y ago
Lots of data, but preciously little information. For example: - Why is python so much slower on Mac OS X? if I had to guess, I would say it is more due to a version difference than due to a compiler change. It might also be some particularly slow library code on Mac OS X, though. - What is causing the sometimes huge differences in CPU-only tasks such as the matrix computations? Compiler, system overhead due to inefficient code, system overhead because the system also performs other work (for example, did the OS index files during some of the runs?), or maybe architectural choices that lead to system overhead (for example, a scheduler might switch tasks more often in order to keep UI response low)? With such info, this would have been a great article. As it stands now, all I conclude from it is that OpenGL is slower under Ubuntu, and that Linux filesystems are faster (but there, it would help to know why. HFS probably is a large part of the explanation, but it also might be that Mac OS ports of various tools do slightly different things, e.g. By flushing to disk more aggressively.

68 points by quicklycode 14y ago

Like the article title, there is no clear winner but it does look like certain operations have been optimised on each platform in response to, I'd guess, expected user needs. Which isn't surprising, but it is interesting seeing what specifically each OS designer thinks is the expected use that does require time spent optimising. Also, quite an in-depth piece of coverage, a lot of benchmarking doesn't take real world into account and just runs 3DMarks or similar.

stephencanon 14y ago
If there are cases important to you as a user, file bug reports about the performance. OS developers can (and will) guess about what scenarios are most performance-critical, but feedback from developers and users ("doing X is important to me, and is slow on your system" / "why is Y slower on your system than on your competitors" / etc) is enormously valuable and quite welcome. The squeaky wheel gets the grease. Speaking as an OS developer, poor performance absolutely is a bug, and your bug report will be taken seriously.
- nicholassmith 14y ago
  I don't think it's specifically poor performance, but places that have attracted more optimisation attention. An operating system is a big old chunk of work, and not every element can be honed to a fine edge. It's good that OS developers do actually respond and focus on areas users have flagged up as well, builds better systems.

mcormier 14y ago

15 pages for this article seems quite excessive

jpk 14y ago
Mildly OT, but... Agreed. Unfortunately, nearly every hardware review like this does that. I almost invariably skip to the last page to read the conclusion first, and only poke through the middle pages if I'm curious why the reviewer drew the conclusion they did. This one is particularly bad, though, because the pages aren't labeled, and there's no table of contents. Which makes "poke through the middle pages" for the purpose of finding a specific test related to the conclusion hard. We all know it's to bump up the number of page views and therefore ad revenue, but I wonder: For a review like this, how many ad impressions do you need to break even on the reviewer's time, hardware cost (if any), and other overhead? Is ridiculous pagination a necessary evil for these sorts of reviews to exist? Or is it solely to pad margins?
lathamcity 14y ago
The text loads quickly enough, but the pictures really kill it. I clicked page 2 and got one line of text and a bunch of white, and decided to make that the extent of my user experience.

davidjgraph 14y ago

What a waste of time, OS X has a usable, stable graphical interface, Ubuntu doesn't ( a lot of the lack of usability comes from the lack of quality of third-party apps ). The only use for Ubuntu is as a server, which means testing things for speed that are server related. And only on a Mini. And yes, I've tried both Ubuntu 12.04 and OS X 10.8 for several days as my only work-based machine to road test both properly.

shell0x 14y ago
I will migrate my Macbook to Ubuntu soon :)The latest Ubuntu is ok with a few modifications.
mhd 14y ago
If only it were stable, given the skeuomorphic crimes of recent releases and the whole full-screen/unified-expose&virtual-desktop debacle. Not that Ubuntu/Unity fares better in that department, but right now, all the major players aren't really doing a swell job with their desktops.
calinet6 14y ago
You're going to get flack for that opinion, but you have a point. The truth is, Unity sucks. A lot. Even after you get used to it. It was a poor decision by a set of developers out of touch with reality or their user-base, and it's only beginning to recover and move in the right direction slowly. Everyone I know is migrating to Mint and Xfce/KDE4/Cinnamon (Gnome 3 fork), and with good reason. OSX is still way ahead of the curve, but Linux (not necessarily only Ubuntu) still has a chance. The idea of Linux is still great: we're hackers, so if you feel there's a problem, help fix it. Not easy, but the fact that you can is pretty cool.
- davidjgraph 14y ago
  Oh yeah, the Ubuntu fanboys will be busy. But seriously, I need my main work machine to just work (TM). I need audio to work without a 4 second lag on skype (I know that's skype's fault). I need Eclipse to default to fonts that I don't notice, rather than spending 20 minutes tweaking. I need Teamviewer to work correctly when I have 2 monitors on Ubuntu. I know that's off-topic, but I really think everyone/thing Ubuntu related should be focusing on the fact that the desktop doesn't cut it, the OA is off-topic if you like. When you write software, you write it first, properly, then worry about optimization when it's functional (with a few exceptions). If it doesn't do its job, who cares that it can unzip twice as fast.

frownie 14y ago

No clear winner... I'd say there is a winner : Linux. Isn't it interesting that a software developped by a team of hacker (I'd say a chaotic, huge and ever growing) can make a monster like Ubuntu and get performances that don't pale against another OS which is designed by a very organised, super funded, super focused team of carefully selected programmers... Ok, I assume a rather romatic view of linux development, but I'd bet I'm not that far from reality :-)

SIULHT 14y ago
Your argument is that Linux is the winner because despite all odds, it doesn't suck shit as much as you'd expect it to?
- jcbrand 14y ago
  How does comparing very favourably against OSX imply that it merely "sucks shit" less?
  - mitchty 14y ago
    Probably referring to the All Software Sucks camp of philosophy.
- bgilroy26 14y ago
  It's a pretty solid argument if the premises are correct: The article benchmarks the performance of Ubuntu and OSX, and finds them to be in a dead heat. Ubuntu was developed at a lower investment on Canonical's part (OR with a larger team of less skilled developers, I can't tell which is more important to frownie). Therefore Ubuntu represents a greater achievement relative to the resources invested than OSX does.
  - orlandu63 14y ago
    Let's not forget that GNU/Linux is intended to run on any damn machine you throw at it, while the latest version of OSX is only intended to run on Apple hardware that is no more than 3 years old. It's amazing that Ubuntu is even competitive with OSX given these circumstances.

stephencanon 14y ago

Some of these tests are clearly testing the compiler more than the OS. Looking over the scimark tests, for example, the FFT, LU, and SOR numbers are (at least) an order of magnitude too low to be using optimized libraries that are part of the system. For these benchmarks, we're seeing the compiler quality of compiler codegen on a reference C implementation (OS tasks like malloc/free efficiency may have a small effect, but it should be essentially negligible). No one interested in performance would actually use that code on OS X or Ubuntu; you would use a tuned library.

indiecore 14y ago

I'm kind of wondering why you would use a Mac without using OSX. The only reason to have that computer is to run the OS that ONLY runs on that computer. Otherwise you can get a better computer for less money that is more flexible.

antirez 14y ago
Apple laptops are (IMHO the best, but let's say among) the best in the market. So it is no surprise that Linux users may want one but without osx.
- whalesalad 14y ago
  Yep. Apple notebook notebook hardware cannot be beat. Zero contest from anyone else.
  - CoffeeDregs 14y ago
    Apple [now] uses the same components as everyone else, so I'd be surprised if Apple had significantly different performance. Their construction/build-quality is clearly exceptional, though I'll be curious to see if my [incoming] Thinkpad X1 Carbon comes close.
    - whalesalad 14y ago
      I was primarily pointing out the construction and build quality. An Apple chassis cannot be beat =)
- gbrindisi 14y ago
  Do you run Linux on a macbook? If yes how is it performing?
  - chimeracoder 14y ago
    In case you're not being rhetorical/sarcastic: Linux (or any other OS) runs terribly on Apple hardware, because the hardware is intentionally designed to run OS X and only OS X. If you try to run, say, Ubuntu on a Macbook, you'll notice that you won't be able to do basic things like adjust the fan speed (it'll be turned up to the max the entire time) and the battery life will thus be horrible, not to mention that the computer will also be insanely hot. This is because of the way an alternate OS is installed on Apple hardware - it's actually running on top of a low-level layer that prevents the OS from having proper access to the hardware. The only time I've ever heard of people running into driver issues these days on Linux for basic work is on Apple hardware, and that's the reason. So even if you like Apple hardware, you still wouldn't be getting your money's worth running any other OS on it. If you want to run Linux, go for almost any other laptop on the market - you'll get better priced hardware, and the performance will be better.

darkstalker 14y ago

The ioquake3-based Urban Terror game was also much faster with Apple's graphics drivers for the Intel HD Graphics than under Ubuntu Linux with Mesa. They're comparing apples to oranges. Mesa are like default graphic drivers, with very little or no hardware acceleration.

zobzu 14y ago
that's incorrect. mesa is the 3d graphic library on top of the native intel hd graphic drivers. the comparison actually make sense. you're not going to intel's osx drivers running in linux and you're not going to get mesa and intel's linux drivers running in osx, you realize that right? right? Oh right it's just trendy to hit on Phoronix for no reason. Well, they're testing the same games in the different environments to decide which environment is the fastest. That's very much apple to apple. I'd like to hear your test that's so much more accurate between Linux and Apple gaming with the same hardware. PLEASE ENLIGHTEN US.
- numbsafari 14y ago
  You seem to be really angry about this. I mean, shockingly so. Your comment just really jumped out at me and caught me by surprise. So much so that I felt like it might be good for someone to point it out to you. I'd like to prescribe an afternoon of UrT. Take about 45 minutes to frag the hell out of a Steve Jobs avatar and post back in the morning. :)
  - tedunangst 14y ago
    Anger aside, he's right. mesa is used even if the intel driver is used too. mesa includes a software implementation, but it's also the gateway to the hardware drivers.
    - ajross 14y ago
      Better stated: the Intel graphics driver is part of the Mesa/DRI2 framework. Mesa's core provides the OpenGL front end (API implementation, shader compilers, high level optimizers). Intel provides a low level component that takes that shader stream and turns it into programs and commands for the hardware, which is managed by a DRI driver in the kernel.

cydonian_monk 14y ago

I haven't yet benchmarked any flavour of Linux running natively on my current-gen Ivy Bridge 13" MacBook Pro, but I have compared Windows 7 to both System 10.7 and 10.8. Consistently Win7 out performs OS-X on "raw" math-heavy benchmarks. I've only uploaded my results form Geekbench-2, but the other benchmarking utils show similar numbers: Geekbench-2 Scores and [Results]: OS-X 10.8: 8743 [1] Windows 7: 10092 [2] Getting a Linux install to run via Bootcamp on this MBP is one of my next projects (most likely Gentoo), and I'm curious to see how well its performance stacks up against the other two. That is taking some time as... well, some things just don't work yet. (Thankfully I don't have to deal with the hybrid GPU setup that's in the Retina MBP.) 1: http://browser.primatelabs.com/geekbench2/902437 2: http://browser.primatelabs.com/geekbench2/810264

donniezazen 14y ago

Power consumption has been such a big problem on Linux. Their was PCIe-ASPM bug before and now new power bugs. Their is no end to it.

unicornporn 14y ago

I was just about to try Xubuntu on my MBP, but after seeing those power consumption stats I don't think I can justify it. Sad but true: Linux worked better in the old school Desktop era when batteries, power consumption, sleep/hibernation and wireless networks weren't there to mess things up.

malkia 14y ago

Things like TLB might also affect your performance, at the cost of memory fragmentation. For example bigger page size (1, 2 or 4MB) would need only two 2 page directory/table lookups, while 4096 pages would need 3. It's just one of the examples. For example 4096bytes vs 4MB page size: http://en.wikipedia.org/wiki/File:X86_Paging_4K.svg http://en.wikipedia.org/wiki/File:X86_Paging_4K.svg http://en.wikipedia.org/wiki/File:X86_Paging_4M.svg http://en.wikipedia.org/wiki/File:X86_Paging_4M.svg

navpatel 14y ago

All these benchmarks are great and all... but in terms of using Linux on a Macbook Pro, the shitty support for the multitouch trackpad/touchpad alone makes linux barely usable. Unless this has changed recently, in which case someone please correct me

Someone 14y ago

Lots of data, but preciously little information. For example: - Why is python so much slower on Mac OS X? if I had to guess, I would say it is more due to a version difference than due to a compiler change. It might also be some particularly slow library code on Mac OS X, though. - What is causing the sometimes huge differences in CPU-only tasks such as the matrix computations? Compiler, system overhead due to inefficient code, system overhead because the system also performs other work (for example, did the OS index files during some of the runs?), or maybe architectural choices that lead to system overhead (for example, a scheduler might switch tasks more often in order to keep UI response low)? With such info, this would have been a great article. As it stands now, all I conclude from it is that OpenGL is slower under Ubuntu, and that Linux filesystems are faster (but there, it would help to know why. HFS probably is a large part of the explanation, but it also might be that Mac OS ports of various tools do slightly different things, e.g. By flushing to disk more aggressively.