5 ms·

DSLRs got the same problem : just compare the Canon 70D ($900) with a Nikon D3300 ($400) on DxOMark. The Nikon has better image quality despite its low price an

DSLRs got the same problem : just compare the Canon 70D ($900) with a Nikon D3300 ($400) on DxOMark. The Nikon has better image quality despite its low price and bad reviews. We need objective benchmarks for everything. Especially when marketing is growing bigger each year. Even "Tech websites" are biased and not objective anymore.

dom0 9y ago
> We need objective benchmarks for everything. You might want to read Zen and the Art of Motorcycle Maintenance which digs into this fallacy. You might also be surprised how misleading objective IQ benchmarks are; the post processing in the camera's JPEG (and RAW processing... raw hasn't been raw for a long time) does a lot of stuff which is not well reflected by photographing a Siemens star.
- grimoald 9y ago
  I'm curious, do you mind expanding on the fallacy? I read the book you mentioned some years ago but don't know what you mean.
accountyaccount 9y ago
Aren't spec sheets objective benchmarks? Image quality can be subjective too. Also, the 70D might be $900, but that doesn't matter so much when someone has $4k worth of Canon equipment. Personally I've found I enjoy purchases more by making fewer of them and worrying less about specification minutiae. Does the product do what I want? Do I enjoy using it? Is it reliable? Done.
- arnaudsm 9y ago
  It depends of the specs covered. It can lead to some kind of Cargo Cult. Think of the 128 bit consoles, the 128Gb of RAM and the 12 Cores CPU for Gaming PCs. It's meant for advertising, but useless for the end user.
  - accountyaccount 9y ago
    True — it would be more useful to the user as "Can it run Crysis"
    - chris_7 9y ago
      Is there a more recent game that has taken over that position, or is Crysis still the benchmark? It's super old now! (not really into games anymore so no idea myself)
      - accountyaccount 9y ago
        It really became a meme, not sure if there's a de-facto replacement.
      - sp332 9y ago
        Crysis was hard on systems partly because the graphics were good and partly because it was not well-optimized. Game graphics have improved since then but they don't tend to take your system down as hard because they are better written. That's why Crysis is still a joke.
  - deleted 9y ago
    [deleted]
  - Veratyr 9y ago
    I assume by "128Gb", you meant 128GB. 128Gb of RAM is only 16GB, which can be found even in regular consumer PCs. Edit: To be a little clearer, lower case "b" is the unit for a bit. Upper case "B" represents a byte.
    - ReverseCold 9y ago
      Oh come on.
- beat 9y ago
  Image quality is extremely subjective. We don't shoot test patterns. We photograph the real world. And even within that ground, the basics of exposure (aperture, shutter speed, ISO) matter tremendously. The same subject, shot at f/2.8 and 1600 ISO, can look very different when shot at f/5.6 and 6400 ISO. And if you're relying on "program mode" or "scene mode" to automagically compute your exposure for you, all bets are off. The sensor doesn't matter nearly as much as the decision-making software. As a mostly-manual shooter, what matters to me is good ergonomics. One of these days, I'm going to bite the bullet and invest in a new Fuji rig, just for the superb manual-shooting ergonomics. But then I'd have to switch my brain from its familiar Nikon-shaped ruts...
  - jdmichal 9y ago
    This is one of the reasons I like DPReview over DxOMark. They have both sample images and a standardized studio image. The studio image lets you compare the same picture over multiple cameras, while the sample images give you a better feel of the camera in several scenarios. Pick the pictures that look better to you, because that's what you're going to be looking at after you use the thing! I went in to buying a camera last year wanting a Fujifilm X100T. Not least of which because it reminds me of the Minolta camera I learned to shoot on. But I ended up with a Sony RX100 IV instead, based on perceived image quality and video capabilities.
    - seanp2k2 9y ago
      I went through the same thing and ended up with the X100F. I noticed that the T was on sale and realized that the F was up for pre-order. I waited a while but it was worth it. I take it hiking with me all the time now, since it's small enough for that. I really wanted a full-frame DSLR, but the RX1R is expensive and others are too big for convenient travel (for me).
      - jdmichal 9y ago
        Yea. My main use case for it was my then-newborn son, so a compact was a good choice for me. I needed portability and quick pickup-to-use time, because image quality doesn't matter if you've already missed the moment. And 4k video really just sealed the deal. I would still probably pick the RX100 IV or V over the X100F for that alone.
londons_explore 9y ago
It's pretty easy for camera makers to game dxomark. They use those exact test images for tuning all their algorithms, and as a result end up with a camera which is great for those exact things tested, but might be lousy for the real world.
- strgrd 9y ago
  [citation needed]
  - stinkytaco 9y ago
    I think DxO Mark's credibility is easy to call into question. First of all, they offer services to "improve" manufacturers scores. Second of all, they rate their own equipment. Finally, and probably most importantly, they are just flat out wrong sometimes. Look at how highly they rated the Xperia Z5 camera, which was a horrible camera for actually taking pictures. All this said, DxO might be a perfectly credible source, but it's still doesn't reflect real world performance [1]. There are so many additional factors to consider when actually taking pictures. [1]: https://www.theverge.com/2016/4/29/11535102/smartphone-mobile-benchmarks-dxomark-score-test https://www.theverge.com/2016/4/29/11535102/smartphone-mobil...
- coldtea 9y ago
  >They use those exact test images for tuning all their algorithms, and as a result end up with a camera which is great for those exact things tested, but might be lousy for the real world. Well, it's not like actual test images are exactly microbenchmarks.
spott 9y ago
There is a lot more to a DSLR than the image quality. Ergonomics and other features in a number of cases are much more important.
xtreme 9y ago
70D vs D3300 is not a fair comparison. 70D has features geared towards enthusiasts such as articulating screen, more AF points, weather sealing, mg-ally body, pentaprism viewfinder, etc. If these features are not important to you, you can choose the cheaper 700D for about ~$600. However, I do agree that for entry level cameras, Nikon cameras tend to provide more value for money.
- arnaudsm 9y ago
  But why the 70D still not filming at 60fps in 1080p ? Canon is in such market dominance, they probably do this to keep the semi-pro market on the prohibitive 5D Mark IV. The consumer Market needs to know these kind of things, but there's no user-friendly website for comparing objectivly products. Just Amazon reviews, "Tech websites" and the Wallmart seller.
  - throwanem 9y ago
    Ken Rockwell, whatever one may think of him otherwise, does a very good job detailing the characteristics of pretty much all the camera gear. He's not to be taken too seriously in general, but I have yet to find a case where he's genuinely wrong on the facts, and I found his website very useful in deciding which DSLR would best suit me.
  - Veratyr 9y ago
    > But why the 70D still not filming at 60fps in 1080p ? Canon is in such market dominance, they probably do this to keep the semi-pro market on the prohibitive 5D Mark IV. If you want video you should really be using Canon's dedicated cameras or something more video focused like Panasonic's GH4/GH5. Even the 5D MkIV is very limited when it comes to video. It has some nice lenses but for the $3300 price of a MkIV you can get a Panasonic GH5 and an Atomos Ninja Inferno, which allows you to do 4K@60p 10-bit ProRes and HDR. Even without the Ninja Inferno, you can do 4K@60p 8-bit, 4K@30p 10-bit or 1080@60p 10-bit at 2/3rds the price.
  - deleted 9y ago
    [deleted]
stinkytaco 9y ago
Nikon has better sensors, but not so much better that it makes a noticeable difference. For a camera body you are looking for many things: features, durability, ergonomics, etc, but most of all you are looking for what glass works. If you've got a collection of Canon lenses, that Canon is what you should get. Lenses are easily the most important part of a photography rig, and Nikon is not the clear winner there.
- lb1lf 9y ago
  -At least one thing which counts BIG to Nikon's advantage in the lens market is the backward compatibility; just about every Nikon F lens made since -uhm- 1959? will mount on a current Nikon DSLR. (Yes - some will not meter properly, and a couple of rare ones may damage mirror - but the vast majority of Nikkor lenses are still usable, many of which can be bought for a pittance. This may or may not be important to a user.
  - mahyarm 9y ago
    I think in practice most of the market don't care about using 30+ year old lenses. And if your using manual only lenses, you have many options on which body you want to use.
  - throwanem 9y ago
    It was important to me, which is why I have a D5300 - I like shooting the same late-60s, early-70s glass on which I first learned. But the DSLR market is relatively niche, and odd therefore; most casual users not satisfied with phone cameras (and most are!) go for a point-and-shoot due to their relative simplicity.
    - baq 9y ago
      I bought a DSLR and almost never use it anymore. (Obviously not a photographer.) Should've bought a mirrorless with a pancake lens and WiFi/BT so I wouldn't have to juggle cards or cords.
      - 9y ago
vmarsy 9y ago
In France a retailer similar to BestBuy has a "Lab"[1] that usually has objective reviews, using a set of standards to compare items of the same category. Here's an example: [2], you can scroll to the conclusion to see the grades, but you also have an explanation behind it. [1] http://labo.fnac.com/ http://labo.fnac.com/ [2] http://labo.fnac.com/video-photo-numerique/appareil-photo-reflex/18208948314/test-labo-nikon-d5600-18-140mm-option-a-considerer/ http://labo.fnac.com/video-photo-numerique/appareil-photo-re...
bluetwo 9y ago
The problem with truly objective benchmarks is that it has both value and cost, meaning there becomes a market for the information itself, separate from the product.
sh87 9y ago
I feel the D5xxx series 5300, 5500 hit the sweet spot for me as a quasi casual photographer with its price, picture quality, amazing battery life and lightish weighing body. Felt slightly overpriced at $800 with 2 lenses for my budget. So far so good.
- throwanem 9y ago
  Which body and kit did you get? I found the D5300 kit (18-55 VR, 70-300) for $600 new on Amazon. (Incidentally, that 70-300 has a surprising resale value! I traded mine in at Service Photo toward a 70-300 VR II, and was very surprised to get $120 for it instead of the $50 or so I was expecting for an unstabilized kit lens with no meaningful secondhand market. Granted, that's one small shop in one town, but if anything I'd expect it to be low, considering that Service Photo is very much geared to the professional end and likely has the revenue problems of any small business.) (Further incidentally, I had a chance for a hands-on with a D5500 recently, and found it rather distasteful. The touchscreen seems nice, but would easily be nose-triggered when using the viewfinder, so you'd need to fold the screen in and out all the time, and it does not offer any of the PSAM modes on the selector dial! It's light and easy to maneuver, but all the same, I was glad I'd gone with the previous model.)
  - sh87 9y ago
    I got the 5500 with the 18/55 VR2and 55/200 VR1 lens from B&H. In hindsight though I feel a 18/140 vr2 would have suited me better. The touchscreen is always off for me when using the viewfinder. Never was an issue.
    - throwanem 9y ago
      That's reasonable - I have my 5300 configured the same way, and hadn't thought about that in relation to the 5500. If you can find a shop that sells used gear and has an 18-140 in stock, you might be able to trade the two kit lenses toward it - I can't recommend buying used sight unseen, lenses being the relatively delicate creatures that they are, but certainly the trade I mentioned before worked out really well for me! If my experience is any guide, you might reasonably expect to pay about half the new price for a gently used model with a few wear marks but perfect functionality, and the kit lenses together, if they're both in like-new condition, probably have enough resale value to cover most or all of the cost.
matthewmacleod 9y ago
There is no such thing as a truly "objective" benchmark, even for something as simple as a set of headphones. A DSLR camera is an order of magnitude more complex - it seems even less likely we'd be able to establish one for them.

dom0 9y ago

> We need objective benchmarks for everything. You might want to read Zen and the Art of Motorcycle Maintenance which digs into this fallacy. You might also be surprised how misleading objective IQ benchmarks are; the post processing in the camera's JPEG (and RAW processing... raw hasn't been raw for a long time) does a lot of stuff which is not well reflected by photographing a Siemens star.

grimoald 9y ago
I'm curious, do you mind expanding on the fallacy? I read the book you mentioned some years ago but don't know what you mean.

accountyaccount 9y ago

Aren't spec sheets objective benchmarks? Image quality can be subjective too. Also, the 70D might be $900, but that doesn't matter so much when someone has $4k worth of Canon equipment. Personally I've found I enjoy purchases more by making fewer of them and worrying less about specification minutiae. Does the product do what I want? Do I enjoy using it? Is it reliable? Done.

arnaudsm 9y ago
It depends of the specs covered. It can lead to some kind of Cargo Cult. Think of the 128 bit consoles, the 128Gb of RAM and the 12 Cores CPU for Gaming PCs. It's meant for advertising, but useless for the end user.
- accountyaccount 9y ago
  True — it would be more useful to the user as "Can it run Crysis"
  - chris_7 9y ago
    Is there a more recent game that has taken over that position, or is Crysis still the benchmark? It's super old now! (not really into games anymore so no idea myself)
    - accountyaccount 9y ago
      It really became a meme, not sure if there's a de-facto replacement.
    - sp332 9y ago
      Crysis was hard on systems partly because the graphics were good and partly because it was not well-optimized. Game graphics have improved since then but they don't tend to take your system down as hard because they are better written. That's why Crysis is still a joke.
- deleted 9y ago
  [deleted]
- Veratyr 9y ago
  I assume by "128Gb", you meant 128GB. 128Gb of RAM is only 16GB, which can be found even in regular consumer PCs. Edit: To be a little clearer, lower case "b" is the unit for a bit. Upper case "B" represents a byte.
  - ReverseCold 9y ago
    Oh come on.
beat 9y ago
Image quality is extremely subjective. We don't shoot test patterns. We photograph the real world. And even within that ground, the basics of exposure (aperture, shutter speed, ISO) matter tremendously. The same subject, shot at f/2.8 and 1600 ISO, can look very different when shot at f/5.6 and 6400 ISO. And if you're relying on "program mode" or "scene mode" to automagically compute your exposure for you, all bets are off. The sensor doesn't matter nearly as much as the decision-making software. As a mostly-manual shooter, what matters to me is good ergonomics. One of these days, I'm going to bite the bullet and invest in a new Fuji rig, just for the superb manual-shooting ergonomics. But then I'd have to switch my brain from its familiar Nikon-shaped ruts...
- jdmichal 9y ago
  This is one of the reasons I like DPReview over DxOMark. They have both sample images and a standardized studio image. The studio image lets you compare the same picture over multiple cameras, while the sample images give you a better feel of the camera in several scenarios. Pick the pictures that look better to you, because that's what you're going to be looking at after you use the thing! I went in to buying a camera last year wanting a Fujifilm X100T. Not least of which because it reminds me of the Minolta camera I learned to shoot on. But I ended up with a Sony RX100 IV instead, based on perceived image quality and video capabilities.
  - seanp2k2 9y ago
    I went through the same thing and ended up with the X100F. I noticed that the T was on sale and realized that the F was up for pre-order. I waited a while but it was worth it. I take it hiking with me all the time now, since it's small enough for that. I really wanted a full-frame DSLR, but the RX1R is expensive and others are too big for convenient travel (for me).
    - jdmichal 9y ago
      Yea. My main use case for it was my then-newborn son, so a compact was a good choice for me. I needed portability and quick pickup-to-use time, because image quality doesn't matter if you've already missed the moment. And 4k video really just sealed the deal. I would still probably pick the RX100 IV or V over the X100F for that alone.

londons_explore 9y ago

It's pretty easy for camera makers to game dxomark. They use those exact test images for tuning all their algorithms, and as a result end up with a camera which is great for those exact things tested, but might be lousy for the real world.

strgrd 9y ago
[citation needed]
- stinkytaco 9y ago
  I think DxO Mark's credibility is easy to call into question. First of all, they offer services to "improve" manufacturers scores. Second of all, they rate their own equipment. Finally, and probably most importantly, they are just flat out wrong sometimes. Look at how highly they rated the Xperia Z5 camera, which was a horrible camera for actually taking pictures. All this said, DxO might be a perfectly credible source, but it's still doesn't reflect real world performance [1]. There are so many additional factors to consider when actually taking pictures. [1]: https://www.theverge.com/2016/4/29/11535102/smartphone-mobile-benchmarks-dxomark-score-test https://www.theverge.com/2016/4/29/11535102/smartphone-mobil...
coldtea 9y ago
>They use those exact test images for tuning all their algorithms, and as a result end up with a camera which is great for those exact things tested, but might be lousy for the real world. Well, it's not like actual test images are exactly microbenchmarks.

spott 9y ago

There is a lot more to a DSLR than the image quality. Ergonomics and other features in a number of cases are much more important.

xtreme 9y ago

70D vs D3300 is not a fair comparison. 70D has features geared towards enthusiasts such as articulating screen, more AF points, weather sealing, mg-ally body, pentaprism viewfinder, etc. If these features are not important to you, you can choose the cheaper 700D for about ~$600. However, I do agree that for entry level cameras, Nikon cameras tend to provide more value for money.

arnaudsm 9y ago
But why the 70D still not filming at 60fps in 1080p ? Canon is in such market dominance, they probably do this to keep the semi-pro market on the prohibitive 5D Mark IV. The consumer Market needs to know these kind of things, but there's no user-friendly website for comparing objectivly products. Just Amazon reviews, "Tech websites" and the Wallmart seller.
- throwanem 9y ago
  Ken Rockwell, whatever one may think of him otherwise, does a very good job detailing the characteristics of pretty much all the camera gear. He's not to be taken too seriously in general, but I have yet to find a case where he's genuinely wrong on the facts, and I found his website very useful in deciding which DSLR would best suit me.
- Veratyr 9y ago
  > But why the 70D still not filming at 60fps in 1080p ? Canon is in such market dominance, they probably do this to keep the semi-pro market on the prohibitive 5D Mark IV. If you want video you should really be using Canon's dedicated cameras or something more video focused like Panasonic's GH4/GH5. Even the 5D MkIV is very limited when it comes to video. It has some nice lenses but for the $3300 price of a MkIV you can get a Panasonic GH5 and an Atomos Ninja Inferno, which allows you to do 4K@60p 10-bit ProRes and HDR. Even without the Ninja Inferno, you can do 4K@60p 8-bit, 4K@30p 10-bit or 1080@60p 10-bit at 2/3rds the price.
- deleted 9y ago
  [deleted]

stinkytaco 9y ago

Nikon has better sensors, but not so much better that it makes a noticeable difference. For a camera body you are looking for many things: features, durability, ergonomics, etc, but most of all you are looking for what glass works. If you've got a collection of Canon lenses, that Canon is what you should get. Lenses are easily the most important part of a photography rig, and Nikon is not the clear winner there.

lb1lf 9y ago
-At least one thing which counts BIG to Nikon's advantage in the lens market is the backward compatibility; just about every Nikon F lens made since -uhm- 1959? will mount on a current Nikon DSLR. (Yes - some will not meter properly, and a couple of rare ones may damage mirror - but the vast majority of Nikkor lenses are still usable, many of which can be bought for a pittance. This may or may not be important to a user.
- mahyarm 9y ago
  I think in practice most of the market don't care about using 30+ year old lenses. And if your using manual only lenses, you have many options on which body you want to use.
- throwanem 9y ago
  It was important to me, which is why I have a D5300 - I like shooting the same late-60s, early-70s glass on which I first learned. But the DSLR market is relatively niche, and odd therefore; most casual users not satisfied with phone cameras (and most are!) go for a point-and-shoot due to their relative simplicity.
  - baq 9y ago
    I bought a DSLR and almost never use it anymore. (Obviously not a photographer.) Should've bought a mirrorless with a pancake lens and WiFi/BT so I wouldn't have to juggle cards or cords.
    - 9y ago

vmarsy 9y ago

In France a retailer similar to BestBuy has a "Lab"[1] that usually has objective reviews, using a set of standards to compare items of the same category. Here's an example: [2], you can scroll to the conclusion to see the grades, but you also have an explanation behind it. [1] http://labo.fnac.com/ http://labo.fnac.com/ [2] http://labo.fnac.com/video-photo-numerique/appareil-photo-reflex/18208948314/test-labo-nikon-d5600-18-140mm-option-a-considerer/ http://labo.fnac.com/video-photo-numerique/appareil-photo-re...

bluetwo 9y ago

The problem with truly objective benchmarks is that it has both value and cost, meaning there becomes a market for the information itself, separate from the product.

sh87 9y ago

I feel the D5xxx series 5300, 5500 hit the sweet spot for me as a quasi casual photographer with its price, picture quality, amazing battery life and lightish weighing body. Felt slightly overpriced at $800 with 2 lenses for my budget. So far so good.

throwanem 9y ago
Which body and kit did you get? I found the D5300 kit (18-55 VR, 70-300) for $600 new on Amazon. (Incidentally, that 70-300 has a surprising resale value! I traded mine in at Service Photo toward a 70-300 VR II, and was very surprised to get $120 for it instead of the $50 or so I was expecting for an unstabilized kit lens with no meaningful secondhand market. Granted, that's one small shop in one town, but if anything I'd expect it to be low, considering that Service Photo is very much geared to the professional end and likely has the revenue problems of any small business.) (Further incidentally, I had a chance for a hands-on with a D5500 recently, and found it rather distasteful. The touchscreen seems nice, but would easily be nose-triggered when using the viewfinder, so you'd need to fold the screen in and out all the time, and it does not offer any of the PSAM modes on the selector dial! It's light and easy to maneuver, but all the same, I was glad I'd gone with the previous model.)
- sh87 9y ago
  I got the 5500 with the 18/55 VR2and 55/200 VR1 lens from B&H. In hindsight though I feel a 18/140 vr2 would have suited me better. The touchscreen is always off for me when using the viewfinder. Never was an issue.
  - throwanem 9y ago
    That's reasonable - I have my 5300 configured the same way, and hadn't thought about that in relation to the 5500. If you can find a shop that sells used gear and has an 18-140 in stock, you might be able to trade the two kit lenses toward it - I can't recommend buying used sight unseen, lenses being the relatively delicate creatures that they are, but certainly the trade I mentioned before worked out really well for me! If my experience is any guide, you might reasonably expect to pay about half the new price for a gently used model with a few wear marks but perfect functionality, and the kit lenses together, if they're both in like-new condition, probably have enough resale value to cover most or all of the cost.

matthewmacleod 9y ago

There is no such thing as a truly "objective" benchmark, even for something as simple as a set of headphones. A DSLR camera is an order of magnitude more complex - it seems even less likely we'd be able to establish one for them.