4 ms·

It seems they found it was actually less legible than the old ones - especially in the inverted color scheme some signs use. The previous tests were brand new

It seems they found it was actually less legible than the old ones - especially in the inverted color scheme some signs use. The previous tests were brand new Clearview signs beating old-and-faded classic signs -- which seems like an unbelievably embarrassing mistake.

ergothus 11y ago
Perhaps I'm more forgiving, but it seems easy for someone just thinking "It seems more clear to me, but how do I know for sure? Oh! I'll put them up on a road and see if the accident rates change!". Only when someone points out to me that the signs they are replacing are probably not new would it become "obvious". (In fact, while I read my first article on this, my internal thought was 'so why isn't this better if it tested so well?', then they mentioned the old signs, and my thought was 'oh, right, that makes sense'. So embarrassing mistake? No doubt. Unbelievably embarrassing mistake? No way, not with the sadly believable mistakes I've made in the past.
- joshschreuder 11y ago
  Well presumably it's not your job to do that sort of thing, but it becomes a bit more embarrassing when it is.
- iSnow 11y ago
  But wouldn't you do A/B tests in a driver lab first before going out into the wild with the fonts?
  - onion2k 11y ago
    Yes, but that could still generate in a positive result for the new font that wouldn't be seen in the real world over a long period of time. The simple fact that the new font is new could result in an improvement that wouldn't necessary last in the real world. Hence why the font is being retired after a decade of experience and data in a proper situation. A/B testing is notoriously hard to do well, especially if you're comparing something the user already knows well with something different. Controlling for familiarity is pretty much impossible.
- deleted 11y ago
  [deleted]
- kriro 11y ago
  Disagree. Very embarrassing mistake. Also very embarrassing that this wasn't caught by whoever signed off on replacing signs (but not surprising, I doubt those studies are read in detail often and people probably only browse the intro and conclusions). Basically a "you failed science 101" level of embarrassment mistake in my estimation. Testing two brand new signs with the different fonts should be a fairly obvious first idea. The hard/interesting part is taking into account learning effects for the previous font. There's a pretty interesting research question in how you'd age signs in there as well (as you want to know if the gains are positive over the entire lifetime of the sign) and some interesting ethics questions about field testing.
  - deleted 11y ago
    [deleted]
- phphphph 11y ago
  > It seems more clear to me, but how do I know for sure? Oh! I'll put them up on a road and see if the accident rates change!". Meh. The American way would be "Build a street for font, and see which one people are willing to pay more for"
vonklaus 11y ago
Edit, I had to delete most of the original post. The original testing methodology for the test is actually fairly well documented[1] and the font was designed by Donald Meeker, Terminal Deisgn agency[2][3] and some people from Pensylvania and Texas transportation/highway admin roles[3]. There is more on this topic in the citylab article[5] but it turns out Meeker developed and tested the font. Meeker, for his part says: “Helen Keller can tell you from the grave that Clearview looks better,” and thinks it could be a matter of cost according to the article[5]. TerminalFonts posted a hilarious tweet about it as well[4]. They seemed to have done a decent job researching it, and it was not as cut and dry as simply an a/b test with old signs, but it was rigorously documented. Although, I find reading font testing methodology absurdly boring so I skimmed it. Not really sure what the cost to the government was, but they are certainly killing the font according to every source. According to the citylab article: The FHWA has not yet provided any research on Clearview that disproves the early claims about the font’s benefits. So, I have to apologize, I briefly insinuated that maybe these guys forged the test to get the contract. Instead, they seem like pretty passionate design nerds who meticulously researched this topic and then developed a font around their findings. Apparently, they did a poor job though, according to some unreleased report. [1]http://clearviewhwy.com/ResearchAndDesign/_articles/TRB_Paper.pdf http://clearviewhwy.com/ResearchAndDesign/_articles/TRB_Pape... [2]http://www.terminaldesign.com/web-fonts/ http://www.terminaldesign.com/web-fonts/ [3]https://en.wikipedia.org/wiki/Clearview_(typeface) https://en.wikipedia.org/wiki/Clearview_(typeface) [4]https://twitter.com/TerminalDesign/status/693173610538799105 https://twitter.com/TerminalDesign/status/693173610538799105 [5]http://www.citylab.com/commute/2016/01/official-united-states-highway-sign-font-clearview/427068/ http://www.citylab.com/commute/2016/01/official-united-state... PRICING from www.citylab.com ============== Jurisdictions that adopt Clearview must purchase a standard license for type, a one-time charge of between $175 (for one font) and $795 (for the full 13-font typeface family) and up, depending on the number of workstations. (Meeker and Associates disclaimed exclusive rights to the “Clearview” copyright.)
- kriro 11y ago
  Thanks for digging up those studies. I hope those weren't the only tests that were conducted. Two groups of 12, 65 years plus, daytime and night time. Word recognition test+legibility test (2 studies with the same setup/subjects etc.). Tested words: Purcel, Dorset, Conyer, Bergen, Ordway, Gurley. Parked car (1993 Ford) at varying distances. Repeat-measures design. ANOVA for material and ANOVA + t-tests for fonts. They used the same sign with bolt-on names for all different cases so the issue linked in the OP doesn't apply to this test. So we have no weather conditions, a very selective population (also notable that gender isn't mentioned at all which is normally standard practice in the subjects section), a questionable list of words (no mention what word/letter distributions look like and why this specific group was picked...they do explain why they built the word group the way they did however). Only one vehicle type. No discussion of sample size, power etc. in the linked studies. It's a decent set of constraints for a single test but we're talking about highway signs for the general population I'd expect more extensive tests than that.

ergothus 11y ago

Perhaps I'm more forgiving, but it seems easy for someone just thinking "It seems more clear to me, but how do I know for sure? Oh! I'll put them up on a road and see if the accident rates change!". Only when someone points out to me that the signs they are replacing are probably not new would it become "obvious". (In fact, while I read my first article on this, my internal thought was 'so why isn't this better if it tested so well?', then they mentioned the old signs, and my thought was 'oh, right, that makes sense'. So embarrassing mistake? No doubt. Unbelievably embarrassing mistake? No way, not with the sadly believable mistakes I've made in the past.

joshschreuder 11y ago
Well presumably it's not your job to do that sort of thing, but it becomes a bit more embarrassing when it is.
iSnow 11y ago
But wouldn't you do A/B tests in a driver lab first before going out into the wild with the fonts?
- onion2k 11y ago
  Yes, but that could still generate in a positive result for the new font that wouldn't be seen in the real world over a long period of time. The simple fact that the new font is new could result in an improvement that wouldn't necessary last in the real world. Hence why the font is being retired after a decade of experience and data in a proper situation. A/B testing is notoriously hard to do well, especially if you're comparing something the user already knows well with something different. Controlling for familiarity is pretty much impossible.
deleted 11y ago
[deleted]
kriro 11y ago
Disagree. Very embarrassing mistake. Also very embarrassing that this wasn't caught by whoever signed off on replacing signs (but not surprising, I doubt those studies are read in detail often and people probably only browse the intro and conclusions). Basically a "you failed science 101" level of embarrassment mistake in my estimation. Testing two brand new signs with the different fonts should be a fairly obvious first idea. The hard/interesting part is taking into account learning effects for the previous font. There's a pretty interesting research question in how you'd age signs in there as well (as you want to know if the gains are positive over the entire lifetime of the sign) and some interesting ethics questions about field testing.
- deleted 11y ago
  [deleted]
phphphph 11y ago
> It seems more clear to me, but how do I know for sure? Oh! I'll put them up on a road and see if the accident rates change!". Meh. The American way would be "Build a street for font, and see which one people are willing to pay more for"

vonklaus 11y ago

Edit, I had to delete most of the original post. The original testing methodology for the test is actually fairly well documented[1] and the font was designed by Donald Meeker, Terminal Deisgn agency[2][3] and some people from Pensylvania and Texas transportation/highway admin roles[3]. There is more on this topic in the citylab article[5] but it turns out Meeker developed and tested the font. Meeker, for his part says: “Helen Keller can tell you from the grave that Clearview looks better,” and thinks it could be a matter of cost according to the article[5]. TerminalFonts posted a hilarious tweet about it as well[4]. They seemed to have done a decent job researching it, and it was not as cut and dry as simply an a/b test with old signs, but it was rigorously documented. Although, I find reading font testing methodology absurdly boring so I skimmed it. Not really sure what the cost to the government was, but they are certainly killing the font according to every source. According to the citylab article: The FHWA has not yet provided any research on Clearview that disproves the early claims about the font’s benefits. So, I have to apologize, I briefly insinuated that maybe these guys forged the test to get the contract. Instead, they seem like pretty passionate design nerds who meticulously researched this topic and then developed a font around their findings. Apparently, they did a poor job though, according to some unreleased report. [1]http://clearviewhwy.com/ResearchAndDesign/_articles/TRB_Paper.pdf http://clearviewhwy.com/ResearchAndDesign/_articles/TRB_Pape... [2]http://www.terminaldesign.com/web-fonts/ http://www.terminaldesign.com/web-fonts/ [3]https://en.wikipedia.org/wiki/Clearview_(typeface) https://en.wikipedia.org/wiki/Clearview_(typeface) [4]https://twitter.com/TerminalDesign/status/693173610538799105 https://twitter.com/TerminalDesign/status/693173610538799105 [5]http://www.citylab.com/commute/2016/01/official-united-states-highway-sign-font-clearview/427068/ http://www.citylab.com/commute/2016/01/official-united-state... PRICING from www.citylab.com ============== Jurisdictions that adopt Clearview must purchase a standard license for type, a one-time charge of between $175 (for one font) and $795 (for the full 13-font typeface family) and up, depending on the number of workstations. (Meeker and Associates disclaimed exclusive rights to the “Clearview” copyright.)

kriro 11y ago
Thanks for digging up those studies. I hope those weren't the only tests that were conducted. Two groups of 12, 65 years plus, daytime and night time. Word recognition test+legibility test (2 studies with the same setup/subjects etc.). Tested words: Purcel, Dorset, Conyer, Bergen, Ordway, Gurley. Parked car (1993 Ford) at varying distances. Repeat-measures design. ANOVA for material and ANOVA + t-tests for fonts. They used the same sign with bolt-on names for all different cases so the issue linked in the OP doesn't apply to this test. So we have no weather conditions, a very selective population (also notable that gender isn't mentioned at all which is normally standard practice in the subjects section), a questionable list of words (no mention what word/letter distributions look like and why this specific group was picked...they do explain why they built the word group the way they did however). Only one vehicle type. No discussion of sample size, power etc. in the linked studies. It's a decent set of constraints for a single test but we're talking about highway signs for the general population I'd expect more extensive tests than that.