9 ms·
The story behind football's innovative yellow first down line
- ksenzee 12y agoI love this. I've spent a lot of time between plays imagining just what kind of engineering is behind that magic first down line. Of course, I should have known it came down to this: "Our engineering team... worked around the clock through the summer..." Sometimes I wonder if there is any such thing as a correctly scoped project that can be completed during daylight hours on weekdays.
- xirdstl 12y agoSadly, "around the clock" makes for a sexier story. I'm reminded of a previous job where working late and being super busy was worn like a badge of honor - while 75% of the day was filled with meetings. Also, many people didn't take much vacation time until the last half of December, because they were just too busy all year long to take it beforehand. "Hey, how's it going?" "Super busy!" I don't miss that part of the culture.
- chrismcb 12y agoSometimes there are just hard deadlines you can't do a whole lot about. In this case, it was pretty much hunker down and get it done in 4 months, or wait another year. For a fledgling company, that extra 12 months was probably the difference between success and failure.
- jabelk 12y agoI have dabbled in a little bit of video production, and on the (rare) occasion that I catch a few minutes of televised football, that yellow line always blows me away. Mentioning how difficult it must be to whoever is actually watching the game results in weird stares as if it's the most natural thing in the world, but I'm glad to read that I'm not going crazy. And I had no idea how extensive the setup actually was.
- cclogg 12y agoYeah totally, having some experience with 3D motion tracking and green screens, I was thinking: wow, it's so smooth and so perfectly tracked, AND cuts between player/grass so well. I know how finicky it can be to key out elements, so it is quite amazing that whatever algorithm they have in play can do it so well (is there a jersey color that's close to grass color lol? Wonder what the edge cases are).
- halisaurus 12y agoIt's alluded to in the article that outdoor fields and green jerseys are the hardest. For outdoor fields the lighting is affected by the time of day, cloud coverage, players moving along the field, and plenty of other things I'm sure. It says they spent a lot of time creating a unique palette for the fields and that domed stadiums were preferred by the engineers on site. Indoor lighting probably reduced the shadows a lot.
- baddox 12y agoI wonder if they use any tracking technology that looks at large groups of pixels over time. If you could track a player as a closed shape moving through space, you could mask out that player from the background even if pixels on the player's uniform were identical in color to the field's palette.
- nmjohn 12y agoThere are many teams that have very similar jersey colors to grass. Moreover, what is even more impressive, is there are some astroturf fields that aren't even green [1]! And the first down line works just fine on that as well. [1]: http://upload.wikimedia.org/wikipedia/en/5/56/Bronco_Stadium_September_25_2010.JPG http://upload.wikimedia.org/wikipedia/en/5/56/Bronco_Stadium...
- chaz 12y ago> Hill liked the concept, of course, but Fox thought that the price, $25,000 per game, was too steep. [...] Our last stop was ESPN and the reception was decidedly different. So interesting to see that FOX balked at the seemingly low $25k but ESPN didn't flinch and insisted on exclusivity for games they didn't even have rights to. Telling of how the networks viewed themselves and each other, but also how important it is to find the customers that are the true believers.
- jnazario 12y ago> insisted on exclusivity for games they didn't even have rights to given the price per game and the market value for that, it seems like a low price to screw your competition. by insisting on an exclusive deal they couldn't use it in the playoffs - anywhere. what a well played "screw you buddy" move.
- AznHisoka 12y agoReminds me of those Excite and Yahoo idiots who refused to buy Google for a cheap $1 million. Some people just don't deserve to be at the top.
- bbwharris 12y agoBut would google have ended up as google if they were bought at that price?
- Goopplesoft 12y agoCliche but, at least his, hindsight is 20/20
- KPLauritzen 12y agoThere is a recent interview with Marc Andreesen on EconTalk where he talk about how hard it would have been to realize that Google was a good buy at the time. Details here: http://www.econtalk.org/archives/2014/05/marc_andreessen.html http://www.econtalk.org/archives/2014/05/marc_andreessen.htm...
- 12y ago
- digikata 12y agoI had a cousin who worked with Sportvision at the time. I recall he had mentioned that early on, the support team for the line was comprised of classic nerd engineers who couldn't care less about football. And ironically, they had somebody's dream job of being sent to every single football game being covered by the line tech.
- alelefant 12y agoHis mention of the glowing hockey puck made me realize that I haven't seen a glowing puck for awhile. Granted, it may be that I haven't watched hockey as much as I did when I was a kid, but I certainly didn't see it last year when I watched some games. When I did see the glowing puck it made me feel like I was watching NBA Jam on Ice.
- clutchski 12y agoThis was a terrible invention.
- jnazario 12y agoit was a terrible invention. thankfully it didn't survive long.
- rurounijones 12y agoNever seen hockey before and had a look on youtube. The blue glowy bit is ok I think but that red tail comet is atrocious, it is completely distracting, covers up waaay to much of the screen. Would you have been happy with just the blue glowy affect?
- Lutrinae 12y agoBefore HD tvs it might have helped a lot, seeing a puck on an old screen is very difficult. But on the newer HD tvs, you can see the puck clearly enough I wouldn't want it, it probably isn't necessary.
- jnazario 12y ago> Would you have been happy with just the blue glowy affect? nope, agree with the other follow up - HD is good enough that any "enhancements" are now a distraction.
- frenchman_in_ny 12y agoNot only was it a (blue) glowing puck, but it would also make a red-orange streak on screen when there was a slapshot (er... "one-timer"). https://en.wikipedia.org/wiki/FoxTrax https://en.wikipedia.org/wiki/FoxTrax
- RogerL 12y agoI work here, if you have any questions...
- KnightHawk3 12y agoI don't actually know what this yellow line is, probably because I am from Australia. Have you got a sample video for it?
- namityadav 12y agoThe yellow line in the 3 to 5 seconds mark in this video: https://www.youtube.com/watch?v=QtxoYaMebF0 https://www.youtube.com/watch?v=QtxoYaMebF0
- KnightHawk3 12y agoThat is pretty cool, thanks!
- icambron 12y agoWhich also shows the evolution of the yellow line idea: another line (usually blue) for the line of scrimmage, which turns out to be really useful. I think I've also seen a line for maximum field goal range.
- Someone 12y agoAnd you probably also have seen advertisements inserted behind the end zone. I do not know how much use they make of it, but that allows the broadcaster to send different advertisements to different markets (the low tech variant that I have seen of this in soccer is to have one country show the main overview from one side of the field, and another country the main overview from the other side. That way, one can sell the main advertising space at the center of the field across from the main camera twice) Also, in speed skating, the technology gets used to project a moving line indicating where a skater should be to reach some goal (win the race, world record, etc)
- Gracana 12y agoThe few times I've watched football, I've been fascinated by this technology. I imagine football fans take it for granted at this point, but I just want to say "wow, are you SEEING this!?" For a live broadcast to be augmented in that way is damn impressive.
- habosa 12y agoIn the Eagles vs. Lions game earlier this year there was about 6" of snow on the field and tons more coming down. The field barely showed any hints of being green underneath. The photographers couldn't even use autofocus because of the size and frequency of the snowflakes. Due to the yellow line technology, or some other augmentation they were doing live on the feed, all of the players would appear translucent for brief periods. It was really weird, but it made me stop and think about how many impressive live-processing algorithms get their hands on the video before it gets to me. It was the first time I appreciated how tough it is to do the "yellow line". This article is a very interesting insight to how it came about.
- deleted 12y ago[deleted]
- function_seven 12y agoI remember that game. They were using the overlay to project all the yard markers (numbers, lines were dug out). Detroit was wearing their whites, and were sometimes mistaken as snow by the computer, causing them to appear as apparitions beneath the markings. That was my favorite game of the year.
- StavrosK 12y agoDoes anyone have a video of that?
- zenojevski 12y agoI never watched the this sport, but I think it could be this one? [1] When it fails, it makes people look life frosted glass [2], but actually it's extraordinary that it's still working almost perfectly. [1]: https://www.youtube.com/watch?v=E9qYZCl_F8I https://www.youtube.com/watch?v=E9qYZCl_F8I [2]: http://i.imgur.com/FExuLmY.png http://i.imgur.com/FExuLmY.png
- StavrosK 12y ago
- AndrewKemendo 12y agoI have always thought that it was a complex computer vision problem that made this work, very interesting that it was more than just software and makes sense. I am tempted to think that this was the first widely adopted (and accepted) instance of augmented reality. Very cool in that context!
- RogerL 12y agoA lot of this has changed since then, of course. No grass swatches anymore! We have a lot more computer horsepower to throw at the problem now, plus a lot of gained experience. We have a bay in our lab with a to-scale football field where we can run indoor tests, but on the field it is computer vision that is doing it all. The cameras are instrumented to get attitude, zoom, and so on (we have to modify the lens to get the accuracy we need), and then of course you have to compensate for lens distortion. If you look closely, the "line" is not a line, but a curve. If we drew a line it would look stupid, and not reg well in most parts of the image.
- AndrewKemendo 12y agoVery cool. Are you guys doing statistical filtering at all or is it strictly palette matching? Seems like your data inputs are specific enough that you are trying to get where you don't need much buffer on either side of the wavelength.
- RogerL 12y agoI will have to ask. I don't work with this technology, and haven't touched the code. I work in computer vision for baseball, with a completely different set of challenges.
- kanamekun 12y agoIs an automated "yellow box" to show the strike zone ever going to happen? I feel like that would have a huge impact on the viewing experience!
- brucehart 12y agoWhen this first debuted, my grandfather (born in 1917) thought that they had a piece of nylon material that was connected between the first down markers on the sidelines. That's when you know a technology is good. It's so seamless that it never occurs to a layman that it might even be possible.
- nikcub 12y agoInteresting from a startup perspective is that they raised a lot of money first and then tried to figure out if they could implement the idea. This definitely is an idea where if it isn't being developed in a lab at a large corp or by government grant you would need to raise money for.
- AznHisoka 12y agoI always thought it a neon lights in the actual football field.
- taftster 12y agoHere's a decent youtube video that basically shows the tech in action. http://youtu.be/Vh9af_gXxlM http://youtu.be/Vh9af_gXxlM
- morgante 12y agoImpressive technology, but perhaps someone with more football knowledge could explain why they don't just paint lines on the field?
- imjared 12y agoThe line moves on every play. Paint would be just a bit cumbersome :)
- ken_railey 12y agoIt represents the "line of scrimmage", which is where the ball starts at the beginning of each play.
- chrishas35 12y agoThe yellow lines marks the line to gain for a first down, not the line of scrimmage. Some broadcasts are showing the line of scrimmage now usually as red or blue IIRC, but yellow is used across the board for the line to gain.
- kaptain 12y agoThe rules for American football can get pretty complicated, but an abbreviated explanation for the yellow line might go something like this: The team that has the ball has four tries (i.e. downs) to advance the ball 10 yards. If the team that has possession is abl to advance the ball 10 yards or more within four tries, the marker is reset to the next 10 yards. That's why it's insufficient to mark the lines on the field: if I started at the 10 yard mark and advanced to the 12 yard mark, the line would have to be placed at the 22 yard mark. Painting lines would mean that the entire field would eventually have to be painted and unpainted. I love American football but it's pretty much an acquired taste (lots of stoppage time, tons of instant replay, incomprehesible rules, heart-breaking inconsistency in the officiating, etc.)
- logn 12y agoI can't see how exclusivity into the playoffs was a mistake. ESPN believed in the technology and they wanted to punish their competitors. I think the creators of the yellow line should be proud to reward ESPN for their foresight and take part in enjoying the Emmy together as opposed to sharing the limelight with another TV network who never believed in the company. Financially, I would think you're in better bargaining position too: well, this year the line is $30K/game since it's proven to work and you don't need to do R&D work like ESPN did with us.
- catshirt 12y agoi'm not a huge football fan but i've always really loved the new technology they bring to the show. the yellow line has always impressed me, and the fly-by cameras they introduced are really cool too. i think i've seen something similar, but it would be cool to have full on video-game style indications. player markers, etc.
- seanhandley 12y agoAmerican Football.
- oneeyedpigeon 12y agoExactly. Across the world, "football" usually refers to soccer; please have some consideration for the non-USA audience.
- stuartcw 12y agoPerxactly!
- dragop 12y agoStrictly speaking that should be association football, although everyone calls it football outside the US of course. "Football" existed before, but simplifying somewhat, just about all the modern games of football derive from rules codified at British private-education schools in the 19th century. So for example, rugby football originated at Rugby private-education school, which then developed into American football. Off the top of my head I can think of association football, American football, Canadian football, rugby football, Australian rules football and Gaelic football.
- beloch 12y ago"Gepner, a TV production expert, engineering guru Stan Honey and I (the business person) -- had been inspired by Stan's invention of the glowing hockey puck technology the previous year for Fox, technology which we licensed upon leaving the company in exchange for an equity interest for News Corp." This article opens in May of 1998, just one month before the final use of the glowing puck in June of 1998. By this point everybody in that meeting must have known how hockey fans had reacted to the glowing puck. The interesting thing is that they bravely forged ahead anyways! Execution may have been the key difference in fan acceptance. Fox made the puck into a giant glowing and color-shifting comet that practically screamed, "PEW PEW PEW!!!". These guys made a simple line that was meant to look, as much as the technology would allow, like a simple chalk line on the turf that some poor slob had to create (while erasing another) after every down. Would we still have puck-following tech in the NHL if Fox had been run by people with even the tiniest amount of taste or restraint?
- eropple 12y agoI think the glowing puck would have still died when high-definition TV came along. I think HDTV saved the hockey viewing experience, because it made it a whole bunch easier actually watch the game. The problems solved by the glowing puck then stopped being problems.
- BioGeek 12y agoThe latest sport where Sportvision used this technology is sailing, called LiveLine there. During the last America's Cup [1] they used it to show the starting, finish and boundary lines, distance between the boats,... [1] http://youtu.be/r0LH5cCuc_4 http://youtu.be/r0LH5cCuc_4
- snowwrestler 12y agoThat was an incredible use of the technology. I'd watched sailing on TV before, but for the first time I could actually see and understand the scope of the race and the strategies of the skippers.
- deleted 12y ago[deleted]
- jccooper 12y agoI was hoping they'd get into the later patent struggle and competitive technology. (But I guess it's really a piece about a single interview someone manager to score.) Another company in the field (PVI) at about the same time used vision-based technology to similar effect. They got into a mutual patent infringement battle and ended up settling by cross-license. I'm still trying to figure out why NBC (maddeningly) doesn't show the line of scrimmage. At one point I saw some vague report that NBC uses the third company in this space (SportsMEDIA) and that company has a license to the patent for the first down line but not the patent for the line of scrimmage. Which sounds silly and plausible at the same time. However, they advertise line of scrimmage as available: http://www.smt.com/products/smart/line-visual-insert-publisher http://www.smt.com/products/smart/line-visual-insert-publish... It is probably a policy decision based on a 2008 in-game controversy where the line of scrimmage was a yard off and one of the broadcasters (Madden) was using it as a basis for evaluating the call, and didn't know that the officials can't see the line. In either case, they need to deal with it and get the line of scrimmage back in. It's a very annoying omission.
- alistairSH 12y agoThe yellow line really is amazing from a "user interface" perspective. It's so natural that I had forgotten there was a time when it didn't exist. And, as mentioned elsewhere in the comments, it seems everybody has an old uncle that swears it's just nylon between the down markers (or paint, or something else that isn't "magic").
- shekhar101 12y agoThis is amazing. I only had read about it but this one gives a first hand perspective. Hats off to the team! Can someone point to the algorithm/exact technology used in this? Would love to read more about the technology.
- EEGuy 12y agoPreceding his accomplishments at Sportvision, Stan Honey created, in 1985, an automobile navigation system showing your car's position on a moving map. This was before Google Maps, before GPS, and before digital maps of cities. You couldn't just go out and buy them in 1983; Stan's company, Etak, had to create them itself [1]. Per [2], here's a description of the the hardware platform: The original Etak Navigator was a specially-packaged Intel 8088-based system with 256K RAM, 32K EPROM, 2K SRAM, and a cassette tape drive on which digital maps and some of the operating system were stored. The 8088 has a 16-bit address bus, thus can directly address only 64 K. Must have been some bank switching to get to the full 256 K RAM. This for all the code and geodata. edits for formatting --------- [1] Source: Personal conversation [2] https://en.wikipedia.org/wiki/Etak https://en.wikipedia.org/wiki/Etak
- bitwize 12y agoThe 8088 can address 1 MiB with a somewhat wonky segment:offset addressing scheme.
- tn13 12y agoThat is not football. It is Egg Shaped Handball.
- cweiss 12y agoI'd seen an interview[1] elsewhere with, I believe, the founder of SportVision where they indicated that their primary source of revenue now is selling the vast amounts of telemetry data they collect during events. His example was of the MLB, SportVision has high-resolution data on every single pitch, the MLB teams are paying through the nose for that data to run through their analytics. SportVision indicated that, based on ball trajectory, they often know more about the state of a given pitcher's health than the team or the pitcher themselves. [1] http://www.sportvision.com/media/hank-adams-presshere http://www.sportvision.com/media/hank-adams-presshere