6 ms·

Virtual lane graphics for swimming with CSS3

355 points by CWIZO 14y ago

kayoone 14y ago
Thats pretty neat, i was always wondering how they really do this in big football(soccer) machtes or olympics where its done realtime and with no control over camera images. This explains in nicely! Also well done on the project, i bet it really added value for spectators!
- chili_powder 14y ago
  First, let me say that this is a very neat project for a high schooler before what I say next because I'm not trying to downplay what he did. He figured out how to overlay an image in realtime over video, which is neat. However, this is only a small part of what is actually done to present the final image. The more complicated part is the computer vision which allows for things like composing the image so that the swimmers are always above the labels (i.e. the superimposed labels appear to be at the bottom of the pool). They also are using computer vision to recognize where the labels should go, rather than some kind of positional mount on the camera as he suggests. The image is processed through a computer which is capable of recognizing the lane lines, the boundaries of the pool, and the bottom of the pool. This way, when the camera moves the image is updated automatically. Again, I'm not trying to downplay what was done here, it is a very nice hack.
  - pionar 14y ago
    In American football, there is also some basic "green screen" technology put in so that first down lines appear under the players and other objects. That's why sometimes, parts of the players go missing because the colors are too close to the color of the field.
  - Zimahl 14y ago
    The NFL strictly uses positional cameras. The first down markers, ads, and other on-field graphics are only available from specific views. It's actually a very low-tech approach. College and NFL football typically has 3 to 4 stationed cameras up at ~30 degree angle. These cameras are bolted down and do not move from their positions. They are able to transmit their zoom, angle, and pitch very precisely which allows the production team to know exactly where they are pointing at all time. During the transmission they can choose the exact location for the first down markers, etc., and all the high-angle cameras will add it in. To do the overlay they choose a swatch on the field to use as a green-screen. They only trouble that arises is when a teams uniforms is very close to the field color.
    - dsl 14y ago
      They don't use green screen. Before the game with the field empty they do a pan and scan of the entire field from every camera. That gives the system a baseline for change detection that works regardless of team colors.
      - Zimahl 14y ago
        By 'green-screen' I meant that they take a small snapshot of the field and use those colors as a replacement. A scan before the game would do them very little good, especially in open-air stadiums where game conditions can change immediately.
        dsl 14y ago
        Well apparently it does them a lot of good, because that is what they do. Check out the Wikipedia article on First and Ten I posted further up in the thread.
        Zimahl 14y ago
        You could also watch a Youtube[1] video explaining (near the end) what happens when field conditions change - fog, mud, etc. [1] https://www.youtube.com/watch?v=Vh9af_gXxlM https://www.youtube.com/watch?v=Vh9af_gXxlM
ck2 14y ago
I suspect with this ability the star wars opening credits in pure css3 is coming soon (or maybe already done). added: ah of course: http://blogs.sitepointstatic.com/examples/tech/css3-starwars/index.html http://blogs.sitepointstatic.com/examples/tech/css3-starwars...
- ErikRogneby 14y ago
  Nice. "Also by mentioning 'Star Wars' everyone will understand what I mean. And I'll receive several thousand more visits."
EEGuy 14y ago
Nice work. The pro company that does this for broadcast is Sportvision, www.sportvision.com/
randall 14y ago
This is great! We (Vidpresso) think this sort of thing is the future of TV, and that's why we're building it. Currently, our first product is a way for TV stations to take Facebook and Twitter, and show it using their own graphical look on-air. We have some big broadcasters, like Anderson Cooper's show Anderson Live, as well as some interantional stations in Germany (QVC!), Panama, and Lebanon. In the US we have more than a dozen US TV stations, and Univision uses us for their network programming. Here's some samples: http://www.youtube.com/user/randallvpp http://www.youtube.com/user/randallvpp All we use is Google Chrome on the output side to make it simple to display the graphics. All three of us have video & TV backgrounds, but we all turned into coders somehow. We're always looking to meet more interesting folks who are interested in the intersection of video and the web. I have info in my profile, plus you can check out our website: http://www.vidpresso.com/ http://www.vidpresso.com/
- geuis 14y ago
  Are you guys perhaps in SF? I'm an engineer that has some (amateur) experience doing video editing and some graphics with Premiere and AE.
  - randall 14y ago
    Not in SF... but we're a virtual team. Shoot me a note... we should chat!
    - geuis 14y ago
      Email sent!
- graupel 14y ago
  Heh, as soon as I saw this, I expected to pop in here and see this EXACT comment from you! PS: Vidpresso is the real deal, good stuff.
  - randall 14y ago
    :) You going to NAB? We'll be there, so stop by!
jezfromfuture 14y ago
Yeah there are countless programs to do this outside web browsers some of my favorites of the last 20 years http://en.wikipedia.org/wiki/Scala,_Inc http://en.wikipedia.org/wiki/Scala,_Inc (Amiga tool , see we did stuff too before the web !) http://en.wikipedia.org/wiki/Video_Toaster http://en.wikipedia.org/wiki/Video_Toaster (another awesome amiga based h/w ) The point is it doesn't take any messing about with web technologies , the people doign this sort of titling professionaly for Sky/Fox BBC are using a combination of broadcast hardware & occasionally Apple Shake in the case of sky sports though i may be out of touch now since shake is not avail much longer.
- Samuel_Michon 14y ago
  Right, Apple Shake is no more, but a lot of its features found their way to Apple Motion (which, at $50, is a steal) [1]. Apart from Sportvision for live sports broadcasting, the tools of the trade are still Adobe After Effects, Avid Motion Graphics, and Autodesk Flame. [1] http://www.apple.com/finalcutpro/motion/ http://www.apple.com/finalcutpro/motion/
danso 14y ago
This reminds me of a humorous blog post by the New York Times interactive graphics desk, in which they debated several options for overlaying flags for swimmers...not for live TV but for their 3D renders: http://chartsnthings.tumblr.com/post/29133053109/the-olympics-are-almost-over http://chartsnthings.tumblr.com/post/29133053109/the-olympic...
bsimpson 14y ago
I'm not sure I understand why CSS was chosen for this project. It looks to me like he's creating a static image, transforming it, and feeding it as a green screen overlay into a video mixer. Since everything is static, wouldn't it have been simpler to just make a PNG in PhotoShop and use that?
- mintplant 14y ago
  The names of the swimmers have to be updated dynamically. Easier to update via HTML and CSS, and since he's piping through his laptop's VGA output as the keyed overlay, it can be modified in realtime.
dsl 14y ago
For anyone who is interested on how this works in the real world, the system is called First and Ten, and is built by Sportvision. (I'm pretty sure they have also patented the hell out of it) http://en.wikipedia.org/wiki/1st_%26_Ten_(graphics_system) http://en.wikipedia.org/wiki/1st_%26_Ten_(graphics_system) http://www.sportvision.com/ http://www.sportvision.com/ EDIT: The demo reel is pretty good nerdporn too http://www.sportvision.com/media/sportvision-reel-2012 http://www.sportvision.com/media/sportvision-reel-2012
jaseflow 14y ago
This is very cool! Things like this are what's going to allow CSS to become the design language of future UI. I can see this being used in a car where an interface is displayed over the top corner of a screen.
shabble 14y ago
There was a Wired article[1] I saw here a while back about some of the tech being developed to do this (and much much more) dynamically for sailboat racing, filmed live from helicopter(s). The technical challenges, as well as potential for live info-graphics in sport make this a pretty interesting area, and it's nice to see relatively modest setups capable of producing what were, until recently, digustingly complex and expensive video effects/compositing tasks. [1] http://www.wired.com/playbook/2012/08/ff_americascup_ellison/all/ http://www.wired.com/playbook/2012/08/ff_americascup_ellison...
jQueryIsAwesome 14y ago
My (nonexistent) OCD was kicking in and I had to fix the skew in demo.html: http://rapedinheaven.com/with_skew.html http://rapedinheaven.com/with_skew.html (5deg, 0deg)
rtexal 14y ago
there are quite a few videos i see out there that place augmented objects in a video which moves accordingly to the shift of the camera. However that is not live. Combined with AR algorithms, I am pretty sure that this could be a kickass overlay for web live videos in the future.
rtexal 14y ago
there are quite a few videos i see out there that place augmented objects in a video which moves accordingly to the shift of the camera. However that is not live. Combined with AR algorithms, I am pretty sure that this could be a kickass overlay for web live videos in the future.

355 points by CWIZO 14y ago

Thats pretty neat, i was always wondering how they really do this in big football(soccer) machtes or olympics where its done realtime and with no control over camera images. This explains in nicely! Also well done on the project, i bet it really added value for spectators!

chili_powder 14y ago
First, let me say that this is a very neat project for a high schooler before what I say next because I'm not trying to downplay what he did. He figured out how to overlay an image in realtime over video, which is neat. However, this is only a small part of what is actually done to present the final image. The more complicated part is the computer vision which allows for things like composing the image so that the swimmers are always above the labels (i.e. the superimposed labels appear to be at the bottom of the pool). They also are using computer vision to recognize where the labels should go, rather than some kind of positional mount on the camera as he suggests. The image is processed through a computer which is capable of recognizing the lane lines, the boundaries of the pool, and the bottom of the pool. This way, when the camera moves the image is updated automatically. Again, I'm not trying to downplay what was done here, it is a very nice hack.
- pionar 14y ago
  In American football, there is also some basic "green screen" technology put in so that first down lines appear under the players and other objects. That's why sometimes, parts of the players go missing because the colors are too close to the color of the field.
- Zimahl 14y ago
  The NFL strictly uses positional cameras. The first down markers, ads, and other on-field graphics are only available from specific views. It's actually a very low-tech approach. College and NFL football typically has 3 to 4 stationed cameras up at ~30 degree angle. These cameras are bolted down and do not move from their positions. They are able to transmit their zoom, angle, and pitch very precisely which allows the production team to know exactly where they are pointing at all time. During the transmission they can choose the exact location for the first down markers, etc., and all the high-angle cameras will add it in. To do the overlay they choose a swatch on the field to use as a green-screen. They only trouble that arises is when a teams uniforms is very close to the field color.
  - dsl 14y ago
    They don't use green screen. Before the game with the field empty they do a pan and scan of the entire field from every camera. That gives the system a baseline for change detection that works regardless of team colors.
    - Zimahl 14y ago
      By 'green-screen' I meant that they take a small snapshot of the field and use those colors as a replacement. A scan before the game would do them very little good, especially in open-air stadiums where game conditions can change immediately.
      - dsl 14y ago
        Well apparently it does them a lot of good, because that is what they do. Check out the Wikipedia article on First and Ten I posted further up in the thread.
        Zimahl 14y ago
        You could also watch a Youtube[1] video explaining (near the end) what happens when field conditions change - fog, mud, etc. [1] https://www.youtube.com/watch?v=Vh9af_gXxlM https://www.youtube.com/watch?v=Vh9af_gXxlM

ck2 14y ago

I suspect with this ability the star wars opening credits in pure css3 is coming soon (or maybe already done). added: ah of course: http://blogs.sitepointstatic.com/examples/tech/css3-starwars/index.html http://blogs.sitepointstatic.com/examples/tech/css3-starwars...

ErikRogneby 14y ago
Nice. "Also by mentioning 'Star Wars' everyone will understand what I mean. And I'll receive several thousand more visits."

EEGuy 14y ago

Nice work. The pro company that does this for broadcast is Sportvision, www.sportvision.com/

randall 14y ago

This is great! We (Vidpresso) think this sort of thing is the future of TV, and that's why we're building it. Currently, our first product is a way for TV stations to take Facebook and Twitter, and show it using their own graphical look on-air. We have some big broadcasters, like Anderson Cooper's show Anderson Live, as well as some interantional stations in Germany (QVC!), Panama, and Lebanon. In the US we have more than a dozen US TV stations, and Univision uses us for their network programming. Here's some samples: http://www.youtube.com/user/randallvpp http://www.youtube.com/user/randallvpp All we use is Google Chrome on the output side to make it simple to display the graphics. All three of us have video & TV backgrounds, but we all turned into coders somehow. We're always looking to meet more interesting folks who are interested in the intersection of video and the web. I have info in my profile, plus you can check out our website: http://www.vidpresso.com/ http://www.vidpresso.com/

geuis 14y ago
Are you guys perhaps in SF? I'm an engineer that has some (amateur) experience doing video editing and some graphics with Premiere and AE.
- randall 14y ago
  Not in SF... but we're a virtual team. Shoot me a note... we should chat!
  - geuis 14y ago
    Email sent!
graupel 14y ago
Heh, as soon as I saw this, I expected to pop in here and see this EXACT comment from you! PS: Vidpresso is the real deal, good stuff.
- randall 14y ago
  :) You going to NAB? We'll be there, so stop by!

jezfromfuture 14y ago

Yeah there are countless programs to do this outside web browsers some of my favorites of the last 20 years http://en.wikipedia.org/wiki/Scala,_Inc http://en.wikipedia.org/wiki/Scala,_Inc (Amiga tool , see we did stuff too before the web !) http://en.wikipedia.org/wiki/Video_Toaster http://en.wikipedia.org/wiki/Video_Toaster (another awesome amiga based h/w ) The point is it doesn't take any messing about with web technologies , the people doign this sort of titling professionaly for Sky/Fox BBC are using a combination of broadcast hardware & occasionally Apple Shake in the case of sky sports though i may be out of touch now since shake is not avail much longer.

Samuel_Michon 14y ago
Right, Apple Shake is no more, but a lot of its features found their way to Apple Motion (which, at $50, is a steal) [1]. Apart from Sportvision for live sports broadcasting, the tools of the trade are still Adobe After Effects, Avid Motion Graphics, and Autodesk Flame. [1] http://www.apple.com/finalcutpro/motion/ http://www.apple.com/finalcutpro/motion/

danso 14y ago

This reminds me of a humorous blog post by the New York Times interactive graphics desk, in which they debated several options for overlaying flags for swimmers...not for live TV but for their 3D renders: http://chartsnthings.tumblr.com/post/29133053109/the-olympics-are-almost-over http://chartsnthings.tumblr.com/post/29133053109/the-olympic...

bsimpson 14y ago

I'm not sure I understand why CSS was chosen for this project. It looks to me like he's creating a static image, transforming it, and feeding it as a green screen overlay into a video mixer. Since everything is static, wouldn't it have been simpler to just make a PNG in PhotoShop and use that?

mintplant 14y ago
The names of the swimmers have to be updated dynamically. Easier to update via HTML and CSS, and since he's piping through his laptop's VGA output as the keyed overlay, it can be modified in realtime.

dsl 14y ago

For anyone who is interested on how this works in the real world, the system is called First and Ten, and is built by Sportvision. (I'm pretty sure they have also patented the hell out of it) http://en.wikipedia.org/wiki/1st_%26_Ten_(graphics_system) http://en.wikipedia.org/wiki/1st_%26_Ten_(graphics_system) http://www.sportvision.com/ http://www.sportvision.com/ EDIT: The demo reel is pretty good nerdporn too http://www.sportvision.com/media/sportvision-reel-2012 http://www.sportvision.com/media/sportvision-reel-2012

jaseflow 14y ago

This is very cool! Things like this are what's going to allow CSS to become the design language of future UI. I can see this being used in a car where an interface is displayed over the top corner of a screen.

shabble 14y ago

There was a Wired article[1] I saw here a while back about some of the tech being developed to do this (and much much more) dynamically for sailboat racing, filmed live from helicopter(s). The technical challenges, as well as potential for live info-graphics in sport make this a pretty interesting area, and it's nice to see relatively modest setups capable of producing what were, until recently, digustingly complex and expensive video effects/compositing tasks. [1] http://www.wired.com/playbook/2012/08/ff_americascup_ellison/all/ http://www.wired.com/playbook/2012/08/ff_americascup_ellison...

jQueryIsAwesome 14y ago

My (nonexistent) OCD was kicking in and I had to fix the skew in demo.html: http://rapedinheaven.com/with_skew.html http://rapedinheaven.com/with_skew.html (5deg, 0deg)

rtexal 14y ago

there are quite a few videos i see out there that place augmented objects in a video which moves accordingly to the shift of the camera. However that is not live. Combined with AR algorithms, I am pretty sure that this could be a kickass overlay for web live videos in the future.

rtexal 14y ago