Nippon TV’s Mixed Reality - The next step to changing our TV viewing experience forever
The technological advance has given us VR and AR, creating new experiences, but not really changing the way we entertain ourselves most frequently – in front of the screen. Now, Japan’s Nippon TV has presented a new technology, mixing traditional TV viewing with Mixed Reality content outside the screen and in our own living room.
Imagine watching a live sports event and simultaneously getting access to stats and life-size images of stars in the space around the screen. Pretty soon that will be possible.
Yako Molhov talked about this exciting next step in changing our TV viewing experience forever with Ayato Fujii from the Engineering & Technology Department of Nippon TV.
Imagine watching a live sports event and simultaneously getting access to stats and life-size images of stars in the space around the screen. Pretty soon that will be possible.
Yako Molhov talked about this exciting next step in changing our TV viewing experience forever with Ayato Fujii from the Engineering & Technology Department of Nippon TV.
Mr. Fujii, last month you organized an event, showcasing your latest Mixed Reality (MR) innovations. What is this Mixed Reality about and how is it blending with traditional TV viewing?
We believe MR is a technology that allows reality space to overlap with 3D computer graphics. In virtual reality, the head-mounted display one wears does not show the TV screen. The Microsoft HoloLens MR device, however, is transparent and allows the user to enjoy the real-life TV show while showing 3D computer graphics outside of the screen. It occurred to us that we can expand the possibilities of our content by using this technology. For this event, we featured demo content much like how motor shows display concept cars.
Judging by Nippon TV’s promo video for the Microsoft HoloLens glasses, this is essentially a holographic experience. Is this the next step in our technological evolution?
We see the Microsoft HoloLens MR technology as the next development of Virtual Reality and Augmented Reality. Our promotion video only allows us to show what it is all about in 2D but the actual experience is 3D, and it was difficult for us to share it in its full glory.
What are the main elements of this MR that viewers can experience when using their lenses and TV sets?
Viewers can continue to enjoy their favorite TV shows with their own eyes, just like they always have. But now we can add new experiences, as our demo content shows. By wearing the smart glasses during live sports broadcasts, viewers can see the score board or player information displayed outside of the TV set. They can also have sports stars or their favorite musicians appear life-size in their living room. Commercials and weather forecasts can also be seen in 3D, right on top of the coffee table.
You have tapped this MR “a completely new viewing experience”. How is it different from other Virtual Reality (VR) systems available now?
With VR, you enter the virtual world while wearing the device so you cannot see the TV in your actual world. With MR, you can enjoy what is on the TV screen and the MR content simultaneously, even while drinking coffee or eating.
What interactive possibilities does your technology offer?
Our latest demo content allows multiple people to watch TV in the same space and also enjoy the MR 3D computer graphics together, with each person free to move around and get their own unique vantage point of the life-sized athletes and stars they love.
Are you producing content specifically for this MR viewing or you can adapt content, produced in the traditional way? Must the main TV content be shot using special equipment in order to be used in MR?
For the demo content, everything that is confined within the TV screen was produced using the traditional methods. The MR and computer graphics that are displayed outside of the TV screen, however, were produced separately.
How expensive is it to produce MR content?
It depends on the quality, but you can expect the 3D computer graphics portion to cost about the same as 3D computer graphics created for TV shows. Recording 3D images of people, however, is still a lot more expensive than using the traditional methods of capturing them on video.
Are you familiar with the Interactive Mixed Reality (IMR) technology developed by Norway’s The Future Group which was used in the gameshow Lost in Time? How does your technology compare to theirs?
We are familiar with Lost in Time. The IMR technology it employs enables synthesizing of images within the TV screen, so it is different from our MR technology.
Will you work exclusively with Microsoft on this technology?
Our current technology development is being done completely in-house. At the moment, Nippon TV is not working with Microsoft and there are no plans to do so in the near future.
The price of the HoloLens is still very high - $3.000 - and used by developers for the time being. How long would it take to reach mass proportions and be adopted by millions of people around the world?
We do not know how long it will take for MR to reach mass proportions, but when the issues of price, weight, and scope of display are solved, it is highly likely that we will see widespread use.
Aren’t you afraid that Mixed Reality could inherit some of the major drawbacks of VR, i.e. expensive and cumbersome equipment, the need for special content; etc. and with time turn into a flop like 3DTV?
We cannot say we did not worry about this, but nothing will be discovered for the future if we do not take on challenges simply because we are anxious. Only those who embrace challenges grow rich in experiences and knowledge. As a leading company, Nippon TV has a mandate to create for the future.
When will viewers and consumers of all types of content be able to enjoy immersive technology without special gears? Is the time of holo-projectors near or it will remain in the realm of science fiction for now?
That, we do not know. Based on our knowledge, there are currently no technologies of that sort that can quickly be put to practical use.
We believe MR is a technology that allows reality space to overlap with 3D computer graphics. In virtual reality, the head-mounted display one wears does not show the TV screen. The Microsoft HoloLens MR device, however, is transparent and allows the user to enjoy the real-life TV show while showing 3D computer graphics outside of the screen. It occurred to us that we can expand the possibilities of our content by using this technology. For this event, we featured demo content much like how motor shows display concept cars.
Judging by Nippon TV’s promo video for the Microsoft HoloLens glasses, this is essentially a holographic experience. Is this the next step in our technological evolution?
We see the Microsoft HoloLens MR technology as the next development of Virtual Reality and Augmented Reality. Our promotion video only allows us to show what it is all about in 2D but the actual experience is 3D, and it was difficult for us to share it in its full glory.
What are the main elements of this MR that viewers can experience when using their lenses and TV sets?
Viewers can continue to enjoy their favorite TV shows with their own eyes, just like they always have. But now we can add new experiences, as our demo content shows. By wearing the smart glasses during live sports broadcasts, viewers can see the score board or player information displayed outside of the TV set. They can also have sports stars or their favorite musicians appear life-size in their living room. Commercials and weather forecasts can also be seen in 3D, right on top of the coffee table.
You have tapped this MR “a completely new viewing experience”. How is it different from other Virtual Reality (VR) systems available now?
With VR, you enter the virtual world while wearing the device so you cannot see the TV in your actual world. With MR, you can enjoy what is on the TV screen and the MR content simultaneously, even while drinking coffee or eating.
What interactive possibilities does your technology offer?
Our latest demo content allows multiple people to watch TV in the same space and also enjoy the MR 3D computer graphics together, with each person free to move around and get their own unique vantage point of the life-sized athletes and stars they love.
Are you producing content specifically for this MR viewing or you can adapt content, produced in the traditional way? Must the main TV content be shot using special equipment in order to be used in MR?
For the demo content, everything that is confined within the TV screen was produced using the traditional methods. The MR and computer graphics that are displayed outside of the TV screen, however, were produced separately.
How expensive is it to produce MR content?
It depends on the quality, but you can expect the 3D computer graphics portion to cost about the same as 3D computer graphics created for TV shows. Recording 3D images of people, however, is still a lot more expensive than using the traditional methods of capturing them on video.
Are you familiar with the Interactive Mixed Reality (IMR) technology developed by Norway’s The Future Group which was used in the gameshow Lost in Time? How does your technology compare to theirs?
We are familiar with Lost in Time. The IMR technology it employs enables synthesizing of images within the TV screen, so it is different from our MR technology.
Will you work exclusively with Microsoft on this technology?
Our current technology development is being done completely in-house. At the moment, Nippon TV is not working with Microsoft and there are no plans to do so in the near future.
The price of the HoloLens is still very high - $3.000 - and used by developers for the time being. How long would it take to reach mass proportions and be adopted by millions of people around the world?
We do not know how long it will take for MR to reach mass proportions, but when the issues of price, weight, and scope of display are solved, it is highly likely that we will see widespread use.
Aren’t you afraid that Mixed Reality could inherit some of the major drawbacks of VR, i.e. expensive and cumbersome equipment, the need for special content; etc. and with time turn into a flop like 3DTV?
We cannot say we did not worry about this, but nothing will be discovered for the future if we do not take on challenges simply because we are anxious. Only those who embrace challenges grow rich in experiences and knowledge. As a leading company, Nippon TV has a mandate to create for the future.
When will viewers and consumers of all types of content be able to enjoy immersive technology without special gears? Is the time of holo-projectors near or it will remain in the realm of science fiction for now?
That, we do not know. Based on our knowledge, there are currently no technologies of that sort that can quickly be put to practical use.
