Taiwan Blazes New Trails With Extended Reality

Taiwan has already begun forging a reputation as a producer of VR creative content. Now the introduction of 5G is driving the development of its VR industry to new levels.

Taiwan-produced works made with virtual reality (VR) have become a regular feature at international film festivals. Industry promoters see this development as confirming that Taiwan has become one of the top players in extended reality (XR) content, which includes VR and other forms of computer-altered reality.

Part of Taiwan’s success in this area is due to its technological ecosystem, which has helped it attract international filmmakers and production companies for cooperation on creating XR content.

The early results have been promising. In three of the four years since the prestigious Venice Film Festival started a competition for VR films, Taiwan-made works have been nominated in that category. In 2019, Taiwan had the third most nominations in the VR category – seven out of a total of 39 works – behind only the U.S. and UK.

Three Taiwanese directors talk about their VR works during an online panel discussion held in Taipei as part of this year’s Venice Film Festival. From left to right: Tang Chi-Chung, director of The Sick Rose; Hsu Chih-yen, director of Jiou Jia (Home); and John Hsu, director of Great Hoax: The Moon Landing. Photo: Louise Watt

Now the introduction of 5G services in Taiwan is driving interest in the industry to new levels and pushing the island’s telecom operators to develop their own XR content. These companies hope that the faster download and image display speeds – showcased through VR film works, virtual concerts, and sports events – will entice more people to sign up for 5G services.

“5G brings users a brand-new experience via its features of high bandwidth and low latency,” says Chen Ming-shih, president of Chunghwa Telecom’s Mobile Business Unit. “In the past, people often experienced dizziness or discomfort when viewing VR content. The reason was blurred images or unstable video streaming – not because the resolution of the video wasn’t good enough, but because the internet speed wasn’t fast enough.”

However, with new VR devices becoming lighter and more powerful and 5G connection allowing for smoother video quality, it will now be possible to watch 8K VR content comfortably, Chen says.

Extended reality (XR) is an umbrella term covering all forms of computer-altered reality, such as virtual reality (VR), augmented reality (AR), and mixed or merged reality (MR). VR is a complete immersive experience, usually involving a special headset that allows viewers to enter a 360-degree cinema experience. AR, accessible on smartphones, overlays virtual objects onto a real-world environment. MR combines elements of both VR and AR, enabling real-world and virtual objects to interact.

In recent years, Taiwan has been developing hardware and software for XR, such as HTC’s VR headsets that were first used for gaming. Filmmakers and companies including HTC are aiming to use technology to deepen the film-going experience.

HTC Vive Originals, the company’s content production brand, has been around since 2017. Several of its works have been shortlisted in the Venice Film Festival’s VR section. They include productions by both emerging and renowned directors, such as the multi-award-winning Tsai Ming-liang. His 55-minute The Deserted is a 360-degree film centered on a man who lives in a dilapidated house and is visited by the ghosts of his mother and partner. Viewers are placed right in the middle of the various scenes to observe the characters and their environments.

Liu Szu-ming, president of HTC Vive Originals, in front of the HTC Vive Originals logo. Photo: Louise Watt

Liu Szu-ming, president of HTC Vive Originals, says hardware such as the VR headsets primarily used by gamers has been driving the XR content industry. He cites the American VR company Oculus as an example and says that Vive plays a similar role in Taiwan.

“However, the U.S. is very big compared to Taiwan, so it would help if the creators here were more concentrated” and could share resources, says Liu.

“At the beginning, the market isn’t big because there aren’t as many people using these devices, so we have no way to really build an ecosystem,” Liu says. “Our strategy is to achieve success at international film festivals, and then more filmmakers will be willing to invest. They want to invest in VR because they hope that Taiwan and their creative energy can be seen.”  

Taipei was one of 15 cities around the world chosen to screen VR films as part of this year’s Venice Film Festival in September. Curious movie-goers could don headsets and watch films – both those that require viewers to stand up and move their bodies to interact with the story, and those where they sit down and swivel around a 360-degree environment.

One Taiwanese offering was the short VR film Jiou Jia (Home), directed by Hsu Chih-yen, co-director of the 2018 hit film  Dear Ex, and produced by Kaohsiung Film Archive. It was featured in the “Best of VR – Out of Competition” section and is a snapshot of life in a Kaohsiung family home. The viewer is given the role of the disabled and mute grandmother, whose family visit and talk to her, before leaving her alone again with her maid. Liz Rosenthal, Venice Film Festival programmer, called Home “a very moving portrait of old age and isolation.”

The works were screened in the Ambassador Theater on Changchun Road, which has started Taiwan’s first commercially operated VR cinema in partnership with HTC.

Taiwan has garnered a strong reputation for VR films over the past two years thanks to government support, the island’s advanced tech ecosystem, and a free creative and artistic environment. Not least, Taiwanese filmmakers and companies have been able to benefit from experience and expertise via international collaborations – particularly with France, which was previously seen as a pioneer in developing VR cultural works. There have also been collaborations with the U.S., Japan, and Argentina.

The Kaohsiung Film Archive, with funding from the Ministry of Economic Affairs and Ministry of Culture, set up Asia’s first immersive VR theater in 2017. The Kaohsiung VR Film Lab also funds VR projects.

The Ministry of Culture established Taiwan Content Creative Agency, or TAICCA, last year to promote the development of Taiwan’s cultural content industry. The government aims to raise the global profile of Taiwanese works, as part of its attempts to play catch-up with South Korea’s cultural industries.

XR content is one of the areas that TAICCA is promoting so that it can become a “prominent art form in the market,” says Hu Ching-fang, president of TAICCA. 

She says that Taiwan’s success with VR filmmaking “came as a surprise to us,” and that the fledgling VR industry had fed off the pre-existing technology ecosystem in Taiwan.

“It just happened because we have all the right technology companies here, and we have creators, as well as a tradition of nourishing and encouraging high art forms,” Hu says. She adds that the Venice Film Festival’s selection of seven Taiwanese VR films for last year’s competition was especially eye-opening for her.

Among those films was Gloomy Eyes, a VR animation about a little girl and a zombie-child in search of the sun. It was produced by HTC Vive Originals and French production company Atlas V.

Aurélien Dirler, a consultant at TAICCA, says that Taiwan’s strength could also be seen in its presence at large film markets, such as the one held each year alongside the Cannes Film Festival.

“In the Cannes market last year, Taiwan was the third country by the number of attendees after France and the U.S. and that was something very new,” he says. “Four years ago, Taiwan was only about VR hardware, but in just a few years it has become a major player for content.”

This year, TAICCA has launched immersive content grants to encourage co-productions between Taiwan-based and foreign companies. In June, it awarded a total of NT$3.5 million to five projects “to explore the international market potential for XR technology.”

Dirler says that Taiwan’s experience in hardware manufacturing was a major attraction for foreign studios. “One of the advantages of Taiwan is that they have experience in collaborations and co-productions,” he says. “They have this ecosystem which is very complete because they are the only country in the world along with the U.S. to actually have hardware manufacturers like HTC, MSI, and Acer. That’s why there was interest from studios from Europe to find partners in Taiwan.”

Deputy Minister of Foreign Affairs Tseng Ho-jen watches the Foreign Ministry’s latest VR short film, Three Crucial Steps, about Taiwan’s response to COVID-19. Photo: Louise Watt

While not likely to win any awards, even the foreign ministry is using VR to get out its messaging. Underlining the government’s support for VR, in October the ministry released a short VR film, Three Crucial Steps, explaining how Taiwan responded to COVID-19.

At a press conference, Deputy Foreign Minister Tseng Ho-jen donned a headset and watched a virtual version of the former vice president, Chen Chien-jen, and medical and quarantine inspection staff, explain the steps they took. “We trusted the people, and the people trusted us,” Chen said in the film, extolling the government’s transparency.

Entering a new phase

The introduction of 5G in Taiwan in June has accelerated the development of XR content.

Local telecom companies are now also getting in on the act. Chunghwa Telecom plans to work with Verizon Media, part of New York-based Verizon Communications Inc., to develop AR and VR sports-related content. It is also using 8K live broadcast technologies to stream virtual concerts.

“Through 5G, remote concertgoers can see their favorite performers from a short distance away from various viewpoints, just like being in a real performance,” says Chunghwa Telecom’s Chen. “XR content is absolutely the easiest way for users to feel the power of 5G.”

Both Chunghwa and Taiwan Mobile are collaborating with XRSpace, a company founded by former HTC CEO Peter Chou that has created its own VR headset and VR world. In May, it launched what it billed as the world’s first 5G mobile VR headset to support 5G technology, plus the world’s first social VR platform designed for mass market users. In this new environment, people can create their own avatars and interact with friends, colleagues, or strangers.

Attracting consumers to watch and pay for XR content is one way Taiwan’s telecom companies hope to get back some of their hefty 5G investment. While specific 5G applications may change the way many industries work in years to come, companies like Chunghwa are expecting ordinary consumers to adopt 5G services and drive initial growth.

“At the initial promotion stage, we will still prioritize content that is likely to resonate in the consumer market,” says Chen. “Entertainment content is currently our main focus of development.”

VR content is a smart way for telecom companies to demonstrate 5G services to a skeptical public, says Jimmy Cheng, director of content at Digital Domains, a visual effects company.

“I think over 80% of the people in the world still don’t understand why we need 5G,” he told a Taipei panel that was part of the Venice Film Festival in September. Our current home Wi-Fi is quite sufficient for us to do our work, upload and download, and stream most of the content we want to watch, he said.

“So, to promote 5G, telecoms need to find a good way to showcase its capabilities, and VR is actually a very good way to do that,” Cheng says, noting that the bandwidth of the Wi-Fi we are used to “is really a problem” when downloading VR content.

Can Taiwan continue to be a leader in XR content? For HTC Vive Originals, VR film is just the beginning. The company is developing a blockchain trading platform for XR content, and recently produced what was billed as the world’s first immersive musical experience combining VR and a somatosensory device – a chair that moves automatically and is meant to make the watching and listening experience feel that much more immersive.

The work is a VR musical video produced for Taiwanese singer Anpu, who appears in 3-D animated form in the virtual landscape. HTC Vive Originals say this creates a “brand-new business model” for the music industry.

Cheng, who licenses and distributes XR content, said that to succeed, Taiwanese cultural content has to be able to cross over into other markets.

“When I try to display cultural content from Africa to Taiwanese, normally it’s going to fail because we don’t understand the culture,” he says. “I think the best way to make Taiwanese cultural content is to work with people around the world and make the content very diversified.”