ShadowPlay2.5D: A 360-Degree Video Authoring Tool for Immersive Appreciation of Classical Chinese Poetry
(A) An example of Chinese shadow play: the performer animates the arm piece by fixing one control rod, and manipulating another control rod. (B) Similar to Chinese shadow play, we have designed ShadowPlay2.5D: users animate the image element by indicating the rigid parts, fixed points, free points first, and then manipulating one rigid part by rotating the free point with a stylus. ((A) photo credit goes to Chinese Shadow Puppetry, used with permission)
Abstract
An immersive experience brought about by virtual reality can potentially enhance the appreciation of classical Chinese poetry, which is difficult to describe clearly in everyday language or ordinary media. However, making 3-dimensional illustrations for a 360-degree display in virtual reality is usually a labor-intensive and time-consuming procedure and hard to master for non-professional media creators, such as teachers. Motivated by the homology theory of classical Chinese poetry and painting, we propose an image-based approach of building 2.5-dimensional immersive stories to visualize classical Chinese poetry. Specifically, using Chinese shadow play as a metaphor, we have designed and implemented ShadowPlay2.5D, a sketch-based authoring tool to help novices create 360-degree videos of classical Chinese poetry easily. To ensure coverage of the diverse themes in Chinese poetry and preserve the sense of culture, we build a Chinese ink-painting style image repository of essential poetic elements identified via crowdsourcing. To facilitate construction of 2.5-dimensional scenes, we design features that support puppet-like animation, instancing, and camera organization in a 3-dimensional environment. Through two user studies, we show that ShadowPlay2.5D can help novices make a short 360-degree video in about 10-15 minutes, and the 2.5D stylized illustrations created can bring about a better immersive experience for poetry appreciation.
Created Results
Authoring Process
Supplementary materials
Citation
@article{10.1145/3352590,
author = {Zhao, Zhenjie and Ma, Xiaojuan},
title = {ShadowPlay2.5D: A 360-Degree Video Authoring Tool for Immersive Appreciation of Classical Chinese Poetry},
year = {2020},
issue_date = {February 2020},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {13},
number = {1},
issn = {1556-4673},
url = {https://doi.org/10.1145/3352590},
doi = {10.1145/3352590},
journal = {J. Comput. Cult. Herit.},
month = feb,
articleno = {Article 5},
numpages = {20},
keywords = {virtual reality, Classical Chinese poetry, education, authoring tool, image-based modeling, pen-based interface, 2.5D animation, 360-degree video, storytelling}}