User interaction in mixed reality interactive storytelling

• Published in: The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings pp. 304 - 305
• Main Authors: Cavazza, M., Martin, O., Charles, F., Marichal, X., Mead, S.J.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2003
• Subjects: Animation; Cameras; Engines; Graphics; Image analysis; Image segmentation; Mirrors; Real time systems; Rendering (computer graphics); Virtual reality
• ISBN: 0769520065; 9780769520063
• DOI: 10.1109/ISMAR.2003.1240732

In this paper, we describe a mixed reality system based on a "magic mirror" model, in which the user's image is captured in real time by a video camera, extracted from his/her background and mixed with a 3D graphic model of a virtual image including the synthetic characters taking part in the story. The resulting image is projected on a large screen facing the user, who sees his/her own image embedded in the virtual stage with the synthetic actors. The graphic component of the mixed reality world is based on a game engine, Unreal Tournament 2003. This engine not only performs graphic rendering and character animation but incorporates a new version of our previously described storytelling engine. A single 2D camera facing the user analyses the image in real-time by segmenting the user's contours.