Enactive Ciné. Multimodal interaction with tagged audiovisual streams, ANR project
A research project, Enactive Ciné, has been filed in February to the ANR to answer the program “Contenu et interactions” (Content and interaction), call for project 2009.
At a time where the share of digital broadcasting rises in the content of the networks, we must take interest in purposes that go beyond the simple "consumption" of audiovisual streams (watch a video from the beginning to the end, for instance) in order to explore new practical forms of collaborative reading and writing (1).
Some uses of this kind already exist, such as navigation in the stream from a point to another with a pointer linked to a player/text-editor, use and modification in real time of a table of contents or an index in a movie, navigate from a stream to another, tagging videos on a community website, etc.
Interaction with a stream can be made with information from the stream only (sound and pictures), if that's the case, it will be restricted to navigation, but it will be all the more richer if extra information such as tags or metadata is available.
A tagged audiovisual document is a document which is linked to an information structure. Therefore, a tag links digital information (text, data, sound, picture...) to a space-time fragment of the document. A ground example of a tagging structure is the merging of subtitles in a movie. More complex tags exist, such as description of shots, characters, atmospheres, area of significance in the picture, etc.
Tags can also be connected together, in order to show that a movie scene refers to one of another movie, for instance. A corpus of tagged videos is structured sets of audiovisual documents that can offer links between movie fragments.
While tools allowing us to perceive and interact with tagged streams already exist (old use of subtitles, hyper-video browsing used by communautary video websites), it appears necessary in the audiovisual field to visit again the enactive principles and the sense-motor theories. From that, new devices of active perception and reading can be elaborated (Aubert and Prié, 2005 [2]; Prié and Puig, 2009 [3]), in order to pursue the issues about the origins of cinema and, more particularly, the bi-directional role of the camera/projector.
Furthermore, the distinctive aspect of audiovisual streams (the fact that they have a length, to which the reader's stream of consciousness synchronizes), the massive digitization of contents and the societal issues relating to it, the availability at low cost of new modes of interaction (sense of touch, multipoint, gesture and eye-tracking for instances) require that a global reflection must be made on multimodal interaction with time streams.
Notes
1 - Bernard Stiegler, Annotation, navigation, édition électronique : vers une géographie de la connaissance (Tagging, browsing, digital publishing : towards a mapping of knowledge), Ec/arts, 1(2), 2000.
2 - Olivier Aubert and Yannick Prié, « Advene : active reading through hypervideo », in ACM Hypertext’05, Salzburg, Austria, Sep 2005, pages 235–244.
3 - Yannick Prié and Vincent Puig, « Nouveaux modes de perception active de films annotés » (New modes of active perception of tagged films), in Colloque “Cinéma, interactivité et société”, Poitiers, Nov. 2008. to be published in 2009.
Partners :
- IRI, Centre Pompidou (coordination);
- Liris, Cnrs;
- Costech/CRED (Cognitive Research and Enaction Design) , University of Technology of Compiègne;
- ÉESI (European School of Visual Arts);
- INTUILAB;
- MOVEA ;
- Sony CSL;