The image-processing department at the Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institute has developed a system for the conversion of SMS messages into small animations which can be sent to the recipient via MMS. From the input text, speech is synthesized using a TTS (text-to-speech) system. Similarly, lip movements of an arbitrary real or synthetic person are artificially created synchronously to the spoken text. As a result, the chosen character reads the message to the user. Random eye and head movements, emoticons embedded into the text, virtual camera zooms or pans, and voice modifications are implemented in order to make the animation more lively. From the animation, a 3gp video is created that can be sent to mobile phones via MMS. Since the whole application works with exiting infrastructure (with an additional server at the provider), no software has to be downloaded to the mobile device and the animation can be viewed on any mobile phone with video support.
Applications for Text2Video
- SMS to MMS conversion: An arbitrary character reads the text from an SMS
- Individualized MMS commercials
- Intelligent user interfaces: An avatar communicates with the user with user dependent information
- Presentations on web sites
Please check the demo of the system
Features
- Animation of synthetic avatars and real people
- New characters can be created on demand from a single real picture of a person
- Different TTS systems and phoneme alphabets supported (AT&T, Babel, ScanSoft)
- Different languages supported (US/UK, German, French, Spanish, Polish, others on demand)
- Support for emoticons to add facial expressions
- Pitch shift for voice modification
- Camera zoom and pan, eye and head movements for more sophisticated animations
- Different encoding possible 3GP, H263, MPEG-4
- No software required on mobile phone (except for video player)
- Runs on a single Linux server with X and 3D graphics support
The software for the creation of these animations is available at HHI and can be licensed to customers.
Other features and special extensions can be developed on demand.
Examples
Natural (left) and synthetic (middle) character reading a text. Right: Animation on a mobile phone.
Publications
J. Rurainsky and P. Eisert,
Text2Video: A SMS to MMS Conversion,
Proc. 11. Dortmunder Fernsehseminar, Elektronische Medien: Systeme, Technologien, Anwendungen,
Dortmund, Germany, pp. 41-52, September 2005.
J. Rurainsky and P. Eisert
Text2Video: Text-Driven Facial Animation using MPEG-4,
Proc. Visual Computation and Image Processing (VCIP),
Beijing, China, July 2005.
Flyer Text2Video
Contact
Dr. Peter Eisert
Email:
eisert@hhi.fhg.de
Phone: +49 30 31002 614
Fraunhofer Institute for Telecommunications
Einsteinufer 37
D-10587 Berlin
Germany
Start
Organisation