Posts by Collection

portfolio

publications

Generating Animations from Screenplays

Published in *SEM 2019, 2019

Automatically generating animation from natural language text finds application in a number of areas e.g. movie script writing, instructional videos, and public safety. How- ever, translating natural language text into animation is a challenging task. Existing text-to-animation systems can handle only very simple sentences, which limits their applications. In this paper, we develop a text-to-animation system which is capable of handling complex sentences. We achieve this by introducing a text simplification step into the process. Build- ing on an existing animation generation system for screenwriting, we create a robust NLP pipeline to extract information from screen- plays and map them to the system’s knowledge base. We develop a set of linguistic trans- formation rules that simplify complex sentences. Information extracted from the simplified sentences is used to generate a rough storyboard and video depicting the text. Our sentence simplification module outperforms exist- ing systems in terms of BLEU and SARI metrics.We further evaluated our system via a user study: 68% participants believe that our system generates reasonable animation from input screenplays.

Recommended citation: Generating Animations from Screenplays. "Y. Zhang, E. Tsipidi, S. Schriber, M. Kapadia, M. Gross, A. Modi, " Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019) (Minneapolis, Minnesota, USA, June, 2019), pp. 292–307 http://polybahn.github.io/files/publication-genAniFromTxt.pdf

talks

teaching

Natural Language Processing (252-3005-00L)

Master course, Computer Science Department, ETH Zurich, 2019

This course presents an introduction to general topics and techniques used in natural language processing today, primarily focusing on statistical approaches. The course provides an overview of the primary areas of research in language processing as well as a detailed exploration of the models and techniques used both in research and in commercial natural language systems.

Deep Learning (263-3210-00L)

Master course, Computer Science Department, ETH Zurich, 2019

Deep learning is an area within machine learning that deals with algorithms and models that automatically induce multi-level data representations.