Embodied Semiotics

Mission

X00：General Group

Establishment of an academic community and network for the establishment of body semiotics

The General Group provides a so-called "Tamariba" (hangout in Japanese, a place for information sharing and discussion). Specifically, the group holds open online meetings regularly to share the awareness of issues of interest to both humanities and science/engineering fields. Through these activities, the group aims to deepen mutual understanding of methods and trends in research fields different from their own and to accelerate the promotion of research fields.

> Details

A01：GESTURE

Construction of a multimodal semiotics for gestures associated with spoken conversation

Gestures play an important role in daily communication. However, it is not easy to identify the "meaning" of gestures. To address this issue, we are constructing a "multimodal semiotics" based on the method of "multimodal sequential analysis". We aim to develop a "next-generation corpus of gesture" and share it with academic communities．

> Details

A02：SIGN

Building a multimodal semiotics for icons and indices in sign languages.

What kind of language is sign language? The linguistic theory of sign languages is currently under discussion all over the world. Our goal is to establish a theory of multimodal semiotics unique to sign languages, to develop a "next-generation corpus of sign languages", and to share this corpus with academic communities.

> Details

A03：AI

Modality Crossing Based on Latent Structural Understanding in Multimodal Dialogue Translation

Can AI understand human gestures and sign languages? This is not such a simple question yet. We are developing a deep learning platform for advanced recognition of multimodal time-series data contained in human gestures and sign languages.

> Details

B01：DATA-INTEGRATION

Modality crossing based on the construction of a next-generation corpus of humanities and science fusion using information processing technology

We are a crucial research team to promote mutual understanding among the humanities, science, and engineering fields. Specifically, we design and build next-generation corpora that researchers can use and participate in, in collaboration with other research groups. By developing the research environment and data, we will stimulate interest in body symbols in various research fields.

> Details