EuroMov DHM
SemTaxM
Key words
multimodal representation learning, semantics, taxonomy, parametric representations of movement
Semantics & Taxonomy of Movement
Main objective
The theme PIAS involves research belonging to three domains, movement sciences, health sciences and computer (images) sciences. All these domains interact together to focus on a main objective: uncover the laws governing human perception in moving agents (Perception in Action) and human-environment synchronization in general.
The figure above illustrates the interdisciplinary architecture of SemTaxM following the scientific orientations highlighted beside and the interface with the three main themes of the research unit.
Scientific orientations
SemTaxM aims at identifying taxonomic classifications of movement [PIAS] and to defining a theory of the semantics carried by movement and models of semantics grounded in specific contexts. SemTaxM will jointly exploit knowledge representation techniques and formalisms from the semantic web with unsupervised multimodal representation learning techniques. The former will help us design ontologies and knowledge bases [Factory] pertaining to movement taxonomy [PIAS], semantics (reasoning, semantic summarization) and parametric representations, while the latter will allow us to build underlying mathematical representations that capture multiscale invariants of movement [LAC], its complexity and latent properties by fusing and decomposing multimodal observations of movement: parametric models, taxonomic classification, [MIB] brain activity acquired through non-invasive methods (EEG, NIRS) that exhibit neural signatures of movement and consciousness (Motor Imagery and ERPs associated with conceptual and semantic processing).
The more fundamental research direction in SemTaxM will be grounded in and applied to [LAC] rehabilitation medicine and to the study of [PIAS] movement synchronization and system and software engineering of [MIB]
Interdisciplinary transversal scientific questions
- How to define taxonomies of movement with maximal predictive power and generalizability?
- What linguistic formalisms and knowledge representation formalisms for movement semantics and taxonomy?
- – Characterizing movement lexicon, syntax semantics and identifying similarities and differences with existing linguistic formalisms to propose a linguistic formalism of movement.
- – Building interoperable knowledge graphs and ontologies (Linked Open Data) to represent data about movement (taxonomy, semantics, parameters, etc.) that can be linked to relevant linguistic and application domain resources (linguistics, rehabilitation medicine) found on the LOD cloud to enable reasoning and inference about movement.
- How to identify multiple-scale invariants from multiple acquisition modalities and use them in unison with knowledge representations for validation (predictive power), reasoning about movement, classification of movement (contextualized intent, pragmatic interpretation).