Přejít k hlavnímu obsahu


Conference Paper (international conference)

Human Computer Interface Based on Tongue and Lips Movements and its Application for Speech Therapy System

Bílková Zuzana, Novozámský Adam, Bartoš Michal, Dominec Adam, Greško Šimon, Zitová Barbara, Paroubková Markéta, Flusser Jan

: Electronic Imaging, Visualization and Data Analysis 2020

: IS&T International Symposium on Electronic Imaging 2020, (Burlingame, California, US, 20200126)

: TJ01000181, GA TA ČR, SVV–2017–260452, GA MŠk, AP1701, AV ČR

: speech therapy, deep learning, tongue detection, lips detection

: 10.2352/ISSN.2470-1173.2020.1.VDA-389

: http://library.utia.cas.cz/separaty/2020/ZOI/zitova-0524713.pdf

(eng): The novel human computer interface is introduced, based ontongue and lips movements and using video data from a commercially available camera. The size and direction of the movementsare extracted and can be used for setting cursor actions or to otherrelevant activities. The movement detection is based on convolu-tional neural networks. The applicability of the proposed solutionis shown on the ASSISLT system, aimed to support speechtherapy for adults and children with inborn and acquired motorspeech disorders. The system focuses on individual treatment us-ing exercises that improve tongue motion and thus articulation.The system offers an adjustable set of exercises which proper per-formance is motivated using augmented reality. Automatic evaluation of the performance of therapeutic movements allows thetherapist to objectively follow the progress of the treatment.

: JC

: 10201