Anotação multimodal para copilotos de produção de tecnologias assistivas: uma proposta para a audiodescrição
Loading...
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Universidade Federal de Juiz de Fora (UFJF)
DOI
Abstract
This research investigates how the multimodal event annotation in FrameNet Brazil
(FN-Br), developed based on Frame Semantics (Fillmore, 1982), can contribute to
the generation of audio description scripts by artificial intelligence systems acting as
copilots for human audio describers. The study is based on the hypothesis that this
type of annotation, by semantically mapping events and their participants in
audiovisual productions, can support the creation of more accurate and contextually
appropriate audio description scripts. The objective is to observe the performance of
AI systems as assistants in the production of audio description scripts, comparing
those based on frame annotation with those that do not use this approach, while also
reflecting on the potential of this tool to support the work of audio describers. To this
end, scenes from episodes 1 and 7 of the travel series Pedro Pelo Mundo, produced
by GNT, were annotated using the FN-Br model, and preliminary audio description
scripts were generated by AI based on these annotations. The resulting scripts were
then compared and analyzed in terms of their alignment with the criteria outlined in
the Guide for Accessible Audiovisual Productions (Naves et al., 2016). The results
indicate that frame annotations act as a modulator of the model’s performance, more
frequently favoring stronger alignment between the generated descriptions and the
visual organization of the scenes, without compromising the quality of the scripts
when no immediate gains are observed. From a qualitative perspective, the
presence of metadata tends to guide descriptions according to the visual dynamics
of the scenes, whereas their absence favors more stative and enumerative
descriptions. These results reinforce the potential of multimodal annotation based on
Frame Semantics as support for the production of audio description scripts by AI
systems.
Description
Citation
Collections
Endorsement
Review
Supplemented By
Referenced By
Creative Commons license
Except where otherwise noted, this item's license is described as Acesso Aberto

