A visual similarity-based method for children’s handwriting recognition
Loading...
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Universidade Federal de Juiz de Fora (UFJF)
DOI
Abstract
Automatic transcription of words handwritten by children can support the process-
ing of large scale literacy assessments. However, the transcription must preserve the exact
character sequence written by the child, including spelling errors that provide information
about literacy development. This dissertation investigates different Handwritten Text
Recognition (HTR) architectures for recognizing handwriting produced by children. To
conduct the experiments, a dataset of words written by Brazilian children during literacy
assessments was constructed and annotated. The comparison of different architectures
showed that some architectures are not well suited to the specific requirements of this
task. The results indicated that the E-TrOCR architecture with a vocabulary defined at
the character level is more suitable for preserving the original response. Another challenge
is the visual ambiguity between letters in children’s handwriting. To address this problem,
a method that modifies the E-TrOCR loss function was proposed. The new loss function
teaches the model to consider visual similarities between characters. To represent these
relationships, a visual similarity matrix was constructed. The values in this matrix were
calculated using Histogram of Oriented Gradients (HOG) descriptors extracted from
synthetic images rendered in handwriting-style fonts. The method was evaluated using
an exclusive words protocol and increased accuracy from 72.57% to 78.06% and reduced
the Character Error Rate (CER) from 7.61% to 5.78%, contributing to more faithful
transcriptions. These results provide evidence that considering visual similarity between
characters during training improves the transcription of words written by children without
relying on dictionaries, linguistic rules, or text corpora.
Description
Keywords
Citation
Endorsement
Review
Supplemented By
Referenced By
Creative Commons license
Except where otherwise noted, this item's license is described as Acesso Aberto

