Avaliações em larga escala em ciências humanas no Centro de Políticas Públicas e Avaliação da Educação da Universidade Federal de Juiz de Fora (CAEd/UFJF): a construção de um guia de orientação do trabalho de analistas e auxiliares de instrumentos de avaliação em questão

Abstract

This thesis, developed within the scope of Education’s Management and Evaluation Professional Master’s program (PPGP) of the Center of Public Policies and evaluation of Education of the Federal University of Juiz de Fora (CAEd/UFJF), aims to identify and understand how CAEd supports the work of Analysts and Auxiliaries of Evaluation Instruments in the elaboration of items and in the assembly of tests for the field of Human Sciences.Therefore, we assume that the absence of specific information for this field in the Item Preparation Guide (CAED, 2014) interferes in the process of preparing and reviewing items and, consequently, in the quality of the Item Bank and the tests applied by CAEd, because there is often no understanding and/or knowledge about specific guidelines for this field. In order to understand how CAEd acts in support of the work of these professionals, we have as theoretical presuppositions the management of knowledge and information and Psychometrics. As a methodology, we adopted the case study, based on a qualitative approach systematized in a management case, which allows us to critically analyze the context in which the research is inserted. In order to develop this management case, we defined, as methodological procedures for data collection, questionnaire, interview and the expert panel, in order to listen to and understand the historical trajectory of CAEd in the scenario of large-scale evaluations, especially in the field of Human Sciences, as well as the view of Analysts and Assistants on the formative process, when they enter the institution, where they are prepared to elaborate items, and on the document that supports their actions. Such statements were interpreted on the basis of content analysis, which also served to analyze the materials that support this formative process, the CAEd (2014) Item Elaboration Guide, isolated and in comparison with the Inep’s Guide for Elaboration and Item Review (2010). The mapping of this information allowed to identify limits and possibilities of CAEd's performance in the development of large-scale evaluation in Human Sciences and to systematize what would be the specific knowledge needed for the performance of these professionals. This systematization allowed us to structure a proposal for a Guide for Directions on the work of Analysts and Auxiliaries in this field of knowledge, which was analyzed, through a Panel of Experts, by a group of CAEd professionals, responsible for different stages of the instrument evaluation construction. In this way, we understand that, despite the efforts already undertaken by CAEd, there are still gaps in its role in systematizing guidelines for Analysts and Auxiliaries work, due to the lack of information on the Human Sciences in the Guide to Elaboration of Items (CAED, 2014), or due to the lack of focus in this field of knowledge throughout the training process when these professionals enter the institution. Based on this understanding, we propose an Educational Action Plan (PAE) that aims to present ways that can contribute to CAEd’s advance regarding the support to the institution on the work of Analysts and Auxiliaries in the construction of evaluation instruments in the field of Human Sciences. Therefore, we propose actions that aim to provoke the collective and systematic debate about the specificities involving the construction of this type of instrument for this field of knowledge. These actions concern the reformulation of the Reference Matrix and the CAEd Item Preparation Guide (2014) - which, in this case, could be based on the proposed Guide for Directions we have structured - as well as the promotion of collective moments of continuous training for these professionals through the sharing of ideas in regular meetings, thus developing management actions guided in the management of knowledge and information.