Observatório do consumidor: uma ferramenta de mineração de dados de redes sociais para avaliação de tendências de consumo de derivados lácteos no Brasil

Abstract

Technological advancement has profoundly transformed how data is collected, analyzed, and applied across various fields of knowledge. In market research, social media has emerged as a rich and dynamic source of information, continuously fed by millions of users worldwide. The use of social media analysis in scientific studies has grown significantly, driven by the vast volume of available data and its potential to reveal patterns and behavior trends. In this context, this study proposes the development of Observatório do Consumidor, an innovative tool for market research in the dairy sector in Brazil, utilizing social media data. The goal is to offer an efficient, accessible, and representative alternative to traditional methods, which, although scientifically validated, face challenges such as geographic restrictions, high costs, and long periods of application and analysis. To support this proposal, we conducted a systematic literature review to identify the main methods, analyses, and techniques applicable to the development of the tool. Based on these findings and the principles of Business Intelligence, Observatório do Consumidor was created, exploring data from X (formerly Twitter), YouTube, and Google Trends to extract unprecedented information about dairy consumption in Brazil and its market trends. The tool employs natural language processing techniques, machine learning, and database methods to perform in-depth analyses of various aspects of consumption. Among its key features, the tool includes consumer profile analysis, identifying characteristics such as gender, generation, and income; temporal analysis, which allows an understanding of the evolution of consumption over time; and identification of consumption associations, revealing which dairy and non-dairy products mentions together and highlights the most frequent combinations. Additionally, the tool incorporates geographic analysis to map the spatial distribution of consumption. It applies behavioral and qualitative analysis to examine consumer sentiments and product descriptions through adjectives and experience reports, providing a detailed view of how consumption occurs. This integrated approach enables a comprehensive understanding of consumer behavior in the dairy market, providing strategic insights for industry stakeholders and researchers. The results demonstrate the tool’s potential to generate valuable and unprecedented information about the sector, contributing to decision-making and enhancing market strategies.