Um estudo experimental streaming-fog-cloud

Abstract

The increase in data production has been constant over the years. The growth of data production genarates a need for innovation in technologies in the field of data processing. Data streaming tools are powerful software that aim to deal with large amounts of data to maintain, manage, and analyze. The biggest problem when thinking about data streaming is the computational cost to perform tasks for a massive amount of data. Part of this difficulty is ralated to a generation of a large amount of data that the system cannot handle. This happens because the available hardware does not have enough computational power to handle all tasks promptly. The most basic solution involves using more hardware until works. There is also a gap in idle resources, considering the hardware spread across the network that has not yet reached its maximum potential. When this hardware is enabled to be used at the “ends” of the network, availability is created to process, store and manage data close to the data generating element, reducing latencies, increasing hardware availability, providing more computational power and etc. This is Fog. The objective of this work is to promote the use of Fog through a study focused on the field of data, verifying issues such as reducing the overload generated by the flow of massive data, discussing the impact of using streaming data processing tools, verifying resource consumption and the computational power of the platform, validate the use of the environment as a viable solution, among others. In this work, two experiments were carried out, the first being fundamentally more theoretical, with the analysis of the Fog layer from the perspective of the use of Streaming Data Processing Tools (SDPT) and the limitation of computational resources provided by the hardware. The second experiment tries to solve a connectivity problem in the educational environment between education students who are allocated in educational centers and the Cloud managed by the university. In this case, the availability of the internet connection is quite precarious and the intervention of using a Fog system can remedy part of the obstacles for distance education to become more efficient and have the best quality of experience for users. The experiments showed encouraging results. In both cases, Fog’s computational solution do what is expected. In the first experiment, when compared to a system with more computational resources, the hardware performed well and presented results beyond expectations regarding the number of elements processed, while the resource consumption values followed a similar behavior. In the second experiment, the Fog solution go very well, solving the main problems related to the quality of user experience, increasing availability, and ensuring data maintenance.