Data Transformation: An Overview

EasyChair Preprint 4707

Abstract

With the accelerated need for scalability and reliability in todays growing systems, most modern services are shifting towards a microservice-based approach as opposed to a monolithic approach. While the benefits of this new paradigm are undeniable, we also have to be aware of a new set of challenges that microservices bring. To support the need for microservice deployment in scale, data centers and their internal network ( DCN ) are usually the preferred approach. This network can become especially a bottleneck in microservices deployed in a data center, as their communication needs to be as real-time as possible. In order to improve network latency, data transformation capabilities are needed to convert messages back and forth between microservices from their internal formats. This paper discusses the current approaches on data transformation, the bottlenecks that come with current solutions and then explore a new paradigm for Data Transformation, namely Optimus Prime, that introduces a new approach on Data Transformation by using a new transformation schema and a hardware accelerator for optimal optimal cross-microservice communication.

Keyphrases: Hardware Accelerator, Microservices, data transformation

@booklet{EasyChair:4707,
  author    = {Deni Daja},
  title     = {Data Transformation: An Overview},
  howpublished = {EasyChair Preprint 4707},
  year      = {EasyChair, 2020}}