Semantic-Aware Representation of Multi-Modal Data for Data Ingress: A Literature Review
Artikel i vetenskaplig tidskrift, 2024

Machine Learning (ML) is continuously permeating a growing amount of application domains. Generative AI such as Large Language Models (LLMs) also sees broad adoption to process multi-modal data such as text, images, audio, and video. While the trend is to use ever-larger datasets for training, managing this data efficiently has become a significant practical challenge in the industry-double as much data is certainly not double as good. Rather the opposite is important since getting an understanding of the inherent quality and diversity of the underlying data lakes is a growing challenge for application-specific ML as well as for fine-tuning foundation models. Furthermore, information retrieval (IR) from expanding data lakes is complicated by the temporal dimension inherent in time-series data which must be considered to determine its semantic value. This study focuses on the different semantic-aware techniques to extract embeddings from mono-modal, multi-modal, and cross-modal data to enhance IR capabilities in a growing data lake. Articles were collected to summarize information about the state-of-the-art techniques focusing on applications of embedding for three different categories of data modalities.

data modality

information retrieval

multi-modal data

literature review

data lake

embedding

Författare

Pierre Lamart

Software Engineering 2

Göteborgs universitet

Yinan Yu

Chalmers, Data- och informationsteknik, Funktionell programmering

Christian Berger

Göteborgs universitet

Software Engineering 2

Publicerad i

Proceedings of the Euromicro Conference on Software Engineering and Advanced Applications, EUROMICRO-SEAA

2640592X (ISSN) 23769521 (eISSN)

Nummer/häfte 2024 s. 122-125

Kategorisering

Ämneskategorier (SSIF 2025)

Språkbehandling och datorlingvistik

Systemvetenskap, informationssystem och informatik

Identifikatorer

DOI

10.1109/SEAA64295.2024.00026

Mer information

Senast uppdaterat

2025-03-10