The artificial intelligence advantage: Supercharging exploratory data analysis
Review article, 2024

Explorative data analysis (EDA) is a critical step in scientific projects, aiming to uncover valuable insights and patterns within data. Traditionally, EDA involves manual inspection, visualization, and various statistical methods. The advent of artificial intelligence (AI) and machine learning (ML) has the potential to improve EDA, offering more sophisticated approaches that enhance its efficacy. This review explores how AI and ML algorithms can improve feature engineering and selection during EDA, leading to more robust predictive models and data-driven decisions. Tree-based models, regularized regression, and clustering algorithms were identified as key techniques. These methods automate feature importance ranking, handle complex interactions, perform feature selection, reveal hidden groupings, and detect anomalies. Real-world applications include risk prediction in total hip arthroplasty and subgroup identification in scoliosis patients. Recent advances in explainable AI and EDA automation show potential for further improvement. The integration of AI and ML into EDA accelerates tasks and uncovers sophisticated insights. However, effective utilization requires a deep understanding of the algorithms, their assumptions, and limitations, along with domain knowledge for proper interpretation. As data continues to grow, AI will play an increasingly pivotal role in EDA when combined with human expertise, driving more informed, data-driven decision-making across various scientific domains. Level of Evidence: Level V - Expert opinion.

exploratory data analysis

orthopedic research

machine learning

artificial intelligence

feature engineering

Author

Felix C. Oettl

Hospital for Special Surgery - New York

University of Zürich

Jacob F. Oeding

Mayo Clinic Alix School of Medicine

Sahlgrenska University Hospital

University of Gothenburg

Robert Feldt

Chalmers, Computer Science and Engineering (Chalmers), Software Engineering (Chalmers)

Christophe Ley

University of Luxembourg

Michael T. Hirschmann

Canton Hospital Basel-Land

Kristian Samuelsson

Sahlgrenska University Hospital

University of Gothenburg

Knee Surgery, Sports Traumatology, Arthroscopy

0942-2056 (ISSN) 1433-7347 (eISSN)

Vol. In Press

Subject Categories

Computer Science

DOI

10.1002/ksa.12389

PubMed

39082872

More information

Latest update

8/6/2024 1