The Magic of Vision: Understanding What Happens in the Process

Licentiatavhandling, 2021

How important is the human vision? Simply speaking, it is central for domain related users to understand a design, a framework, a process, or an application in terms of human-centered cognition. This thesis focuses on facilitating visual comprehension for users working with specific industrial processes characterized by tomography. The thesis illustrates work that was done during the past two years within three application areas: real-time condition monitoring, tomographic image segmentation, and affective colormap design, featuring four research papers of which three published and one under review.

The first paper provides effective deep learning algorithms accompanied by comparative studies to support real-time condition monitoring for a specialized microwave drying process for porous foams being taken place in a confined chamber. The tools provided give its users a capability to gain visually-based insights and understanding for specific processes. We verify that our state-of-the-art deep learning techniques based on infrared (IR) images significantly benefit condition monitoring, providing an increase in fault finding accuracy over conventional methods. Nevertheless, we note that transfer learning and deep residual network techniques do not yield increased performance over normal convolutional neural networks in our case.

After a drying process, there will be some outputted images which are reconstructed by sensor data, such as microwave tomography (MWT) sensor. Hence, how to make users visually judge the success of the process by referring to the outputted MWT images becomes the core task. The second paper proposes an automatic segmentation algorithm named MWTS-KM to visualize the desired low moisture areas of the foam used in the whole process on the MWT images, effectively enhance users'understanding of tomographic image data. We also prove its performance is superior to two other preeminent methods through a comparative study.

To better boost human comprehension among the reconstructed MWT image, a colormap deisgn research based on the same segmentation task as in the second paper is fully elaborated in the third and the fourth papers. A quantitative evaluation implemented in the third paper shows that different colormaps can influence the task accuracy in MWT related analytics, and that schemes autumn, virids, and parula can provide the best performance. As the full extension of the third paper, the fourth paper introduces a systematic crowdsourced study, verifying our prior hypothesis that the colormaps triggering affect in the positiveexciting quadrant in the valence-arousal model are able to facilitate more precise visual comprehension in the context of MWT than the other three quadrants. Interestingly, we also discover the counter-finding that colormaps resulting in affect in the negative-calm quadrant are undesirable. A synthetic colormap design guideline is brought up to benefit domain related users.

In the end, we re-emphasize the importance of making humans beneficial in every context. Also, we start walking down the future path of focusing on humancentered machine learning(HCML), which is an emerging subfield of computer science which combines theexpertise of data-driven ML with the domain knowledge of HCI. This novel interdisciplinary research field is being explored to support developing the real-time industrial decision-support system.