Introduction: How to approach the field of human-machine interfaces (HMI)?
HMIs are a vast multidisciplinary field that is accelerating, in particular due to advances in artificial intelligence. Several posts published on open-organization.com are related to this very dynamic theme:
Case Studies
- Faurecia’s strategic shift towards smart interiors, Presans, open-organization.com (2018) – Presans accompanies Faurecia in its transformation towards a role of car interior architect within a complex industrial ecosystem.
- Synergy Factory HMI
Technology Overviews
- Le cockpit du futur : vers la plateforme intelligente, Presans, open-organization.com (2019) – Albert Meige’s web TV intervention at a round table on the cockpit of the future.
- Six quick insights on the autonomous vehicle, Presans, open-organization.com (2018) – A technology fact sheet on the autonomous vehicle, including the five levels of autonomy.
- Five quick insights on the industrial applications of complexity theory, Presans, open-organization.com (2018) – A synthesis concerning complex systems of some facts considered significant by our Fellows in the Presans Platform activity.
An important concept for understanding the abundant field of HMIs is the concept of cyber-physical system (CPS), which is defined as an integration of calculations (computations), networks and physical processes. Integrated computers and networks monitor and control physical processes, with feedback loops where physical processes affect computations and vice versa (1).
The HMI itself is a system involving interaction between the machine CPS, the (human) operator and the context in which the interaction occurs. Each of these dimensions has a number of elements summarized in the figure below.
The machine includes real-time data acquisition, data processing, artificial intelligence, models, and is itself made up of a hierarchy of interconnected systems.
Humans have their own specificities and intra-individual and inter-individual performances, from a physical, physiological, psychological point of view, including the mobilization of cognitive and training capacities in the face of the complexity of the task at hand.
The human-machine interface involves a number of (possible) channels, including (but not limited to) vision and imaging, voice and sound, touch and haptics, 2D-3D immersion and the possible development of (direct) brain-machine interactions.
The channels involve both man-to-man and man-to-machine directional exchanges. They aim to provide information, to implement decisions, to act and to verify the consequences of the action for the best recovery on both the human and machine sides.
A first approximation of the field of HMIs can thus be obtained by distinguishing three areas:
- The set of technologies related to HMIs, in which we can include the exploitation of interaction channels.
- The set of applications in which HMIs are used.
- The field of human-CPS interactions, which mobilizes in particular all that concerns the knowledge of the human system.
In what follows we propose to sketch the contours of these three domains.
First point: HMI technologies are an explosive growth area
Everyone can observe that tactile interfaces have revolutionized the most commonly used CPSs over the last ten years or so. But the revolution continues and new technological worlds are emerging. The channels of human-CPS interaction are developing both in the sense of perfecting already well-established modalities, exploring new channels, and efficiently combining these different channels. In concrete terms, visual, auditory or haptic modalities are augmented by channels exploiting hand gestures, voice, or eye movements, or even brain activity in the case of neural interfaces.
Added to this are advances in artificial intelligence technology, which are increasingly disrupting the division of roles between humans and machines. Even if one accepts the general principle that the combination between human and machine prevails over the machine alone, this leaves open the question of the precise articulation between operator and CPS.
During Dystopia 2019, the subject of HMIs was addressed in particular through the use of video games, during a round table discussion in which Daphne Bavelier, Ordinary Professor, University of Geneva; Mr Phi, Youtuber; Jean-Benoît Dunckel, Musician; and Frédéric Charon, Technology Strategy Manager, Faurecia, who led these exchanges, took part.
Second point: identifying the application context and the limits of HMIs
The application context is the uses and the populations that correspond to these uses. It is worth noting the wide variation in performance within the entire population within the same HMI system. These inter-individual variations constitute an essential factor to be taken into account in the design of HMI.
Indeed, MCHs affect groups as dissimilar as medical personnel, for example in the context of surgery, and the population of car drivers, which itself includes senior citizens, varying levels of education or training, etc. The application context factor massively drives the design constraints of an HMI system.
Similarly, the technologies used have limitations. Concretely, the fact that the range of a sensor for visualising the automotive environment is, for example, 100 m must lead to the consideration of accident scenarios involving phenomena that can be integrated by the human operator but not by the CPS. In order to carry out this task, the use of simulation is unavoidable.
Third point: measuring the interaction within the human-machine team
The question of the limits of a CPS immediately raises the question of the conditions under which it can be taken over by the human operator. However, the latter is also subject to intra-individual variations, due to the variability of his behaviour as a function of psychological and physiological factors.
It is at this level that very concrete questions of performance measurement arise (how to measure a response time?), in order to determine intervention frameworks that meet the objective of safely taking over a CPS in the event of failure of its autopilot system.
Having measurement methods is a key issue for HMIs, in order to improve the relevance and performance of the human-machine team. The quality of decisions is largely due to a good distribution of roles between the operator and the intelligence embedded in a CPS.
Conclusion
In fact, it is not self-evident that humans always bring an increase in the performance of a CPS: it is a nice idea that is meant to be optimistic about the future of humans, but its proof can only be provided through systems proposing concrete articulations between the two members of a HMI team.
This article has benefited from the assistance of Bernard Favre, Fellow Presans.
(1) The concept of SPC plays a central role in the definition of the Industry 4.0 agenda, a genealogy of which we sketched in an earlier post.
