Department of Psychosocial Science, University of Bergen, Christiesgt, 12, Bergen 5015, NorwayNorway
Department of Anaesthesia and Intensive Care, Haukeland University Hospital, Haukelandsveien 22, Bergen 5021, Norway; Department of Clinical Medicine, University of Bergen, Haukeland University Hospital, Jonas Lies vei 87, Bergen 5021, NorwayNorway
Swedish Defense University, Drottning Kristinas väg 37, Stockholm 114 28, Sweden; Centre for Crisis Psychology, University of Bergen, Møllendalsbakken 9, Bergen 5009, NorwayNorway
Department of Anaesthesia and Intensive Care, Haukeland University Hospital, Haukelandsveien 22, Bergen 5021, NorwayNorway
Centre for Crisis Psychology, University of Bergen, Møllendalsbakken 9, Bergen 5009, NorwayNorway
Also available as PDF (figures and tables appear in the PDF).
Background Empirical research on shared mental models (SMM) in virtual environments are almost non-existent. Pre-hospital emergencies presents an opportunity to examine team processes in virtual teams because the dispatcher is geographically separated from the ambulance and at the same time plays a significant role in coordinating, organizing, obtaining, evaluating, and conveying relevant information to the deployed ambulance. The present study aimed at mapping team behavior and cognition in critical real-life emergency medical missions based on the concept of SMM. Methods By investigating the frequencies of coordinating mechanisms and team competencies based on voice recordings from real-life missions, differences in team behavior between low and high-complexity missions were investigated. Results Lower frequencies of team competencies and coordinating mechanisms were found in high compared to low-complexity missions. The results showed a different profile in communication between high and low-complexity missions with more frequent use of both coordinating mechanisms and team competencies in low-complexity missions. Furthermore, the profiles revealed that SMM and closed loop communication were the only coordinating mechanism used, and leadership and team orientation were the only competencies exercised. Conclusion It was concluded that the lack of visual input of a team member during team interaction could lead to team process loss due to a breakdown of the team into sub-units. Potential improvement of team behavior is discussed within the SMM framework.
Keywords: Virtual medical first responder teams; Complexity; Shared mental models; Team processes; Coordinating mechanisms
Empirical research on shared mental models (SMM) in virtual environments are almost non-existent. Pre-hospital emergencies presents an opportunity to examine team processes in virtual teams because the dispatcher is geographically separated from the ambulance and at the same time plays a significant role in coordinating, organizing, obtaining, evaluating, and conveying relevant information to the deployed ambulance. The present study aimed at mapping team behavior and cognition in critical real-life emergency medical missions based on the concept of SMM.
By investigating the frequencies of coordinating mechanisms and team competencies based on voice recordings from real-life missions, differences in team behavior between low and high-complexity missions were investigated.
Lower frequencies of team competencies and coordinating mechanisms were found in high compared to low-complexity missions. The results showed a different profile in communication between high and low-complexity missions with more frequent use of both coordinating mechanisms and team competencies in low-complexity missions. Furthermore, the profiles revealed that SMM and closed loop communication were the only coordinating mechanism used, and leadership and team orientation were the only competencies exercised.
It was concluded that the lack of visual input of a team member during team interaction could lead to team process loss due to a breakdown of the team into sub-units. Potential improvement of
Virtual medical first responder teams; Complexity; Shared mental models; Team processes; Coordinating mechanisms.
he combination of time pressure and coordinated action is critical in emergency medical situations. Optimal interaction T in and between multidisciplinary teams is therefore crucial for pa- 1,2 tient safety and survival,1,2 and communication failure has been identified as a key contributing factor to adverse events in surgical teams.1 Following from this, we assume that virtual teams may present a heightened risk of communication failure and disruption of team interactions when team members are geographically dis-
cc Copyright 2022 by Johnsen BH. This is an open-access article distributed under Creative Commons Attribution 4.0 International License (CC BY 4.0), which team behavior is discussed within the SMM framework.
persed and, in most cases, also deprived of visual feedback. Not being able to maintain visual contact will increase the need to rely on communication technology in order to coordinate and communicate during emergency medical operations. Although obvious differences exist in the dynamics between face-to-face and virtual team interactions, there are also essential similarities in the behavior needed to maintain effective teamwork. This includes team competencies, like sharing of information, generating and building on other team members’ ideas, decision-making, and identification and evaluation of alternatives.3 The increased potential for
information exchange with various, but also new technological aids could facilitate, impair, or have a neutral effect on team effective- 4-6 ness.
Leasure et al7 suggested the Big Five model as a promising approach for identifying areas for improving high-performing multidisciplinary medical health care teams. The model, initially developed for military expert teams, has also been applied in studies of heath care professionals.7 The “Big Five” model was originally developed by Salas et al8 after reviewing 138 teamwork models. From empirically tested relationships between team behavior and performance as well as similarities in descriptions of teamwork, they proposed five core teamwork competencies. The competencies include leadership, team orientation, mutual performance monitoring, backup behavior, and adaptability, all of which are supposedly essential for promoting team efficiency. In addition, three coordinating mechanisms were identified. The three coordinating mechanisms are shared mental models (SMM), closed loop communication (CLC), and mutual trust. These mechanisms are proposed to serve as decisive tools to uphold and inform the “Big Five” teamwork processes. However, these three underlying and theoretically based coordinating mechanisms have to a lesser degree been empirically tested. While there is an abundance of research on the “Big Five” competencies, there is to our knowledge only one study that investigates the relative importance of the three coordinating mechanisms and their impact on team effectiveness.9 In a simulated police operation study, only the mechanisms of CLC and SMM predicted performance indicators, with SMM showing the highest predictive value.9
A meta-analysis reported a strong relationship between team cognitive structures, team processes, and team performance.10 Both the similarity and accuracy of the organized structures (i.e., shared mental models) are strongly related to effective performance due to positive team interaction processes like coordination, communication, and cooperation.11
The “Big Five” model has received considerable 12-15 attention by health practitioners. Improvement in the quality of teamwork and the effectiveness of multidisciplinary teams are claimed to have more positive influence on patient outcomes than other improvement strategies. Following this, several tools to diagnose team deficits or training needs have been developed based on the “Big Five” theory.
In a study of geographically dispersed pre-hospital medical teams, Johnsen et al19 studied teams of emergency medical communication centre (EMCC) operators and ambulance paramedics, using the “Big-Five” model. The study encompassed both the five competencies, as well as the three coordinating mechanisms. Path analyses showed that the SMM mechanism was positively associated with team effectiveness measured as performance satisfaction and situational awareness, and negatively related to mission complexity.
Increased complexity of a situation also results in an increased flow of information, which in turn could cause a variety in information perceived by each team member and hence a diversity
in information sharing within the team. Thus, the complexity of a situation will call for additional ways to organize knowledge structures. Thus, high-complexity situations may present an increased risk of misrepresentation of the actual situation between team members. Accuracy and sharedness of mental models are claimed to moderate the relation between complexity and teamwork behaviors.2 Less sharing of mental representations has also been reported as a consequence of increased complexity of the 7 21 situation.21 In the medical domain, a complex situation is reported to change the nature of medical care from technical procedures to advocacy, resulting from increased communication characterized by negotiations or collective enactments within the team or within communication with patient and families.22
The relationship between mental models and team performance has usually been perceived and thus studied because of model similarity and accuracy. However, Uitdewilligen et al11 suggested model complexity as an additional and more important factor. The rationale being that mental model accuracy and similarity could be obtained based on irrelevant and/or a limited number of task characteristics. Thus, a shared and inaccurate, or an accurate but over-simplified model, could both be guiding team behavior.
Comparing the communication between the EMCC and one or more dispatched ambulance units is one way to understand the complexity of mental models in the pre-hospital medical domain. Mental models contribute to the organization of key domain information elements.23 Model complexity refers to the amount and relation of the significant information elements in the model. When the number of dispatched ambulance units increases, a more complex mental model has to be generated and maintained because the EMCC has to follow and organize a larger amount of task-relevant knowledge representations. Thus, the present study mapped the use of coordinating mechanisms, and the “Big Five” team competencies, comparing high and low-complexity emergency situations. Because the dispatch of more ambulance units involves an increased number of critical elements in the environment, with an increased need for processing, integrating, and organizing key elements, consequently the mental models would be more complex compared to a situation involving only 16,17 one ambulance unit. This is in line with the definition of “requisite complexity”, characterized as “the matching of an individual, team or 25,26 organization’s level of complexity to the demands of its environment”.
Taken together, mental models are domain specific, and to our knowledge no empirical studies have been conducted on virtual (i.e., geographically separated) teams using the “Big Five” approach for investigating the use of coordinating mechanisms and the five competencies in high versus low-complexity situations. It could be argued that the need for generating, maintaining, and updating shared knowledge structures (i.e., use of coordinating mechanisms), as well as coordination, adaptation, monitoring, support behavior and team orientation (i.e., the use of competencies) would be higher in high-complexity compared to low-complexity missions. On the other hand, it could be argued that the use of coordinating mechanisms and team competencies are dependent on the level of complexity of the situation. Low-
complexity missions might have a higher degree of coordinating mechanisms because they are routine missions and the distribution of information to all team members is more frequently used, resting on well-trained procedures. Highly complex missions could be more dynamic, demanding higher frequencies of displayed team competencies to process and integrate a variety of cues for sensemaking and adaptation to the situation28 and to project a likely status of the situation into the near future.
Following from this, we hypothesized that higher frequencies of both coordinating mechanisms and team competencies in high-complexity compared to low-complexity situations would be observed. Finally, an interaction of team behavior (coordinating mechanisms vs. competencies) by degree (high vs. low) of complexity was expected. This was predicted to occur by more frequent use of coordinating mechanisms in the low-complexity mission and more use of team competencies in the high-complexity mission.
Data were derived from a total of 107 real-life voice recordings of emergency calls between one EMCC and ambulances, resulting in dispatch of one or more ambulance units. The audio logs included verbal communication within teams consisting of one EMCC operator and a minimum of two ambulance crew/paramedics. Because the team members are inter-dependent by coordinating their activities towards a common goal (i.e., patients’ health and well-being) and possess complementary skills, they match the definition of a team.17 Five recordings were excluded due to poor sound quality. No data existed on the number of units or persons involved in the last cluster of recordings (two or more ambulance units). Thus, the study included a minimum of 306 subjects (each mission involving at least one EMCC operator and two ambulance personnel). The role of the EMCC operators is to dispatch and coordinate the pre-hospital ambulance missions, thus representing the role of a team leader. Six percent of the emergency dispatchers reported a professional background both as a paramedic and a nurse, and 94% as paramedics. The level of experience ranged from less than one to more than ten-years. The study was performed in accordance with national guidelines and regulations and approved by Haukeland University Hospital, Unit of Emergency Medicine, Division of Surgical Services.
Variables of team behavior were measured as the frequency of verbal communication identified from analyses of the voice recordings, based on Salas et al. Frequencies were divided by the duration of the mission resulting in an index representing communication per minute.
Team mechanisms: Shared mental models were measured as the global anticipation ratio (GAR) and included the frequency of implicitly shared information (“push of information”) divided by the frequency of requests (“pull of information”). Increased GAR scores
indicate higher-levels of shared mental models because implicit sharing of needed information requires a shared understanding of the situation, roles, potential, and limits of the equipment, as well as the task at hand.30 CLC was measured as the frequency of completed communication loops (sending and confirmation).
Team processes: Indicators of team leadership were verbal statements including organizing, dedicating roles, coordinating, establishing expectations, and feedback, as well as seeking and evaluating relevant information. Monitoring was recorded as the identification of errors made by other team members, including feedback to facilitate self-corrections. Supportive behavior was defined as verbal expressions related to recognizing unequal workloads and changes in workload within the team, often represented by help with performing tasks from other team members. Team adaptation was determined by statements indicating changes in the situation, verbally suggesting actions for dealing with the altered situation, or identifying opportunities to improve the action plan. Statements regarding team adaptation were often related to the status of the patient or environmental obstacles to executing their plan of action. Team orientation was measured as the frequencies of statements considering input and suggestions from other team members, including building on other members’ ideas.
Voice recordings of real-life missions between EMCC operators, ambulances, and other units (i.e., helicopters) are routinely recorded and stored. The stored communication files were retrieved from the database, enabling manual scoring of the behavioral markers by playing the sound file on a computer. The scoring was based on pre-defined categories of the behavioral markers and performed by an experienced research assistant trained in the method. Based on the unique identification number of the mission, the anonymous data from the EMCC operators were combined with an anonymized version of the stored communication (i.e., the names of the EMCC operators, ambulance personnel, and patients were omitted from the analysis). The emergency teams (e.g., EMCC operator and ambulance workers) were randomly composed by using standard protocol for handling emergency calls in the EMCC. Thus, assignment to teams were based on availability of EMCC-operator and ambulance unit, distance to the emergency and urgency of need for care.
Forty-one of the 102 included missions involved the dispatch of one ambulance, while 60 missions included the dispatch of two or more units (ambulances, ambulance and rescue helicopters, or ambulance and ambulance boats). Situations consisting of communication between the EMCC operator, and one ambulance unit were classified as “low complexity”, while situations where the EMCC operator was coordinating two or more units were classified as “high complexity” situations.
By separating the data into low and high-complexity missions, the effect of mission complexity on team behavior could be investigated.
The use of coordinating mechanisms and team processes in low versus high-complexity missions was examined by means of a profile analysis. Profile analysis is an application of multivariate analysis of analysis of variance (ANOVA) that allows simultaneous comparison of the two situations for all of the dependent variables (mechanisms and processes). Of primary interest were the tests of parallelism and overall differences between the missions. The former is a test of whether the two situations have parallel profiles, e.g., whether low and high-complexity situations have the same frequency patterns on the various coordinating mechanisms and team processes. This is akin to a test of an interaction in a univariate ANOVA. The test of an overall difference between the situations is equivalent to a main effect of the group, addressing the question of whether one of the situations on average scored higher on the collected set of dependent variables.
Statistically significant results for the test of parallelism and/or overall difference were followed by simple contrasts to pinpoint the source of variability. Specifically, a statistically significant result for the test of parallelisms was followed by a planned contrast comparing low-complexity and high-complexity situations on coordinating mechanisms combined versus team processes, combined.
All statistical analyses were performed using Stata 17.0.
The profiles for low and high-complexity situations are presented
Figure 1. Profiles of Team Processes and Mechanisms for Low-Complexity (one unit) and High-Complexity (two or more units) Situations SMM=Shared mental models; CLL=Closed loop communication
a. For an interaction, the Scheffé adjustment is Fs=(1-p) (k-1) F(p-1)(k-1), k(p-1)(n-1), where p is the number of repeated measures, k is the number of groups, and n is the number of subjects in each group.
in Figure 1. Using Wilks’ λ, the profiles deviated statistically significantly from parallelism, F(6, 95)=10.23, p<0.001, with partial η2=0.39 and a 95% confidence interval (CI) of 0.21 to 0.48. Further, the test for overall levels showed that the two situations differed statistically significantly when considering all processes and mechanism in combination, F(1, 100)=55.07, p<0.001, with partial η2=0.36, 95% CI=0.21-0.48. It is evident from the profiles shown in Figure 1 that low-complexity situations elicited a higher proportion of demonstrated team behaviors when averaged over all behaviors.
To evaluate the deviation from parallelism of the profiles, an interaction contrast was created that contrasted all mechanisms with all processes. A Scheffé adjustment (Fs) to the critical F-value a was applied to hold down the family-wise error rate.a The resulting critical F from the test of the contrast was 29.19 (p<0.001), exceeding the critical Fs of 12.68. The results therefore show that there is an interaction between low and high-complexity situations in the use of coordinating mechanisms and team processes. This interaction is illustrated in Figure 2 showing the mean proportion of mechanisms and processes used in low and high-complexity situations. Although low-complexity situations elicited a higher average percentage of both team behaviors, the difference was clearly larger for coordinating mechanisms than for team processes.
The results revealed different profiles of team behavior in high versus low-complexity missions where the low-complexity situations elicited a higher mean proportion of both coordinating mechanisms and team competencies. This difference was larger
Figure 2. Percentage of Use of Team Processes (Averaged Across Five Process Behaviors) and Team Mechanisms (Averaged Across Two Mechanisms) for Low-Complexity (One for the coordinating mechanisms than for the team competencies.
The profiles showed that SMM and CLC, as well as the team competencies of team orientation and leadership, were the only team behaviors used in both conditions.
Contrary to our hypothesis, the analyses revealed lower frequencies of team behavior during high-complexity missions compared to low-complexity situations. This indicates less information exchange between the EMCC operator and the ambulance crew during high-complexity situations. This could be caused by low-complexity missions being well-known and primarily based on well-trained procedures and a mental surplus resulting in higher frequencies of information sharing. However, the flipside of this could lead to divergent mental representations of the mission within the team caused by process loss due to the virtuality of the team.2 A complicating factor was that one team member (the EMCC operator) was geographically dispersed from the other two team members (the ambulance personnel), potentially resulting in a team breakdown by separating the team into sub-units. During missions including two or more units, direct communication between the ambulance personnel, excluding the EMCC operator, could occur. Although team orientation was the most frequently used team competency, one could argue that the investigated model did not consider team members’ perceptions of team membership. Thus, a lack of identity as a team member could direct the attention towards communication within the subunits, and thus a neglect of communication within the whole team. Because mental models are an emergent state guided by team interaction, the consequence could be lower accuracy and sharedness of the mental models representing vital aspects of
Unit) and High-Complexity (Two Or More Units) Situations the missions.11 The difference between high versus low-complexity missions was greater for coordinating mechanisms compared to team competencies, indicating a vulnerability, especially for coordinating mechanisms. This is in line with Sætrevik21 who reported a reduction in shared cognition within a team caused by task complexity as well as a decrease in shared mental models when psychophysiological indicators of executive functions decreased (i.e., heart rate variability). In the Sæthrevik21 study, the teams investigated consisted of operational staffs involved in handling a simulated fire on an oilrig characterized by face-to-face interaction.
Looking at the profiles (Figure 1), leadership and team orientation were used in both conditions. Leadership was shown in the form of coordinating, organizing, seeking, and evaluating relevant information or other important aspects of ambulance missions. Furthermore, team orientation – measured as suggestions and elaborations of the suggestions – were the most frequently used team behaviors. Both SMM and CLC were used as coordinating mechanisms. SMM was the most frequently used mechanism in both conditions (Figure 1). As also can be seen in Figure 1, the frequency of using SMM was five to six times the use of CLC. This indicates that information sharing without an explicit request was the most salient coordinating mechanism. This is in line with previous studies on pre-hospital emergency teams using the Salas model,19 where shared mental models were the only predictor of performance.
A surprising finding was that only the competencies of leadership and team orientation were used in both low and highcomplexity missions. This excludes team processes like mutual
performance monitoring, back-up behavior and team adaptation. This is in accordance with a review of shared mental models in virtual teams. Schmidtke et al stated that “As virtualness increases, a team’s shared mental models become more complex; this limits the effectiveness of particular teamwork behaviors: mutual performance monitoring, backup behavior and adaptation”. This was due to the technology used, environmental and interpersonal factors, and their interactions.
The team model investigated in the present study has been associated with performance in a variety of contexts, including prehospital and acute medical settings as well as other operational environments.9 Assuming the model’s validity in the present setting, increased use of all team competencies and coordinating mechanisms should increase team performance. This would be of particular interest in highly complex missions. DeChurch et al10 proposed that factors such as team leadership, shared experience, and training in addition to workplace design may enhance team cognitions. The present study gives direction towards an increased focus on team training in the development of EMCC operators and ambulance crew. Education and training should focus on enhancing the team competencies of monitoring, supporting, and adapting as well as the coordinating mechanism of CLC. Several training techniques have been proposed, and various types of cross-training has been presented by Blickensderfer et al32 as well as guided team self-correction training.33
One obvious limitation of this study is the measurement of communication only to and from the EMCC operator due to the content of the stored voice recordings. Communication between the ambulance crew, when more than one unit was dispatched, could have shed further light on team behavior and cognition. However, the focus on the present study was the communication within a virtual emergency team including the EMCC operators and the ambulance personnel.
Empirical research on teams or SMM in the virtual environment are almost non-existent.2 The present study contributes to closing this gap by providing the first empirically anchored mapping of team behavior and cognition used in critical real-life medical emergency missions conducted by virtual teams. The findings of lower frequencies of expressed team behavior and shared cognitions in high compared to low-complexity missions were surprising. We claim that the lack of visual input of a team member leads to process loss due to a breakdown of the team into sub-units. This process loss sems to increase in high-complexity situations. This taken together with the results revealing the absence of the team competencies of mutual monitoring, support behavior and team adaptation could lead to a more targeted training of pre-hospital medical teams.
Yes.
The authors declare that they have no conflicts of interest.