An Integrated Methodological Framework to Investigate Hybrid Work Technologies

Abstract

As hybrid work continues to gain prominence in the lives of people around the world, research struggles to grasp its deeper socio-technical implications. We have identified relevant gaps in the existing methodological landscape and outline early-stage design work on a framework that integrates qualitative ethnographic methods with quantitative, sensor-based data collection techniques. This framework will allow us to explore hybrid work technologies more holistically. Although it is still untested, the framework is based on our previous longitudinal research on ambient displays, which already leveraged a mixed methods approach. We hope that it will contribute to the ongoing methodological discussion within the HCI and CSCW communities, and ultimately result in more ambitious and, importantly, long-term research in this area.

Keywords

hybrid work technologies, ambient displays, field studies, long-term research, methodology

License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Jan Schwarzer, Michael Koch, Susanne Draheim, Julian Fietkau, Kai von Luck (2025). An Integrated Methodological Framework to Investigate Hybrid Work Technologies. A contribution to the 2nd International Workshop on Hybrid Collaboration. In Proceedings of the 23rd EUSSET Conference on Computer-Supported Cooperative Work (ECSCW 2025). https://doi.org/10.48340/ecscw2025_ws02_04

1. Introduction

Hybrid work arrangements have become popular post-pandemic, providing organizations with greater flexibility and alternative modes of working. The work-from-home practice remains important and relevant for most employees even after the pandemic [11, 33], while hybrid co-working is increasingly becoming the norm. Hybrid work is a topic of considerable research interest. This advance is associated with many small-scale, process-related changes and challenges to collaboration, cooperation, and coordination, which are currently being academically explored [21]. Even in human–computer interaction (HCI) research contexts, which prior to the pandemic were primarily focused on supporting interaction in physical or remote collaborative work contexts, researchers must now also address the new boundary condition of hybridity.

This position paper gives an overview of the methodological state of the art regarding empirical studies into hybrid work practices from an HCI and computer-supported collaborative work (CSCW) perspective, and then outlines a work-in-progress methodological framework that is designed to facilitate the exploration of tools for supporting collaboration in hybrid work settings. This framework (see Figure 1) will build largely on automation (i.e., algorithm-based interpretations) and data triangulation (i.e., a diverse repertoire of research methods) to better grasp the wider implications of the evaluated technologies.

We argue that more research at a methodological level is needed to ultimately develop a better understanding of hybrid work in practice. As cooperative work is enabled by different technologies “that support the individual’s activity, (…) coordination, and communication (…)” [2], we approach this endeavour by exploring corresponding tools, or hybrid work technologies. Their usage distills important insights on how hybrid work is operationalized in practice. We first touch on the fundamentals of hybrid work and hybrid work technologies, before discussing existing research gaps and presenting preliminary work. We emphasize that our own work presented here is still in its early stages.

2. Related work

The parallelism of embodied and digital interaction as a constant underlying condition of hybrid work is challenging the support of work with digital tools. To effectively support work with digital tools in the context of hybrid work, it is crucial to analyze the characteristics of hybridity more closely. Lindemann and Schünemann [12] state from a phenomenological viewpoint that physical “copresence is a spatio-temporal phenomenon” and each actor could be described as a “spatially embodied self”. In contrast, the digital space is technologically manufactured and therefore differs from the space of embodied experience. Hybrid collaboration refers to “collaborative practices that involve simultaneous co-located and remote collaboration with phases of both synchronous and asynchronous work that spans multiple groupware applications and devices” [17]. This indicates that hybrid collaboration switches back and forth between all four quadrants of the time-space matrix [8] – there are constant transitions between co-located and remote as well as synchronous and asynchronous collaboration.

Fundamentally, cooperative and coordinated work has been a focus of HCI and CSCW research for at least forty years, while geographically distributed collaboration and meetings have been a particular focus. Therefore, we can draw on this research to inform the future of hybrid work.

Historically, CSCW is based on ethnography for analyzing work practices. As a form of inquiry, ethnography relies heavily on participant observation, which is usually done by human observers [19]. However, this traditional stance has now shifted. Studies that emphasize hybrid work technologies in co-located environments increasingly focus on the challenge of examining user behavior using optical sensors such as depth-based cameras [e.g., 5, 31]. The key advantage is that no human intervention is required and that the cameras can operate around the clock, 24/7. They also supplement existing methods (e.g., observations), make passive and active use explicit, generate a large amount of relevant data, and collect data unobtrusively.

Research into user behavior is, nevertheless, considered complex in nature, which is why manual observations and ethnographic studies have often been predominately used in the past. Yet, these more recent efforts aim to augment and automate the processes of data collection and evaluation. They find motivation in technological advances and, with that, in the desire to learn more about the spatial and temporal behavior of users. Fundamentally, the aim is to gain additional information on content transitions, multi-user and single-user interactions, behavior in the surrounding space, etc. So far, however, there are only a few studies embarking on this avenue [e.g., 5]. Williamson and Williamson [31] identify various questions relevant for this emerging field of research. They revolve around, for example, the influence of the presence of a physical artifact on user walking paths or how different interaction techniques attract potential users.

Nonetheless, the field still has not matured to a significant extent. Automated methods have neither been used for a long period of time nor are there established ways to automatically integrate their data with insights from, for example, observations and system logs (e.g., from a touch-enabled interface). Studies suggest that the latter shows great potential for comprehensive analysis reports [5]. Considering remote and hybrid work settings, on the other hand, we see a similar level of maturity. Recent efforts push forward to understand better, for instance, how enterprise collaboration systems are used to support people’s daily work [e.g., 1, 30] or how territoriality affects hybrid collaboration [e.g., 16]. At a methodological level, this research is, however, still in its infancy and hints a lack of automation (e.g., machine learning algorithms) to more readily quantify and analyze insights [16].

Evidently, the interest in understanding how people work together technologically gains attention around the world. While existing studies [e.g., 4, 7, 14, 22] make important contributions to understanding the phenomena of (hybrid) collaboration with (partly) digitally assisted ethnography, the review of the literature brings to light a gap considering both situations appearing together – individuals using technology to collaborate both locally and remotely in some combination. We argue that existing ethnographic approaches [e.g., 3, 9, 13, 15, 29] require methodological contributions to be able to combine qualitative (e.g., field work) and quantitative (e.g., pattern matching) methods for observing on-site and remote work practices – beyond commonly used proprietary tools such as Maxqda, Atlas.ti, or NVivo. In doing so, much could be learned about how different modalities of interaction influence one another and what kinds of interaction patterns emerge under careful observation. Given ways to integrate behavioral data from various sources, observations could integrate these sources whether they are related spatially or through work processes.

3. Research gaps

We conclude that there is evidently a gap in the literature regarding adequate methodological tools to guide longitudinal, in-the-wild research into hybrid work practices. Specifically, the translation of technology-based remote (e.g., using enterprise collaboration systems) and co-located (e.g., with or in front of physical artifacts or in media spaces) coordination activities into holistic algorithmic interpretations of a situation at hand represents an important research gap requiring attention.

The two central research gaps we wish to emphasize in this text pick up on the idea of a holistic methodological framework to target its fundamental scaffold. The first research gap copes with the question of what methods are required to make a hybrid work practice explicit – or, algorithmically readable. This lays the foundation for the second research gap which concerns the question of how we can create automatic, algorithmic interpretations of hybrid work practices. In the following, we go into more detail with respect to these two questions.

Research qestion 1: What combination of (qualitative and quantitative) methodology is required to make hybrid work practices explicit? As explained earlier, there is no existing methodological foundation that allows for analyzing both co-located (e.g., interaction in front of display installations) and remote (e.g., writing blog posts) collaboration activities with hybrid work technologies to identify, for instance, authentic audience behavior or the wider implications. It remains to be determined what the necessary building blocks are (i.e., the qualitative and quantitative methods) and how those need to be triangulated. Especially qualitative data lacks models to be processed digitally, and existing examples of those models only come in proprietary formats used in tools such as Maxqda and Atlas.ti [20, 32].

Research qestion 2: How do we create automatic, algorithmic interpretations of hybrid work practices? We see a lot of room for improvement with respect to the degree of automation during the analysis process of qualitative and quantitative data. There is a lack of making this data readily machine readable. To the best of our knowledge, efforts need to be made to significantly reduce the overall amount of manual work (e.g., conducting in-situ observations) to make longitudinal, in-the-wild research more practical. Comprehensive tools that allow for both the automatic captioning and analysis of hybrid work activities around the clock with only little human intervention are still broadly missing. The same holds true for corresponding visualization tools such as dashboards that can conveniently aggregate and filter the collected data. Overall, means need to be developed to enable automatic, algorithmic interpretations of hybrid work practices.

4. Preliminary work

Our research is at a crossroad in developing the necessary methodological scaffold to unveil the foundations of technology-based hybrid work practices.

The niche we have been primarily working on is ambient displays used as attractors for co-located interaction and as windows into the digital world. The idea is to form physical windows into the digital space. These could be almost literal windows (i.e., bidirectional camera setups for real-time distance collaborations between teams) or more metaphorical windows showing information (e.g., key data and summaries) to foster insights into remote work and to provoke collaboration and innovation. For several years now, our research has been driven by the motivation to scrutinize our custom display solutions in authentic, semi-public spaces for long periods of time. We aim at finding out how people appropriate such technology in their daily life and what the wider implications are. Our research has contributed insightful findings at a methodological, theoretical, and empirical level [e.g., 23, 24, 25]. With that, we follow a current trend in HCI, CSCW, and UbiComp research that emphasizes the importance of in situ evaluations [28].

However, we realize that we must look beyond a strictly technology-centered methodology to really understand what is going on. For instance, considering a study dealing with the automatic clustering of walking behavior [26], we concluded that we lack a deep understanding of what we are seeing in the data. We ask ourselves “Why do people behave in this way?” or “How does the interaction affect work processes?”. Time and time again, we were required to contextualize camera-based observations with insights from qualitative data to really know what was going on. We learned to accept that to fundamentally grasp the effects of our installations on hybrid work activities, we need to embrace the messy nature of in-the-wild evaluations [28] and will have to turn to a multitude of data sources in the future. While we are indeed confronted with a lack of methodological guidance regarding long-term studies in authentic environments [27], efforts to increase the level of automation has become our focus today [26].

Over the years, this thought process has led us to the research gaps presented above. We intend to extend existing ethnographic approaches and combine methods for observing co-located, remote, and hybrid work activities to learn how these different modalities of interaction influence one another and what kinds of interaction patterns emerge under careful observation. The technological context through which we intend to approach the empirical work is that of ambient displays, as we have extensive experience designing and evaluating ambient display deployments in the wild. The next logical step for us is therefore to turn what we have learned into a coherent methodological framework.

5. Methodological framework

This article’s main contribution picks up on the ideas illustrated previously to outline the development and evaluation of a rigorous methodological scaffold targeting the exploration of how technology is used in authentic hybrid work settings long term. Figure 1 shows an initial sketch of the methodology to be developed and evaluated in future research. The underlying rationale is to explore technology in the wild more easily (i.e., higher level of automation) and more comprehensively (i.e., triangulating qualitative and quantitative data). At its core, the methodology aims to largely automate the data collection and analysis process and, in doing so, allows for automatic, algorithmic interpretations of observations.

The left-hand side in Figure 1 indicates the different qualitative methods to be incorporated. This includes, for example, in-situ observations and interviews. We unveiled insights by using methodologies such as grounded theory [23] and coded data using proprietary tools such as Maxqda. However, the findings unveiled were mostly relatable to the co-located part of hybrid work. Hence, we intend to extend this repertoire of qualitative methods by using the diary studies method [18] to equally unveil activities in the hybrid and remote space.

Considering the right-hand side of Figure 1, the two types of quantitative data (i.e., body tracking data and touch interaction data) are depicted as well as a chain that the data passes through. Usually, we initially check for data plausibility in what we call the Preparation phase (e.g., sensor failures). We also carry out data filtering, if necessary. Subsequently, we typically start to scrutinize the data during what we refer to as the Exploration phase. Here, we use different tools to get a better picture of the data. We use own custom tools such as PoseViz [6] or proprietary tools such as Elastic Search, Microsoft Excel, and SPSS for that purpose. Depending on the research questions, hypotheses, or ideas at hand, we then choose on what information to focus on in the next phase – the Feature extraction phase. Here, we try to pinpoint and revise data to address the very questions brought forth (e.g., converting sensor data to variable data, or features). Finally, we concentrate on the process of retrieving actual insights – the Analysis phase. Here, we use the tools mentioned above, but also apply our own custom algorithms [26].

There is, however, still a notable amount of manual labour involved in this chain, which brings us to the so-called Synchronization process. Throughout this stage, we reflect on the opportunities regarding how insights from both worlds can inform each another. For instance, this may involve tasks such as assigning labels to sensor data to train machine learning models according to qualitative observations. Such models could then be leveraged to categorize patterns automatically in the future. This process will play a crucial role in our future endeavours.

6. Conclusion

Finally, we offer a brief reflection on the state of our efforts in relation to the wider research landscape. As argued above, it is clear that the study of hybrid work technologies and practices requires novel methodological contributions to graduate from insular technocentric results from short-term studies to deep insights based on long-term studies integrating both quantitative and qualitative data in their research. Although there is an increasing amount of in-the-wild research in HCI and CSCW, there are still open questions regarding methodological development.

The methodological framework presented here may be one potential stepping stone towards this goal – albeit one that is based on own extensive experiences from past research. However, we hope that the ideas presented here will encourage fruitful discussions within our communities and help us to advance on these pressing issues in the future.

References