In order to understand how to make useful IoT, it is important to concern with the integration of theories and methodologies to support the combined investigation of machines, humans, and domains of application [18]. Although in the context of computer science ethnography has been adopted within the human-computer interaction community to conduct a study of the workplace and inform the design of computer application [19], developers appear to have limited experience with ethnography [20]. In this manner, end-users are representative of users and developers to some degree do not see themselves are the users as well in the development process. As discussed, ethnographers are unique professionals, and they are good at describing what is and is not important, relevant, interesting, painful, exciting to the users, not to the systems developers. They do not make a priori expectations, judgment, and an intrusive of how users work with technologies but provide a systematic and ongoing data collection and analysis for developers who seek to confirm and disconfirm requirements in the development work [19], [20]. Thus, developers are the users who need help to make implicit explicit from the grounded, real practices of end-users to strengthen the investigations into the human and technology aspects of IoT.
Theories
Phenomenology
Phenomenology concerns experience through the eyes of the first person. Experiences always belong to someone, but how do we subjectively experience a phenomenology and what is the structure of how a phenomenon appears to us? Merleau-Ponty’s use of first person’s perspective [21] tried to changing bodies such an injuries, pain, and suffering helps us contextualize and apply the philosophy as a theory in design context [22]. The first person’s perspective has a natural and vigorous relationship with design practice as it relates our presence and experiences towards our immediate environment and the world around us, which includes everything from people to objects, as well as the relationships between them. The role of embodiment in perception and cognition provides a grounding for understanding interaction through the relationship between the body and the world. It allows researchers to study interaction through a phenomenological lens where people perceive the world through bodily senses and therewith how interpretations, experiences, and intentions revolves around how the human body immerses itself in the world.
However, studying the world through the eyes of conscious experiences carries both a first person and ontological aspects, which Martin Heidgger paid particular attention to in his work. A first person can be labelled as the study of, among other things, structures of conscious experience and what it means that experiences are experienced, it still raises the ontological question of what our experiences, i.e., the nature of the structures of conscious experiences, are [23].
The first person aspect was influenced by and critical of central philosophical figures, both historical and contemporary. Although it is out of the scope to discuss the preceding phenomenology in this paper, it is necessary to mention that the first person as a term can be rooted to Husserl and Heidegger’s critiques of objectivity and objective knowledge that can not properly address important aspects of the world of experiences [24]. This is a shortcoming of natural sciences, further, engineering, since it neglected the animate dimension of the body, i.e., the unique manner in which we experience our bodies as something subjective and categorically different from other people’s experience of embodiment [23]. Heidegger further built an ontological understanding around the very existence of humans in the world to frame concepts, such as subject, object, consciousness, and world into an interconnected and co-existing structure [25].
In line with this, Merleau-Ponty justify how our subjective perspective finds its place in and connects with the world: I can cleary distinguish from myself the world and things, since I certainly do not exist in the way in which things exist [26, p. xiv]. Thus, the perceptual experience involves three elements: the subject, the object, and their relationship. The body represents the subject, the world is the object, and their relationship manifests itself through the consciousness. Of course, we need to consider other types of body, the moving, habitual, intentional, and ill bodies. However, the most fundamental basis for consideration is the relationship between the world and the body – the interactive relationship. Therefore, in order to bring in an extensive understanding for establishing a social internet of things for the maritime domain, we must introduce another theory to bind up individual interactive relationship as a group structure. Thus, the next subsection, we introduce the Actor Network Theory.
Actor network theory
If we understand SIoT is about establishing a useful platform to enable different sensors, physical objects and end-users can work together; then ANT also addresses for better understanding how technology and social can work together for delivering a socio-technical solution for such platform. Actor-network theory is useful because every object in the IoT platform does not work alone. Instead, every object in an IoT platform contributes equally to enable end-users can use the new IoT platform to finish tasks, such as remote control of unmanned ships onshore. ANT thus provides a method for describing how, where, and to what extent technology influences human behavior [27]. Moreover, ANT helps to examine the relations among the end-users, the sensors, and the physical objects and offers a way for engineers to visualize such relations to enable a complex work can be done.
Law [28] defined ANT as follows:
Actor-network theory is a disparate family of material-semiotic tools, sensibilities, and methods of analysis that treat everything in the social and natural worlds as a continuously generated effect of the webs of relations within which they are located. It assumes that nothing has reality or form outside the enactment of those relations. Its studies explore and characterize the webs and the practices that carry them. Like other material-semiotic approaches, the actor-network approach thus describes the enactment of materially and discursively heterogeneous relations that produce and reshuffle all kinds of actors including objects, subjects, human beings, machines, animals, ‘nature,’ ideas, organizations, inequalities, scale and sizes, and geographical arrangements.
Thus, ANT provides a framework and a systematic means for considering all the factors in the social and natural worlds [29]. ANT does not explain why the network exists; it is concerned with the infrastructure of actor networks, how they form and how they can fall apart [30].
ANT combines the so-called generalized symmetry principle, that is, human and nonhuman things should be incorporated into the same conceptual network and be allocated the same amount of agency [31] as well as the interactive relationships. An actor is that which accomplishes or undergoes an act. When we act, we always interact with others. As Law [32] stated, “Interaction is all that there is.” During these interactions, we change the other actors. At the same time, however, the other actors are changing us [33]. We also follow the principle and deal with the cooperation between humans and nonhumans because it is essential for a successful marine operation. ANT considers that not only human beings but also nonhuman entities constantly influence us. Hence, the specific mechanisms at work can be described in detail while allowing the actor to be treated fairly [29]. In this manner, we significantly contribute how to collective the subjects, objects, and their interactive relationships as a whole to define a first person’s perspective on dealing with design to support the relationship between the world and the body.
Additionally, regarding the actor-network, Callon describes how actor-worlds (i.e., a metaphor) function and how the relations between the different actors are organized and structured[34]. Callon [34] explained the following
It is clear that an actor-world may be more, or less, extended, heterogeneous and complex. How shall we describe this range of possibilities and the translations that occur between them? To answer this question, we introduce the notion of actor-network. This concept allows us to describe the dynamics and internal structure of actor worlds.
Based on this explanation, we can understand that the actor-network is the heterogeneous network of human and nonhuman actors. The relationship between them is important; such relationship is not their essential or inherent (Callon, 1986, 1991). ANT assumes that there are no purely human or nonhuman networks. All networks contain elements of both and are heterogeneous and social-technical [37]–[39]. Latour [37] employed to metaphors and advised thinking about nodes, which have as many dimensions as they have connections, instead of thinking about surfaces and the dimensions of these surfaces. For instance, the foundational ANT studies focused on the breakdowns upon which networks become visible. Thus, ANT is concerned with the links that hold and the links that do not hold [40] in network dynamics. Network dynamics assume that actors, human versus nonhuman neutrally, have different and often incompatible interests. The social order or the stability of a network depends on its alignment [41]. Such alignment is based on translation of different interest of actors in a network to define a framework for possible action. The alignment is a necessary process before we get a result of what should be in an actor-network, and it continues until equilibrium has been reached and modified [42].
How theories contribute to IoT
It must be clear for IoT developers that IoT platform is more than the technical-based area that only sensors or physical devices are connected. Also, humans are also connected to the network and play an important to role to coordinate the IoT platform functional in their work context. In most case, developers presume that the “things” on the internet are an object. As defined by the Oxford dictionary, an object is “a material thing that can be seen and touched; a person or thing to which a specified action or feeling is directed [43]”. From both first person and ANT aspects, objects are considered tools used to coordinate the work in everyday life of body where objects are used to realize the work among the collaboration of objects and bodies. As an object, it plays an important role in coordinating work practices [44] and thus is not only about technical but social meanings.
Objects as materials can pre-exist in the technical world. An important investigation is how to bound object via human activities regarding illustrating their work practices that inform design [45]. From a first person perspective, it is a material given by someone’s own experiences. However, in order to make sense such experiences from an ontological level to a wide range of users, we must search for the intra-actions among materials, materials and their locations, and materials and the human mobility patterns associated with different bounding [45]. Regarding design, researchers investigated the role of a particular ecological arrangement of material in performing work [16]. An immutable mobile object needs to be “both plastic enough to adapt to local needs and constraints of the several parties employing them, yet robust enough to maintain a common identity across sites” [46].
As Husserl, Heidegger and Merleau-Ponty’s justify that an object is given meaning by the iterative interaction relationship between the body and the world. In this manner, we should argue that when designing an object, following the interactive relationship is important to bring social context of object use into design and make the digitalization possible. Suchman defines this relationship as the dynamics of object:
The dynamics of computational artifacts extend beyond the interface narrowly defined, to relations of people with each other and the place of computing in their ongoing activities. System design, it follows, must include not only the design of innovative technologies but their artful integration with the rest of the social and material world.
According to this definition, objects consist of more than a physical shape. Artefacts also help users to function in their daily work practices [47]. Moreover, objects are designed to react according to the activities in which they are used, to be incorporated into sophisticated semiotic practices, and to respond to and integrate unfolding events that go beyond function to address social meaning [47].
Thus, it should be clear that ‘things’ in IoT are not only objects to support daily work and collect data. Moreover, social contexts and the work practices in them enable an IoT based work meaningful. In this manner, it will be fruitful to tell the whole story of the relations of thing as the non-material component of human practices in workplace [50]–[52] to make sense of the IoT. Thus, designing social aspect into IoT is about how to unfold the work practices of the body, object, and interactive relations. In that case we must acknowledge that in operations of technologies it is important to know who, where and what is involved in a work and how the work is done. This inquiring requests us to understand ‘things’ as the process of shaping interactive relationships regarding the tasks of each operator and the connections among the various tasks, that is, the interactive relations. This understanding contributes to shifting the focus from the internal opacity [47] of an object to its relations to human practices in a cooperative social context. According to Schmidt and Bansler [47],
We do not need to understand the internal mechanism of an artifact to make rational use of it; nor do we in fact normally do that. One doesn’t need, say, to understand the specifics of the lattice structure of steel alloys causing the operational properties of one’s damascenes kitchen knife: its hardness, its tensile strength, its elasticity. What one needs to understand is its ‘functionality.’ And in the case of machinery, what one needs to know is the dependable regularity of its behavior. That’s all.
According to this, contributing social aspect to IoT development is about how we understand social phenomena and using such resources to make changes and arrangements for the existing algorithm, system structure, as well as the information model. Thus, we believe that such understanding can provide the opportunity to use the ‘things’ to enhance the translation of social theories and related outcomes so that the project audience could shift their focus from the internal mechanism of an object to understanding it as a ‘thing’ and its relations in human practices. This is crucial for the development of IoT.
A maritime case
In this section, we illustrate a maritime case to show how ANT can help with configuring ‘things’ for developers. This could help developers to ensure what is useful in an IoT platform and how to develop sensors in the IoT for safe and efficient work practices with humans.
Norway as a leading nation in maritime domain recently focuses on establish the first remote control center for onboard support of unmanned ships. Ship navigation in the open sea can be nearly autonomy whereas, for some part of a voyage like passing narrow water, it will require supervision, decision-making and full teleoperation onshore ship bridge (see Fig. 1). Traditionally humans operate on the vessels they only need to communicate with traffic service center. However, if humans are moved out from the unmanned vessel but still need to take care of the unmanned vessel regarding safety, humans are not only monitoring information sending back from the unmanned vessel. If necessary, they need to start recovery planning and directly operate ships. This makes huge uncertainties of sensor technologies since we still do not fully understand what information is needed and useful for people who remotely control ships. Therefore, it raises other questions about trust in the systems, information overflow, training humans and their roles in the remote control center, loss of ship senses, and capacity for teamwork and so on.
All above concerns might be discussed too much and early until there is a useful remote control center is established. In line with ANT, it is possible to configure and reconfigure what is important information for onshore control.
From human-in-the-loop to actor-in-the-loop
Human-in-the-loop (HITL) is discussed for a long-term [54]. For example, people are involved in a virtuous circle where they train, tune, and test a particular algorithm. First, humans label data – this gives a HITL model a high-quality training data. Developers learn to make decisions from the data. Second, humans tune the model – this can happen in several different ways, but commonly, humans will score data to account for overfitting, to teach a classifier (i.e., algorithm) about edge cases, or new categories in the model’s purview. Lastly, people can test and validate a model by scoring its outputs, especially in places where an algorithm is unconfident about judgment or overly confident about an incorrect decision.
However, there are several flaws in such model. Humans here, as we assume, they are people who develop the IoT platform. They are not the people who experience, live and work with the IoT platform in their daily life. Developers are not and will not be experts in the professional working context wherein domain knowledge are strongly required [55], [56]. Therefore, we must acknowledge that the front-line workers who are using IoT technology should be involved in the development process.
To engage with front-line workers, a development space should be established for allowing fieldwork experts to gain domain knowledge with front-line workers in their daily life [57]. As front-line workers are experts in their work contexts, they knowledge much better what is needed and have a good reason to select necessary technology to support their work practices [58].
Of course, there are differences between developers and front-line workers regarding background, expertise, and experiences. The important thing is to convert their different interests in the development process as a common goal with the IoT platform – the automation of collecting sensors. ANT is a method to deal with the translation of different interests in the development process. In ANT perspective, there is no difference between humans and nonhumans [29]. Hence, both of them are actors in the development process (see Figure 1).