Skip to Content

Human Experience and Design

Microsoft Research

Social Devices

  • Landing page
  • Meeting with Screen Share
  • selecting people
  • Select people
  • Accept Call
  • Answer

Currently our everyday computing is experienced across a wide range of different devices and form factors such as smart phones, tablets, watches, laptops and TVs.  In spite of this ecosystem and increasing opportunities for connectivity, much of our user experience with these devices remains fractured, with devices operating independently and with limited scope for how nearby devices can combine together in effective and meaningful ways.  With predominant models of everyday computing, devices are unaware of other devices that are around, unaware of the capabilities and state of these nearby devices and, unaware of who owns them.  Functional resources remain locked within a particular device and unavailable to other nearby devices.

Rather than thinking about user experiences operating independently within the confines of a single device, the concept of Social Devices is concerned with how computing experiences may extend across multiple co-proximate devices and applications exploiting their respective affordances of devices/apps in complementary & contextually relevant ways. 

Moreover, the concern is not simply with the design of cross-device user experience.  Rather, there is acknowledgement of the changing and often unpredictable nature of our everyday social and activity experiences.  With this in mind, Social Devices looks to ways that our computing devices/resources/capabilities can be dynamically aggregated and dis-aggregated to create user experiences that adapt to the changing social and activity demands we encounter throughout the day.

  • Roving Camera
  • SkypeSD
  • Camera input

Cross Device experience in Video Mediated Collaboration

In order to explore the social devices ideas further our initial explorations have focused on the area of video mediated collaboration.  Much of what is important about facilitating remote collaboration in both work and domestic contexts is the sharing of artefacts and context around which a conversation can then constructed.  

Current video conferencing applications remain limited in their scope for the configuration of artefact sharing and in the configuration of group collaboration.  In particular there is a tendency for VC design to focus on supporting collaboration between remote participants often ignoring the important dynamics of collaboration for any collocated members of the collaboration. Participants in the local space are currently limited in the ways that they can participate in the meeting and the ways that they can contribute information and media resources from their personal devices. By exploring how these collaboration experiences can be configured across multiple co-proximate devices we can open up intriguing new opportunities for shaping workplace and domestic video mediate collaboration. 

For further information please contact Kenton O'Hara

  • SkypeBeam1

SkypeBeam: Wireless Screen Mirroring in Video calls

In video call scenarios, collocated participants at one end of a call will typically share a single machine hosting a call rather than logging on via separate devices.  While screen sharing is commonplace and an integral part of many video mediated collaborations, the use of a single machine limits the ability to perform hoc screen sharing from personal devices of multiple collocated participants on each end of a video call.  

SkypeBeam is a system that addresses this limitation by enabling lightweight multi-user wireless smartphone mirroring within a video call. The system allows multiple smart phones to share both digital content as well as physical artefacts when mirroring the live view from the smart phone camera feed. 

Utilising cross device interaction design experiences in this way creates new opportunities for enhancing workplace and domestic collaboration:

  • many devices allows content sharing capabilties to be available to all co-proximate parties
  • enables greater particpation in conversation by all collocated collaborators
  • sharing interaction across multiple devices enables parallel activity facilitating fluid transitions from individual I-work to collaborative we-work
  • phone is a personal device so interaction does not have to be negotiatied with others
  • preparation for sharing can be done without disrupting the shared display
  • mobile form factor allows content sharing opportunities to be brought to me and my artefacts – low barrier to entry 
  • mobile phone camera allows physical information artefacts (e.g. flip chart, whteboard, printouts, paper notes) to also be shared in the Skype call from wherever they are in the meeting room


For further information about SkypeBeam please contact Kenton O'Hara

See also: Sorensen, H., O’Hara, K., Gosset, P. and Kjeldskov, J. (2015) Wireless SmartPhone Mirroring in Video Calls. In Proceedings of Interact 2015.

  • Engage
  • Auditorium

Engage: Inclusive Participation in Collaboration

Slide presentations during meetings or classes have long been stuck in a one-to-many paradigm creating a model of collaboration that is speaker-led.  Such speaker-led collaboration models limit the opportunities available for others in the meeting to participate – passively listening rather than actively contributing. Drawing on the concepts of social device thinking we are exploring ways to facilitate more interactive participation in presentations by other co-present collaborators.  In particular we are looking at socially appropriate and useful ways for presenters to extend interactive access/control of presentations beyond the confines of their own devices to the personal devices of their audience. 


For further information please contact Kenton O'Hara

See also: Chattopadhyay, D., O’Hara, K. Rintel, S. and Rädle, R. (2016) Office Social: Presentation Interactivity for Nearby Devices. In CHI 2016.