Publications
WheelPoser: Sparse-IMU Based Body Pose Estimation for Wheelchair Users
Yunzhi Li, Vimal Mollyn, Kuang Yuan, Patrick Carrington - ASSETS 2024
[PDF] [Video Demo] [Code and Dataset]
Despite researchers having extensively studied various ways to track body pose on-the-go, most prior work does not take into account wheelchair users, leading to poor tracking performance. Wheelchair users could greatly benefit from this pose information to prevent injuries, monitor their health, identify environmental accessibility barriers, and interact with gaming and VR experiences. In this work, we present WheelPoser, a real-time pose estimation system specifically designed for wheelchair users. Our system uses only four strategically placed IMUs on the user's body and wheelchair, making it far more practical than prior systems using cameras and dense IMU arrays. WheelPoser is able to track a wheelchair user's pose with a mean joint angle error of 14.30 degrees and a mean joint position error of 6.74 cm, more than three times better than similar systems using sparse IMUs. To train our system, we collect a novel WheelPoser-IMU dataset, consisting of 167 minutes of paired IMU sensor and motion capture data of people in wheelchairs, including wheelchair-specific motions such as propulsion and pressure relief. Finally, we explore the potential application space enabled by our system and discuss future opportunities. Open-source code, models, and dataset can be found here: https://github.com/axle-lab/WheelPoser
Privacy vs. Awareness: Relieving the Tension between Older Adults and Adult Children When Sharing In-home Activity Data
Jiachen Li, Bingrui Zong, Tingyu Cheng, Yunzhi Li, Elizabeth D Mynatt, and Ashutosh Dhekne - CSCW 2023
[PDF]
While aging adults frequently prefer to "age in place", their children can worry about their well-being, especially when they live at a distance. Many in-home systems are designed to monitor the real-time status of seniors at home and provide information to their adult children. However, we observed that the needs and concerns of both sides in the information sharing process are often not aligned. In this research, we examined the design of a system that mitigates the privacy needs of aging adults in light of the information desires of adult children. We apply an iterative process to design and evaluate a visualization of indoor location data and compare its benefits to displaying raw video from cameras. We elaborate on the tradeoffs surrounding privacy and awareness made by older adults and their children, and synthesize design criteria for designing a visualization system to manage these tensions and tradeoffs.
Breaking the "Inescapable" Cycle of Pain: Supporting Wheelchair Users' Upper Extremity Health Awareness and Management with Tracking Technologies
Yunzhi Li, Franklin Mingzhe Li, and Patrick Carrington - CHI 2023
Upper extremity (UE) health issues are a common concern among wheelchair users and have a large impact on their independence, social participation, and quality of life. However, despite the well-documented prevalence and negative impacts, these issues remain unresolved. Existing solutions (e.g. surgical repair, conservative treatments) often fail to promote sustained UE health improvement in wheelchair users’ day-to-day lives. Recent HCI research has shown the effectiveness of health tracking technologies in supporting patients’ self-care for different health conditions (e.g. chronic diseases, mental health). In this work, we explore how health tracking technologies could support wheelchair users’ UE health self-care. We conducted semi-structured interviews with 12 wheelchair users and 5 therapists to understand their practices and challenges in UE health management, as well as the potential benefits of integrating health tracking technologies into self-care routines. We discuss design implications for UE health tracking technologies and outline opportunities for future investigation.
"I Should Feel Like I'm In Control": Understanding Expectations, Concerns, and Motivations for the Use of Autonomous Navigation on Wheelchairs
JiWoong Jang, Yunzhi Li, and Patrick Carrington - ASSETS 2023 Poster
[PDF]
Autonomous navigation on wheelchairs promises to be a significant frontier in the evolution of the power wheelchair as an assistive enabling device, and is increasingly explored among researchers for its potential to unlock more accessible navigation for wheelchair users. While developments on path-planning methods for wheelchairs is ongoing, there is a relative paucity of research on autonomous wheelchair navigation experiences which accommodate potential users’ needs. In this work, we present preliminary design considerations for the user experience for autonomous wheelchair navigation derived from a semi-structured interview with ten (10) current wheelchair users about their willingness to use and applicability of an autonomous navigation function.
Travelogue: Representing Indoor Trajectories as Informative Art
Yunzhi Li, Tingyu Cheng, and Ashutosh Dhekne - CHI 2022 Late-Breaking Work
In this work, we explore if informative art can represent a user’s indoor trajectory and promote user’s self-reflection, creating a new type of interactive space. Under the assumption that the simplicity of a digital picture frame can be an appealing way to represent indoor activities and further create a dyadic relationship between users and the space they occupy, we present Travelogue, a picture-frame like self-contained system which can sense human movement using wireless signal reflections in a device free manner. Breaking away from traditional dashboard-based visualization techniques, Travelogue only renders the high-level extent and location of users’ activities in different informative arts. Our preliminary user study with 12 participants shows most users found Travelogue intuitive, unobtrusive, and aesthetically pleasing, as well as a desired tool for self-reflection on indoor activity.
Flexible Computational Photodetectors for Self-powered Activity Sensing
Dingtian Zhang, Canek Fuentes-Hernandez, Raaghesh Vijayan, Yang Zhang, Yunzhi Li, Jung Wook Park, Yiyang Wang, Yuhui Zhao, Nivedita Arora, Ali Mirzazadeh, Youngwook Do, Tingyu Cheng, Saiganesh Swaminathan, Thad Starner, Trisha L. Andrew, and Gregory D. Abowd - Nature Flexible Electronics
Conventional vision-based systems, such as cameras, have demonstrated their enormous versatility in sensing human activities and developing interactive environments. However, these systems have long been criticized for incurring privacy, power, and latency issues due to their underlying structure of pixel-wise analog signal acquisition, computation, and communication. In this research, we overcome these limitations by introducing in-sensor analog computation through the distribution of interconnected photodetectors in space, having a weighted responsivity, to create what we call a computational photodetector. Computational photodetectors can be used to extract mid-level vision features as a single continuous analog signal measured via a two-pin connection. We develop computational photodetectors using thin and flexible low-noise organic photodiode arrays coupled with a self-powered wireless system to demonstrate a set of designs that capture position, orientation, direction, speed, and identification information, in a range of applications from explicit interactions on everyday surfaces to implicit activity detection.
Duco: Autonomous Large-Scale Direct-Circuit-Writing (DCW) on Vertical Everyday Surfaces Using A Scalable Hanging Plotter
Tingyu Cheng, Bu Li, Yang Zhang, Yunzhi Li, Charles Ramey, Eui Min Jung, Yepu Cui, Saiganesh Swaminathan, Youngwook Do, Manos Tentzeris, Gregory D. Abowd, and HyunJoo Oh. - UbiComp 2021
We present Duco, a large-scale electronics fabrication robot that enables room-scale & building-scale circuitry to add interactivity to those vertical everyday surfaces. Duco negates the need for any human intervention by leveraging a hanging robotic system that automatically sketches multi-layered circuity to enable novel large-scale interfaces. Our technical evaluation shows that Duco’s mechanical system works robustly on various surface materials with a wide range of roughness and surface morphologies. And we demonstrate our system with five application examples, including an interactive piano, an IoT coffee maker controller, an FM energy-harvester printed on a large glass window, a human-scale touch sensor and a 3D interactive lamp.
OptoSense: Towards Ubiquitous Self-Powered Ambient Light Sensing Surfaces
Dingtian Zhang, Jung Wook Park, Yang Zhang, Yuhui Zhao, Yiyang Wang, Yunzhi Li, Tanvi Bhagwat, Wen-Fang Chou, Xiaojia Jia, Bernard Kippelen, Canek Fuentes-Hernandez, Thad Starner, and Gregory D. Abowd - UbiComp 2020
We present OptoSense, a general-purpose self-powered sensing system which senses ambient light at the surface level of everyday objects as a high-fidelity signal to infer user activities and interactions. To situate the novelty of OptoSense among prior work and highlight the generalizability of the approach, we propose a design framework of ambient light sensing surfaces, enabling implicit activity sensing and explicit interactions in a wide range of use cases with varying sensing dimensions (0D, 1D, 2D), fields of view (wide, narrow), and perspectives (egocentric, allocentric). OptoSense supports this framework through example applications ranging from object use and indoor traffic detection, to liquid sensing and multitouch input. Additionally, the system can achieve high detection accuracy while being self-powered by ambient light.
How Presenters Perceive and React to Audience Flow Prediction In-situ: An Explorative Study of Live Online Lectures
Yunzhi Li* , Wei Sun*, Feng Tian, Xiangmin Fan, Hongan Wang (*joint first author) - CSCW 2019
The degree and quality of instructor-student interactions are crucial for the engagement, retention, and learning outcomes of students. However, such interactions are limited in live online lectures, where instructors no longer have access to important cues such as raised hands or facial expressions at the time of teaching. This project presents an explorative study investigating how presenters perceive and react to audience flow prediction when giving live-stream lectures, which has not been examined yet. The study was conducted with an experimental system that can predict audience’s psychological states (e.g., anxiety, flow, boredom) through real-time facial expression analysis, and can provide aggregated views illustrating the flow experience of the whole group.