Chronic diseases such as diabetes, heart failure, and obesity are widespread glob-ally. These diet-related diseases are mainly caused due to limited physical activity and poor eating patterns. Journaling and self-monitoring have been very effective tools in combating diet-related diseases as they help to discover undesired patterns at an early stage and motivate users to lead a healthy lifestyle.

Smartwatches are commonly used for fitness tracking. They can recognize dif- ferent types of physical exercises and provide rudimentary measurements for health metrics such as heart rate variability, energy expenditure, and sleeping hours. While useful, these features do not provide users with a holistic view of how their daily activities influence their health and how their body reacts. For example, knowing how many calories we burn is insufficient unless we compare it to our calorie intake. Current food journaling methods rely heavily on self-report, which suffers from self-bias, recall errors, and low adherence. In the last two decades, researchers have developed several automatic diet monitoring (ADM) systems to address the challenges of traditional journaling techniques. The focus of the diet monitoring research has been on detecting people eat, and identifying what and how much they ate. Ecological validity has been a major issue in ADM research. While many ADM systems obtain high accuracy in lab settings their performance drops significantly when tested in the real world. The most cited reasons for this challenge are the difficulty to build generalizable models using data collected in the lab, the lack of reliable ground truth in free-living environments, privacy concerns, and the social acceptability of the device.

In my research, I tackle these challenges by developing and deploying a number of ADM systems (EarBit and FitByte). These trackers are hosted in commonplace form factors (i.e. headphones and eyeglasses) to ensure their social acceptability. I also worked on designing data collection techniques to build models that work reliably in the real world. The high performance obtained by these models has brought us closer to assessing the utility and usability of ADM systems in the field.

The final piece of my dissertation is a long-term field deployment for an ADM system (FitNibble) based on my previous ADM designs. In this study, I compared traditional self-report journaling and journaling with ADM. Through this evalua- tion, I assessed the factors influencing adherence to journaling like reducing missed events, social acceptability, usability, utility, and privacy concerns. Results have shown that FitByte2.0 improved adherence by significantly reducing the number of missed events (19.6%improvement, p = .0132. Results have shown that participants were highly dependent on the wearable in maintaining their journals. Participants also reported an increase in their awareness of their dietary patterns especially with snacking. All these results highlight the potential of ADM in improving the food journaling experience.

128 pages

Thesis Committee:
Mayank Goel (Chair)
Jeffrey P. Bigham
Geoff Kaufman
Edison Thomaz (University of Texas at Austin)

Jodi Forlizzi, Head, Human-Computer Interaction Institute
Martial Hebert, Dean, School of Computer Science

Return to: SCS Technical Report Collection
School of Computer Science homepage

This page maintained by reports@cs.cmu.edu