Overview
Each year, REU students have the opportunity to engage in a wide variety of research projects related to smart environments. Here are descriptions of some of those project areas.
Monitoring and Modeling Human Behavior
The goal of this research area is to design automated tools for monitoring and modeling human behavior by analyzing data that is collected by sensors embedded in buildings (using environmental sensors that monitor motion, door use, temperature, and lighting) or carried by individuals (e.g., smart phones, wearable sensors). Our current work has shown that activities can be effectively recognized from environmental sensors. REU students will refine and enhance machine learning algorithms to discover and recognize behavioral patterns and activity from raw sensor data. Using wearable sensors, we have pioneered work examining the relationship between sleep and everyday function in the older adult population. By combining information provided by smart environments (e.g., total movement, activity recognition) with data collected by wearable sensors (e.g., an actigraph energy-monitoring watch), REU students will design approaches to automatically estimate calorie consumption and total energy expenditure that is noninvasive and is more accurate than self-reporting.
Identify Correlations between Behavior and Health
Smart home technologies that produce continuous data can provide health care professionals with the opportunity to see variability and trends or trajectories, rather than absolute values, and to identify how daily activities impact traditional measures. Data collected through these technologies, combined with symptoms reported by patients in the clinician’s office, could increase health and well-being through early detection and treatment and increase the accuracy of diagnosis. We are collecting data in 40 smart homes with older adults and are performing monthly well-being assessment of the participants. We have already found correlation between observed task quality of these activities and cognitive health of the participants. REU students will design methods of finding latent correlations between these parameters and determining the ecological validity of standard self-report and laboratory-based assessments. REU students will research methods of identifying correlations between human behavior and resource utilization. Students will have access to real-time, whole-building energy usage data for 15,000 buildings in the Pullman area. We will predict energy consumption based on detected individual behavior and will identify clusters of similar consumption patterns based on building features, demographics, and behavior patterns.
Designed Interventions to Promote Aging in Place
In order to keep adults functioning independently at home, the smart environment needs to play a Cognitive Prosthesis role in extending or enhancing their ability to perform everyday activities using automated interventions. One type of intervention that is useful for an individual with limiting conditions is automated reminders. However, identifying the appropriate timing for a reminder is difficult. We used supervised machine learning techniques to identify contexts in which caregivers provide prompts, and used these contexts to train a machine learning algorithm to automate prompts in similar situations. REU students will interact with health care providers and observe caregiver interactions in adult day care and assisted care facilities to see what types of caregiver assistance are naturally provided and in what contexts caregivers typically intervene. A unique component of our proposed REU research is that our multidisciplinary team will investigate uses of the automation that are beneficial for health and sustainability in the smart environments. REU students will design methods that provide interventions to enhance the capabilities of the resident while still encouraging the residents to perform as many tasks for themselves as they are able. Students will also investigate machine learning approaches for giving these intervention methods the ability to adapt to the changing cognitive and physical abilities of the resident. A type of intervention that has been investigated by the community to assist adults with dementia is the use of computer and card games to assess and improve cognitive performance. REU students will participate in a Microsoft-sponsored project to design serious games. The purpose of these games will be to perform automated assessment of cognitive function and to teach compensatory strategy skills for individuals with cognitive impairment.
Integration of Advanced Sensors
We have extensive experience in deploying and utilizing sensors to accomplish recognition and prediction task in smart environments. Crucial to this work is the integration of new advanced sensors, as they become available. Leveraging our new Kinect One sensors and tools developed through our senior capstone projects, REU students will evaluate the state of the art in free-form voice-based communication with smart environments, as well as issues with human computer interaction. Results will be centered on the effectiveness of this kind of interface for residents in smart homes. The Panasonic GridEye sensor produces richer and more complex data than we have dealt with before. REU students would develop new methods to interpret the data and use established algorithms to determine what the sensor is "seeing."
HC-Search for Activity Recognition
Activity Recognition (AR) is a very important task in the context of smart environments. AR can be considered as an instance of structured prediction, where the sensor data sequence corresponds to the structured input, and the activity sequence corresponds to the structured output. HC-Search is a new framework for structured prediction that was shown to perform significantly better than standard approaches including Conditional Random Fields (CRFs) and recurrent sliding window classifiers on a number of benchmark problems. REU students will apply HC - Search for AR using the data from smart home infrastructure at WSU.
Decision-Theoretic Assistants for Smart Homes
We want to build intelligent assistants to help the people (agents) living in a smart home environment. In this project, REU students will leverage and build on the recent work on restricted classes of POMDPs called Hidden Goal MDPs (HGMDPs) and Helper Action MDPs (HAMDPs). The main challenge is that the goals of the agent are hidden from the assistant. Therefore, the assistant has to estimate the goal of the agent and then select assistive action(s) for a given state. We will evaluate the performance of the assistant with and without the prior-knowledge in our smart home infrastructure at WSU.
Applying Clique Finder for Data Compression
We have developed a fast, parallel, branch-and-bound type algorithm for finding a clique of maximum size in a network. The need for finding maximum cliques arises in many applications ranging from systems biology to machine learning to scientific computing. The goal of this project is to apply the clique finder to a large social network (e.g., a Facebook graph) that will further be used to obtain a compressed representation of the network. Students will conduct research in high-performance graph algorithms and their application.
Network Analysis Tools
Networks have become a worthy modeling language for diverse phenomena in nature and society. Network science is emerging as an interdisciplinary field that seeks to understand the structure and behavior of complex networks and to reason about dynamic processes that take place on them. The goal of this project is to collect some of the most prominent tools available and assess them along several metrics, including capability, ease-of-use, and performance. Students will learn how to select a network analysis tool for a specific need and successfully use the tool to conduct analysis.
Embedded and Pervasive Systems
Embedded and pervasive systems, which utilize wearable and lightweight biosensors, allow for collection and analysis of physiological and contextual information about individuals and provide a means for continuous, real-time, and automated interventions. REU students will be involved in projects that provide them with both a hands-on experience in system integration and an opportunity to explore and address a research problem in biomedical signal processing, pervasive computing, distributed computing, or medical embedded system design. Students will also have a chance to get involved in clinical trials. Furthermore, the interdisciplinary nature of this research allows undergraduate students to participate in meetings with our clinical partners in medicine, nursing, pharmacy, and health sciences. REU students working in this area will be involved in various infrastructure development and hardware/software prototyping efforts related to design, development, and validation of a peripheral edema monitoring system, called Smart-Sock. The project develops a highly wearable sensor platform, embedded in a sock, and accompanying algorithms, software, back-end databases and data analyses techniques to measure lower-limb swelling (i.e., peripheral edema) and report severity of a medical condition. Peripheral edema is one of the early signs of volume overload in the body due to onset or exacerbation of a variety of systemic life threatening diseases. Our pilot application is congestive heart failure and we are currently collaborating with UCLA School of Medicine on this project.
Dynamic Graph Mining
We are developing new methods for pattern learning in dynamic graphs by analyzing only the changes to the dynamic graph, without having to build the entire graph periodically over time. We have developed such methods for streams of additive structural changes. Many approaches to mining graph streams utilize a windowing approach, which analyzes a time-window-based selection of the data stream. REU students will apply this windowing technique, but on the stream of changes, rather than graph snapshots. One promising target for graph mining is sensor data collected from smart environments. We have successfully designed machine learning algorithms to recognize resident activities from this information. While the algorithms typically use summary statistics of sensor events in a feature-vector based approach, there is a rich source of additional information in the relational structure of the event sequence. For example, we can build graphs from motion sensor event sequences, where each node in the graph is a motion sensor, and each edge represents a transition from one sensor to another. REU students will evaluate the use of these graph-based activity signatures to improve the accuracy of activity recognition either by including them as features or performing relational learning based on these structures. We want to scale our sensor data collection to a large collection of ~100 homes and provide longitudinal data that monitors behavioral patterns over multiple years, allowing REU researchers to thus examine behavioral cycles and trends. Finally, we want to include collections of smart environment data for target populations, specifically for older healthy adults and older adults with dementia who can benefit from health and behavior monitoring. The result of this effort will be a massive data collection. Our smart homes generate an average of 5,000 sensor events each day. When smart phone and power meter readings are added, the volume of data will dramatically increase. REU students will design graph-based representations of this data to improve recognition and prediction of activities and energy usage.
Parallel community detection
Graph-based computations are pervasive in a number of scientific applications and one of the most heavily sought after operation is that of identifying community structures within real world networks/graphs. We are working on parallelizing the community detection operation for massive real world networks. This work is focused on developing scalable/parallel algorithms and software that can take advantage of modern day multicore and manycore architectures. We propose to involve undergraduate student researchers to study the peculiarities involved in parallel programming on these emerging architectures, and pass on that learning experience to the project's graduate students who are working on designing efficient algorithms for these platforms. Such a task will also lead to training of the undergraduate student in high performance computing (which is part of an NSF-funded initiative under the TCPP/Parallel and Distributed Computing program).
Topological data analysis
Understanding the structure inherent in data is key to developing advanced analytic functions in domains containing complex data sets. We are investigating approaches that use recent mathematical results from the domain of algebraic topology in analyzing bioinformatics data sets that have shown to be highly complex both by their sheer scale and the number of dimensions they encode. Undergraduate student researchers will help implement the basic core routines of these analysis pipelines and will help graduate students in testing their analysis through data collection. This will also provide a valuable training for the undergraduate researchers in the broad area of data analytics while potentially motivating them to a research based career in this area.
Automated Assessment
REU students will perform side-by-side comparisons of automated functional assessment with assessment based on self reports and assessment based on laboratory testing. Both healthy older adults and individuals with cognitive impairments have completed scripted everyday activities in our smart home testbed (e.g., make oatmeal, fill a medication dispenser). They have also completed self-report questionnaires about their everyday activities and been assessed with clinical and performance-based measures of everyday activities in our laboratory. Where there are unexpected misalignments in assessment results between the three methods the REU students will further investigate to discover the sources of the inconsistency. The results will allow us to improve questionnaire measures and laboratory-based assessment and to design smart environment algorithms that more thoroughly assess the physical and cognitive well being of individuals.
Automated Interventions
REU students will experiment with prompt-based interventions that combine cognitive rehabilitation theory with automated interventions to deliver support for cognitive difficulties within the smart environment. REU students will participate in collecting and analyzing data relevant to several questions (a) Are participants more compliant when prompts are delivered during activity transitions rather than being time-based? (b) How can we incorporate motivators into our prompting system to produce high levels of compliance? (c) Do older adults and those with cognitive impairment prefer verbal prompts or multi-modal-prompts? And (d) Do older adults and those with cognitive impairment prefer tablet-based prompts or phone-based prompts? REU students will also have the opportunity to participate in our iterative participatory design studies with users to construct the best interfaces for our prompting interventions and for our digital memory notebook.
Evaluation of AI Systems
Modern AI systems are typically unable to generalize outside of their limited domain. This is a result of both the lack of diversity in tests and in AI systems. In most cases, tests do not combine different and varying aspects of intelligence, offering instead just one particular metric. This encourages the use of specialized AI systems for a very narrow set of problems. The AI community does not currently have a generally-accepted method to measure the general intelligence of an AI system. A measure of general machine intelligence would allow us to compare different AI methods, provide guidance on which areas of AI need further development, and facilitate the assessment of AI progress over time. To address these challenges we are developing a measure of general machine intelligence, called AIQ, that can serve as an effective benchmark for evaluating and comparing the general intelligence of AI systems. REU students can particpating in the project in several ways. They can compare our approach to other AI system evaluation methods (e.g.,. Open AI Gym, VizDoom, Obstacle Tower). They can integrate new tests into our AIQ test suite. And they can design and evaluate AI agents to perform the tests.