GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization

Abstract

We present the first multi-year mobile sensing datasets. Our multi-year data collection studies span four years (10 weeks each year, from 2018 to 2021). The four datasets contain data collected from 705 person-years (497 unique participants) with diverse racial, ability, and immigrant backgrounds. Each year, participants would install a mobile app on their phones and wear a fitness tracker. The app and wearable device passively track multiple sensor streams in the background 24×7, including location, phone usage, calls, Bluetooth, physical activity, and sleep behavior. In addition, participants completed weekly short surveys and two comprehensive surveys on health behaviors and symptoms, social well-being, emotional states, mental health, and other metrics. Our dataset analysis indicates that our datasets capture a wide range of daily human routines, and reveal insights between daily behaviors and important well-being metrics (e.g., depression status). We envision our multi-year datasets can support the ML community in developing generalizable longitudinal behavior modeling algorithms.

Background

Among various longitudinal sensor streams, smartphones and wearables are arguably one of the most widely available data sources [1]. The advances in mobile technology provide an unprecedented opportunity to capture multiple aspects of daily human behaviors, by collecting continuous sensor streams from these devices [2,3], together with metrics about health and well-being through self-report or clinical diagnosis as modeling targets. It poses unique challenges compared to traditional time-series classification tasks. First, the data covers a much longer time period, usually across multiple months or years. Second, the nature of longitudinal collection often results in a high data missing rate. Third, the prediction target label is sparse, especially for mental well-being metrics.

Longitudinal human behavior modeling is an important multidisciplinary area spanning machine learning, psychology, human-computer interaction, and ubiquitous computing. Researchers have demonstrated the potential of using longitudinal mobile sensing data for behavior modeling in many applications, e.g., detecting physical health issues [4], monitoring mental health status [3], measuring job performance [5], and tracing education outcomes [6]. Most existing research employed off-the-shelf ML algorithms and evaluated them on their private datasets. However, testing a model with new contexts and users is imperative to ensure its practical deployability. To the best of our knowledge, there has been no investigation of the cross-dataset generalizability of these longitudinal behavior models, nor an open testbed to evaluate and compare various modeling algorithms. To address this gap, we present the first multi-year passive mobile sensing datasets to help the ML community explore generalizable longitudinal behavior models.

Methods

Our data collection studies were conducted at a Carnegie-classified R-1 university in the United State with an IRB review and approval. We recruited undergraduates via emails from 2018 to 2021. After the first year, previous-year participants were invited to join again. The study was conducted during Spring quarter for 10 weeks each year, so the impact of seasonal effects was controlled. Based on their compliance, participants received up to $245 in compensation every quarter.

The four datasets (DS1 to DS4) have 155, 218, 137, and 195 participants (705 person-years overall, and 497 unique people). Our datasets have a high representation of females (58.9%), immigrants (24.2%), first-generations (38.2%), and disability (9.1%), and have a wide coverage of races, with Asian (53.9%) and White (31.9%) being dominant (e.g., Hispanic/Latino 7.4%, Black/African American 3.3%).

Part 1: Survey Data

We collected survey data at multiple stages of the study. We delivered extensive surveys before the start and at the end of the study (pre/post surveys) and short weekly Ecological Momentary Assessment (EMA) surveys during the study to collect in-the-moment self-report data. All surveys consist of well-established and validated questionnaires to ensure data quality.

Our pre/post surveys include a number of questionnaires to cover various aspects of life, including 1) personality (BFI-10, The Big-Five Inventory-10), 2) physical health (CHIPS, Cohen-Hoberman Inventory of Physical Symptoms), 3) mental well-being (e.g., BDI-II, Beck Depression Inventory-II; ERQ, Emotion Regulation Questionnaire), and 4) social well-being (e.g., Sense of Social and Academic Fit Scale; EDS, Everyday Discrimination Scale). Our EMA surveys focus on capturing participants’ recent sense of their mental health, including PHQ-4, Patient Health Questionnaire 4; PSS-4, Perceived Stress Scale 4; and PANAS, Positive and Negative Affect Schedule.

We use the depression detection task as a starting point for behavior modeling. We employ BDI-II (post) and PHQ-4 (EMA) as the ground truth. Both are screening tools for further inquiry of clinical depression diagnosis. We focus on a binary classification problem to distinguish whether participants’ scores indicate at least mild depressive symptoms through the scales (i.e., PHQ-4 > 2, BDI-II > 13). The average number of depression labels is 11.6 ± 2.6 per person. The percentage of participants with at least mild depression is 39.8 ± 2.7% for BDI-II and 46.2 ± 2.5% for PHQ-4.

Due to some design iteration, we did not include PHQ-4 in DS1, but only PANAS. Although PANAS contains questions related to depressive symptoms (e.g., “distressed”), it does not have a comparable theoretical foundation for depression detection like PHQ-4 or BDI-II. Therefore, to maximize the compatibility of the datasets, we trained a small ML model on DS2 that has both PANAS and PHQ-4 scores to generate reliable ground truth labels. Specifically, we used a decision tree (depth=2) to take PNANS scores on two affect questions (“depressed” and “nervous”) as the input and predict PHQ-4 score-based depression binary label. Our model achieved 74.5% and 76.3% for accuracy and F1-score on a 5-fold cross-validation on DS2. The rule from the decision tree is simple: the user would be labeled as having no depression when the distress score is less than 2, and the nervous score is less than 3 (on a 1-5 Likert Scale). We then applied this rule to DS1 to generate depression labels.

Part 2: Sensor Data

We developed a mobile app using the AWARE Framework [7] that continuously collects location, phone usage (screen status), Bluetooth scans, and call logs. The app is compatible with both the iOS and Android platforms. Participants installed the app on smartphones and left it running in the background. In addition, we provided wearable Fitbits to collect their physical activities and sleep behaviors. The mobile app and wearable passively collected sensor data 24×7 during the study. The average number of days per person per year is 77.5 ± 8.9 among the four datasets.

We strictly follow our IRB's rules for anonymizing participants' data. Specifically, we employed a PID as the only indicator of a participant. No personal information is included in the dataset. Since some sensitive sensor data (e.g., location) can disclose identities, we only release feature-level data under credentialing to protect against privacy leakage.

Moreover, the data collection dates are randomly shifted by weeks. Therefore, the temporal order of events within the same subject and the day of the week are maintained after date-shifting.

Data Description

We release four datasets, named INS-W_1, INS-W_2, INS-W_3, and INS-W_4. A dataset has three folders. We provided an overview description below. Please refer to our GLOBEM home page [8] GitHub README page [9] for more details.

• SurveyData: a list of files containing participants' survey responses, including pre/post long surveys and weekly short EMA surveys.
• FeatureData: behavior feature vectors from all data types, using RAPIDS [10] as the feature extraction tool.
• ParticipantInfoData: some additional information about participants, e.g., device platform (iOS or Android).

Specifically, the folder structure of a dataset folder is shown as follows:

• SurveyData
  • dep_weekly.csv
  • dep_endterm.csv
  • pre.csv
  • post.csv
  • ema.csv
• FeatureData
  • rapids.csv
  • location.csv
  • screen.csv
  • call.csv
  • bluetooth.csv
  • steps.csv
  • sleep.csv
  • wifi.csv
• ParticipantsInfoData
  • platform.csv

Survey Data

The SurveyData folder contains five files, all indexed by pid and date:

• dep_weekly.csv: The specific file for depression labels (column "dep") combining post and EMA surveys.
• dep_endterm.csv: The specific file for depression labels (column "dep") only in post surveys. Some prior depression detection tasks focus on end-of-term depression prediction.
  These two files are created for depression as it is the benchmark task. We envision future work can be extended to other modeling targets as well.
• pre.csv: The file contains all questionnaires that participants filled in right before the start of the data collection study (thus pre-study).
• post.csv: The file contains all questionnaires that participants filled in right after the end of the data collection study (thus post-study).
• ema.csv: The file contains all EMA surveys that participants filled in during the study. Some EMAs were delivered on Wednesdays, while some were delivered on Sundays.

Survey List

PS: Due to the design iteration, some questionnaires are not available in all studies. Moreover, some questionnaires have different versions across years. We clarify them using column names. For example, INS-W_2 only has "CESD_9items_POST", while others have "CESD_10items_POST". "CESD_9items_POST" is also calculated in other datasets to make the modeling target comparable across datasets.

Feature Data

The FeatureData folder contains seven files, all indexed by pid and date.

• rapids.csv: The complete feature file that contains all features.
• location.csv: The feature file that contains all location features.
• screen.csv: The feature file that contains all phone usage features.
• call.csv: The feature file that contains all call features.
• bluetooth.csv: The feature file that contains all Bluetooth features.
• steps.csv: The feature file that contains all physical activity features.
• sleep.csv: The feature file that contains all sleep features.
• wifi.csv: The feature file that contains all WiFi features. Note that this feature type is not used by any existing algorithms and often has a high data missing rate.

Please note that all features are extracted with multiple time_segments

• morning (6 am - 12 pm, calculated daily)
• afternoon (12 pm - 6 pm, calculated daily)
• evening (6 pm - 12 am, calculated daily)
• night (12 am - 6 am, calculated daily)
• allday (24 hrs from 12 am to 11:59 pm, calculated daily)
• 7-day history (calculated daily)
• 14-day history (calculated daily)
• weekdays (calculated once per week on Friday)
• weekend (calculated once per week on Sunday)

For all features with numeric values, we also provide two more versions:

• normalized: subtracted by each participant's median and divided by the 5-95 quantile range
• discretized: low/medium/high split by 33/66 quantile of each participant's feature value

Naming Format

All features follow a consistent naming format:

[feature_type]:[feature_name][version]:[time_segment]

• feature_type: It corresponds to the six data types.
  • location - f_loc
  • screen - f_screen
  • call - f_call
  • bluetooth - f_blue
  • steps - f_steps
  • sleep - f_slp.
• feature_name: The name of the feature provided by RAPIDS, i.e., the second column of the following figure, plus some additional information. A typical format is [SensorType]_[CodeProvider]_[featurename]. Please refer to RAPIDS's naming format [9] for more details.
• version: It has three versions:
  • 1) nothing, just empty "";
  • 2) normalized, _norm;
  • 3) discretized, _dis.
• time_segment: It corresponds to the specific time segment.
  • morning - morning
  • afternoon - afternoon
  • evening - evening
  • night - night
  • allday - allday
  • 7-day history - 7dhist
  • 14-day history - 14dhist
  • weekday - weekday
  • weekend - weekend

A participant's "sumdurationunlock" normalized feature in mornings is "f_loc:phone_screen_rapids_sumdurationunlock_norm:morning".

Please find the following tables about feature details in our datasets.

Location Details

Phone Usage Details

Call Details

Bluetooth Details

WiFi Details

Physical Activity Details

Sleep Details

Participant Info Data

The ParticipantInfoData folder contains files with additional information.

• platform.csv: The file contains each participant's major smartphone platform (iOS or Android), indexed by pid
• demographics.csv: Due to privacy concerns, demographic data are only available for special requests. Please reach out to us directly with a clear research plan with demographic data.

Usage Notes

We provide a behavior modeling benchmark platform GLOBEM [8,9]. The platform is designed to support researchers in using, developing, and evaluating different longitudinal behavior modeling methods.

Researchers who use the datasets must agree to the following terms.

Privacy
Although the database has been anonymized, we cannot eliminate all potential risks of privacy information leakage. The PI of any research group access to the dataset, is responsible for continuing to safeguard this database, taking whatever steps are appropriate to protect participants’ privacy and data confidentiality. The specific actions required to safeguard the data may change over time.

Misuse
If at any point, the administrators of the datasets at the University of Washington have concerns or reasonable suspicions that the researcher has violated these usage note, the researcher will be notified. Concerns about misuse may be shared with PhysioNet and other related entities.

Our datasets have led to multiple publications. Please find them in the reference list [11-18].

Release Notes

v1.0 - Release of our GLOBEM Dataset

Ethics

Our datasets aim at aiding research efforts in the area of developing, testing, and evaluating machine learning algorithms to better understand college students’ (and the potentially more general population) daily behaviors, health, and well-being from continuous sensor streams and self-reports. These findings may support public interest in how to improve student experiences and drive policy around adverse events students and others may experience.

Privacy is the major ethical concern of our data collection studies. Our study has obtained IRB approval from the University of Washington with the IRB number STUDY00003244. Participants signed the consent form before joining our study. We strictly follow the IRB rules to anonymize participants' data. Anyone outside our core data collection group cannot access direct individually-identifiable information. We also eliminated the data for users who stopped their participation at any time during the study. Since some sensitive sensor data (e.g., location) can disclose identities, we only release feature-level data under credentialing to protect against privacy leakage. 

Acknowledgements

Our multi-year data collection study closely followed a sister study at Carnegie Mellon University (CMU). We acknowledge all efforts from CMU Study Team to provide important starting and reference materials. Moreover, our studies were greatly inspired by StudentLife researchers from Dartmouth College.

Our studies were supported by the University of Washington (including the Paul G. Allen School of Computer Science and Engineering; Department of Electrical and Computer Engineering; Population Health; Addictions, Drug and Alcohol Institute; and the Center for Research and Education on Accessible Technology and Experiences); the National Science Foundation (EDA-2009977, CHS-2016365, CHS-1941537, IIS1816687 and IIS7974751), the National Institute on Disability, Independent Living and Rehabilitation Research (90DPGE0003-01), Samsung Research America, and Google.

Conflicts of Interest

The authors have no conflicts of interest to declare.

References

1. N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T. Campbell. A survey of mobile phone sensing. IEEE Communications Magazine, 48(9), 2010.
2. M. E. Morris, Q. Kathawala, T. K. Leen, E. E. Gorenstein, F. Guilak, W. DeLeeuw, and M. Labhard. Mobile therapy: case study evaluations of a cell phone application for emotional self-awareness. Journal of medical Internet research, 12(2):e10, 2010.
3. R. Wang, F. Chen, Z. Chen, T. Li, G. Harari, S. Tignor, X. Zhou, D. Ben-Zeev, and A. T. Campbell. Studentlife: Assessing mental health, academic performance and behavioral trends of college students using smartphones. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 3–14. ACM, 2014.
4. J.-K. Min, A. Doryab, J. Wiese, S. Amini, J. Zimmerman, and J. I. Hong. Toss “n” turn: Smartphone as sleep and sleep quality detector. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, page 477–486, New York, NY, USA, 2014. Association for Computing Machinery.
5. S. M. Mattingly, J. M. Gregg, P. Audia, A. E. Bayraktaroglu, A. T. Campbell, N. V. Chawla, V. Das Swain, M. De Choudhury, S. K. D’Mello, A. K. Dey, et al. The tesserae project: Large-scale, longitudinal, in-situ, multimodal sensing of information workers. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–8, 2019.
6. R. Wang, G. Harari, P. Hao, X. Zhou, and A. T. Campbell. Smartgpa: how smartphones can assess and predict academic performance of college students. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing, pages 295–306, 2015.
7. D. Ferreira, V. Kostakos, and A. K. Dey. Aware: Mobile context instrumentation framework. Frontiers in ICT, 2:6, 2015.
8. GLOBEM Home Page. https://the-globem.github.io
9. Benchmark Platform GLOBEM. https://github.com/UW-EXP/GLOBEM/
10. Rapids documentation. https://www.rapids.science/1.6/
11. M. E. Morris, K. S. Kuehn, J. Brown, P. S. Nurius, H. Zhang, Y. S. Sefidgar, X. Xu, E. A. Riskin, A. K. Dey, S. Consolvo, and J. C. Mankoff. College from home during COVID-19: A mixed-methods study of heterogeneous experiences. PLOS ONE, 16(6):e0251580, June 2021.
12. P. S. Nurius, Y. S. Sefidgar, K. S. Kuehn, J. Jung, H. Zhang, O. Figueira, E. A. Riskin, A. K. Dey, and J. C. Mankoff. Distress among undergraduates: Marginality, stressors and resilience resources. Journal of American College Health, pages 1–9, July 2021.
13. Y. S. Sefidgar, W. Seo, K. S. Kuehn, T. Althoff, A. Browning, E. Riskin, P. S. Nurius, A. K. Dey, and J. Mankoff. Passively-sensed Behavioral Correlates of Discrimination Events in College Students. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1–29, Nov. 2019.
14. X. Xu, P. Chikersal, A. Doryab, D. K. Villalba, J. M. Dutcher, M. J. Tumminia, T. Althoff, S. Cohen, K. G. Creswell, J. D. Creswell, J. Mankoff, and A. K. Dey. Leveraging Routine Behavior and Contextually- Filtered Features for Depression Detection among College Students. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3(3):1–33, Sept. 2019.
15. X. Xu, P. Chikersal, J. M. Dutcher, Y. S. Sefidgar, W. Seo, M. J. Tumminia, D. K. Villalba, S. Cohen, K. G. Creswell, J. D. Creswell, A. Doryab, P. S. Nurius, E. Riskin, A. K. Dey, and J. Mankoff. Leveraging Collaborative-Filtering for Personalized Behavior Modeling: A Case Study of Depression Detection among College Students. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(1):1–27, Mar. 2021.
16. X. Xu, J. Mankoff, and A. K. Dey. Understanding practices and needs of researchers in human state modeling by passive mobile sensing. CCF Transactions on Pervasive Computing and Interaction, July 2021.
17. H. Zhang, M. E. Morris, P. S. Nurius, K. Mack, J. Brown, K. S. Kuehn, Y. S. Sefidgar, X. Xu, E. A. Riskin, A. K. Dey, and J. Mankoff. Impact of Online Learning in the Context of COVID-19 on Undergraduates with Disabilities and Mental Health Concerns. ACM Transactions on Accessible Computing, page 3538514, July 2022.
18. H. Zhang, P. Nurius, Y. Sefidgar, M. Morris, S. Balasubramanian, J. Brown, A. K. Dey, K. Kuehn, E. Riskin, X. Xu, and J. Mankoff. How Does COVID-19 impact Students with Disabilities/Health Concerns? In arXiv. arXiv, May 2020. arXiv:2005.05438. [cs]