ECR Betty’s Brain-BROMP
Modeling Self Regulated Learning with Betty’s Brain
Overview:
The primary goal for this project is to enhance the theory and measurement of students’ self-regulated learning (SRL) processes during science learning by developing a technology based framework which leverages human expert judgment and machine learning methods to identify key moments during SRL and analyze these moments in depth.
These issues are being studied in the context of Betty’s Brain, an open-ended science learning environment that combines learning-by-modeling with critical thinking and problem-solving skills to teach complex science topics.
We are developing a novel measurement framework that will identify key inflection points in learners’ SRL processes that drive their cognition, metacognition and affect, and inform researchers about these inflection points in real time, so that the researchers can conduct observations and interviews right at the critical moments when the change is occurring. The results of this data collection will be integrated with sequential pattern mining to drive the refinement and extension of an existing theory of SRL (Winne’s COPES framework) to increase its specificity and predictive power.
Funding agency:
Status:
- Analysis of data on student performance, behavior and affect collected during Study 1 (March 2017 – four 6th grade classrooms of Meigs Middle School, Nashville, TN)
- Development of automated detectors for real-time detection of key inflection points in students’ SRL processes (using pattern mining, feature engineering & deep learning approaches)
- Design of Study 2, including –
- Development of an android app (QRF) that notifies classroom researchers, in real time, of inflection points in students’ SRL behaviors
- Development of the app-server-clients communication framework, to prioritize and send detected SRL inflection points from student machines (running Betty’s Brain) to researcher android-phones (running QRF app) via a server
- Pilot study (May 2018 – Jere Baxter Middle School, Nashville, TN) to test the new framework in a classroom setting)
- Deployment of the full framework in Study 2 (Dec 2018 – Meigs Middle School, Nashville, TN) to collect SRL data (on student cognition, affect, metacognition & motivation)
- Addition of new learner scaffolds in Betty’s Brain based on analysis of SRL data collected in Study 2
- Study 3 (Feb 2019 – Meigs Middle School, Nashville, TN), to analyze student responses to the newly-added SRL scaffolds
Researchers currently involved:
Vanderbilt University: Prof Gautam Biswas, Anabil Munshi, Marian Rushdy, Allison Moore
University of Pennsylvania: Prof Ryan Baker, Jaclyn Ocumpaugh, Stefan Slater
University of Illinois at Urbana-Champaign: Prof Luc Paquette, Nigel Bosch
External collaborator for QRF app design: Martin van Velsen
Systems used:
- Betty’s Brain,
- HART (android app used for collecting BROMP data),
- QRF (android app for study 2)
Related publications:
- Munshi, A., Rajendran, R., Moore, A., Ocumpaugh, J., & Biswas, G. (2018). Studying the Interactions between Components of Self-Regulated Learning in Open Ended Learning Environments. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS), London.
- Munshi, A., Rajendran, R., Ocumpaugh, J., Biswas, G., Baker, R. S., & Paquette, L. (2018). Modeling Learners Cognitive and Affective States to Scaffold SRL in Open Ended Learning Environments. In Proceedings of the International Conference on User Modelling, Adaptation and Personalization (UMAP), Singapore.
- Jiang, Y., Bosch, N., Baker, R. S., Paquette, L., Ocumpaugh, J., Andres, J. M. A. L., Moore, A. L., & Biswas, G. (2018) Expert feature-engineering vs. deep neural networks: Which is better for sensor-free affect detection?. In U. Hoppe, C. Rosé, & R. Martinez (Eds.), Proceedings of the 19th International Conference on Artificial Intelligence in Education (AIED 2018). Berlin Heidelberg: Springer-Verlag.
- Andres, A., Ocumpaugh, J., Baker, R.S., Slater, S., Paquette, L., Jiang, Y., Bosch, N., Munshi, A., Moore, A., & Biswas, G. (2019). Affect Sequences and Learning in Betty’s Brain. In Proceedings of the 9th International Learning Analytics and Knowledge (LAK) Conference, Tempe, Arizona.
- Munshi, A., & Biswas, G. (2019). Personalization in OELEs: Developing A Data-Driven Framework to Model and Scaffold SRL Processes. In Proceedings of the 20th International Conference on Artificial Intelligence in Education (AIED 2019), Chicago, pp. 354-358.
- Munshi, A., Mishra, S., Zhang, N., Paquette, L., Ocumpaugh, J., Baker, R., & Biswas, G. (2020). Modeling the Relationships Between Basic and Achievement Emotions in Computer-Based Learning Environments. In: Bittencourt I., Cukurova M., Muldner K., Luckin R., Millán E. (eds) Artificial Intelligence in Education. AIED 2020. Lecture Notes in Computer Science, vol 12163. Springer, Cham.
- Zhang, Y., Paquette, L., Baker, R.S., Ocumpaugh, J., Bosch, N., Munshi, A., & Biswas, G. (2020). The relationship between confusion and metacognitive strategies in Betty’s Brain. In Proceedings of the 10th International Learning Analytics and Knowledge (LAK) Conference, Frankfurt, Germany.
- Paquette, L., Grant, T., Zhang, Y., Biswas, G., Baker, R.S. (2020). Using epistemic networks to analyze self-regulated learning in an open-ended problem-solving environment. Proceedings of the 2nd International Conference on Quantitative Ethnography, 185-201.
- Bosch, N., Zhang, Y., Paquette, L., Baker, R.S., Ocumpaugh, J., Biswas, G. (2021) Students’ Verbalized Metacognition during Computerized Learning. Proceedings of ACM SIGCHI: Computer-Human Interaction.
- Hutt, S., Ocumpaugh, J., Andres, J.M.A.L., Bosch, N., Paquette, L., Biswas, G., Baker, R.S. (2021) Sharpest Tool in the Shed: Investigating SMART Models of Self-Regulation and their Impact on Learning. Proceedings of the International Conference on Educational Data Mining.
- Baker, R.S., Nasiar, N., Ocumpaugh, J.L., Hutt, S., Andres, J.M.A.L., Slater, S., Schofield, M., Moore, A., Paquette, L., Munshi, A., Biswas, G. (2021) Affect-Targeted Interviews for Understanding Student Frustration. Proceedings of the International Conference on Artificial Intelligence and Education. [Won Best Paper Award]
- Hutt, S., Ocumpaugh J., Andres, J.M.A.L., Munshi, A., Bosch, N., Baker, R.S., Zhang, Y., Paquette, L., Slater, S., Biswas, G. (in press) Who’s Stopping You? – Using Microanalysis to Explore the Impact of Science Anxiety on Self Regulated Learning Operations. To appear in Proceedings of the 42nd Annual Meeting of the Cognitive Science Society.
- Ocumpaugh, J., Hutt, S., Andres, J.M.A.L., Baker, R.S., Biswas, G., Bosch, N., Paquette, L., Munshi, A. (in press) Using Qualitative Data from Targeted Interviews to Inform Rapid AIED Development. To appear in Proceedings of the 29th International Conference on Computers in Education.
- Hutt, S., Baker, R.S., Ocumpaugh, J., Munshi, A., Andres, J.M.A.L., Karumbaiah, S., Slater, S., Biswas, G., Paquette, L., Bosch, N., van Velsen, M. (in press) Quick Red Fox: An App Supporting a New Paradigm in Qualitative Research on AIED for STEM. To appear in Ouyang, F., Jiao, P., McLaren, B.M., Alavi, A.H. (Eds.) Artificial Intelligence in STEM Education: The Paradigmatic Shifts in Research, Education, and Technology.
- Zhang, Y., Paquette, L., Baker, R.S., Ocumpaugh, J., Bosch, N., Biswas, G., Munshi, A. (in press) Can strategic behavior facilitate confusion resolution? The interplay between confusion and metacognitive strategies in Betty’s Brain. To appear in Journal of Learning Analytics.