In this episode, we are joined by Rebecca Kantar, Vice President of Education at Roblox. Rebecca shares her experiences and insights into the world of game-based and simulation-based learning and assessment. She digs into the benefits and limitations of these approaches as well as the exciting opportunities she sees for the future.
So our big mistake is thinking that people who are good at solving multiple-choice problems that are fairly trivial, that have a unique correct answer, and that measure things that are never going to matter again in your whole life, that that is going to be predictive of who is going to be able to handle really tough, unstructured, ambiguous, real-world problems.
Dr. Robert Sternberg, former president of the American Psychological Association and current professor at Cornell University
Resources
The following resources are available from AVID and on AVID Open Access to explore related topics in more depth:
- Game-Based Learning as a Medium to Connect With Every Child, with Tammie Schrader (podcast episode)
- Girls Who Game: Equity in STEM, with Dr. Kendall Latham (podcast episode)
- Assessment as Revelation, with Dr. Kelly Camak Niccolls (podcast episode)
- Accelerate Learning With Meaningful, Targeted Assessment (article)
- Inspire Students With Project-Based Learning (article collection)
- Engage Students Through Inquiry Learning (article collection)
- Project-Based Learning, with A.J. Juliani (podcast episode)
Appropriate, Relevant, and Interesting
Three words repeatedly come to the forefront of our conversation about game-based and simulation-based learning: appropriate, relevant, and interesting. These words encapsulate why these approaches are so powerful and so challenging at the same time to create. Our guest, Rebecca Kantar, has experience with developing these types of products and says, “I’m sympathetic to how difficult it is to build something that is fair, valid, reliable, highly scaled, efficient to create and administer, and achieves those things.”
Rebecca talks about the experiences she wants for students using well-designed simulations, saying, “I wanna watch you solve problems. I want to watch you understand systems. I want to watch you create and innovate based on what you see happening in the environment and the data you get back.” It takes a large team to make this type of experience a reality. Rebecca reflects on what it takes to create these experiences, saying, “I have a whole gaming team, and I have a whole learning science, psychometrics data science team. Those folks need to work together at every single stage of creating one of these immersive game-based or simulation-based tasks.”
Rebecca adds, “You have an opportunity in game-based and simulation-based assessment to absolutely look at the deeper and the more critical thinking, but only if you can engineer your assessments to capture that in a fair, valid, and reliable way.” The following are a few highlights from our conversation:
- About Our Guest: Rebecca Kantar is the Vice President of Education at Roblox. Before her time at Roblox, she was a student at Harvard and an entrepreneur who founded two start-up companies.
- Authentic Challenges: Finding classwork in college to be repetitive and monotonous, Rebecca took a break from school to engage in more authentic, real-world challenges and problem-solving, including social-impact work for a nonprofit. She says, “I loved the complexity and the authenticity . . . of the challenges that I was confronting in that work, and I eventually felt like that nature of work where I was constantly thrown into different types of problem-solving situations, where I was having some traceable impact, where I was constantly pushed to demonstrate and try new sets of skills, that’s what I was drawn to, and my classwork was no longer providing that, so I took some time off from school.”
- Building Companies: After leaving school, Rebecca became an entrepreneur, creating two new start-up companies: a networking company that she would sell to the Gerson Lehrman Group and a game-based/simulation-based assessment company, Imbellus, which was later acquired by Roblox.
- An Authentic Goal: Rebecca’s work at Imbellus was founded in real-world problem-solving. She asked herself, “What would it take for our system, on the whole, to raise the collective floor for all students, but also to make sure that students felt very connected to the nature of work and opportunity they were going to face after school . . . such that they would feel really equipped for adulthood?”
- Advantages of Traditional Assessments: Rebecca acknowledges that traditional assessments have their advantages, including “security and robustness against cheating and coaching” as well as “the efficiency both of test creation and of scoring at scale.”
- Challenging Products to Create: Game-based and simulation-based assessments are complex, challenging, and expensive to create. Every variable must be evaluated to determine if the assessment is accurate and fair for all test-takers. Rebecca explains, “Those types of scores—how we process these long strings of clicks, or time stamps, or mouse movements—were really finicky to create, so it took us a long time to get practiced and skilled at separating out that kind of process thinking from an outcome in one of the scenarios we built, and we call those product scores.”
- 21st Century Skills: Rebecca shares, “When you get it right, students also have a lot more dynamism in the learning environment/assessment environment.” Reviewing how students work through these virtual experiences can give educators better insights into a lot of the 21st century skills we want for our students, such as adaptability, resilience, and persistence. She adds, “You have an opportunity in game-based and simulation-based assessment to absolutely look at the deeper and the more critical thinking, but only if you can engineer your assessments to capture that in a fair, valid, and reliable way.”
- Formative Assessment: Rebecca says, “These tests can thrive in a high-stakes environment, but I think they are, perhaps, even better in a formative environment.”
- Roblox: Rebecca shares that Roblox is a “massive online community of millions of creators and millions of community members of the Roblox ecosystem who are able to engage in digital co-experiences. That might be playing games. It might be exploring branded experiences, going to concerts or a fashion experience, or, in our case, learning with Roblox. Roblox is a platform that brings people together through play, through co-experience, through co-creation.”
- Roblox Studio: Roblox Studio is a free game-building tool offered on their website and is a “fantastic way to learn computer science and basic coding skills because you instantly get to see the fruits of your labor, and all of your friends get to experience them with you, so it’s a very rewarding process and feedback mechanism.”
- The Education Side of Roblox: The education arm of Roblox offers many resources for developers, educators, and families. Developers can build with Roblox Studio, educators or providers can partner with Roblox through grant opportunities, and teachers or parents can access free curricular materials posted on the Roblox website. Rebecca says, “Everyone has the capacity to jump into Roblox Studio. Check out some of our free online materials. Check out some of the tutorial videos. Many folks teach with it, so you can learn and have the opportunity to start creating yourself.”
- The Future: Rebecca is excited to see how generative artificial intelligence (AI) will impact the development of game-based and simulation-based experiences. She believes that it can reduce the amount of resources needed to produce these complex tools and potentially make them more affordable and practical to create.
- AI for Students: In addition to revolutionizing the creation of games and simulations, Rebecca believes that students must be exposed to AI and empowered to use it. She says we need to “embrace it, rather than shut it out, as an opportunity for us to expect even more of what they are capable of doing now that they have this very handy and dynamic assistant.”
Guiding Questions
If you are listening to the podcast with your instructional team or would like to explore this topic more deeply, here are guiding questions to prompt your reflection:
- What has your experience been with traditional high-stakes assessments?
- What is your experience with game-based or simulation-based learning platforms or assessments?
- What are the benefits of both traditional and game-based/simulation-based assessments?
- What are the weaknesses of each of these types of assessments?
- How do you see game-based or simulation-based assessments impacting education moving forward?
- What impact might generative AI have on the assessment landscape?
Extend Your Learning
- Roblox (official webpage)
- Roblox Education (official webpage)
- Roblox Studio (official webpage)