WEBVTT 00:00:09.000 --> 00:00:11.000 keep admitting people but it looks like we're right 00:00:11.000 --> 00:00:13.000 at 11:00. » TOM HELIKAR: Good morning, 00:00:13.000 --> 00:00:18.000 good afternoon and good evening to everybody. Welcome 00:00:18.000 --> 00:00:22.000 back to our last day of our cross 00:00:22.000 --> 00:00:28.000 disciplinary research conference. Today's 00:00:28.000 --> 00:00:33.000 programming will start with our keynote speaker, Dr. Kathy 00:00:33.000 --> 00:00:35.000 Perkins. D r. Perkins is a associate professor in physics 00:00:35.000 --> 00:00:40.000 in university of 00:00:40.000 --> 00:00:42.000 Colorado-boulder. Also the director of PhET interactive 00:00:42.000 --> 00:00:45.000 simulations project and the director of university of 00:00:45.000 --> 00:00:50.000 Colorado science education initiative. 00:00:50.000 --> 00:00:56.000 Dr. Perkins received her bachelor's degree in physics, 00:00:56.000 --> 00:01:01.000 master's in chemistry and Ph.D . in atmospheric sciences from 00:01:01.000 --> 00:01:08.000 Harvard university. And she joined the PhET Raj in 2003 00:01:08.000 --> 00:01:13.000 and then since 2008 she has been the director of these 00:01:13.000 --> 00:01:16.000 efforts. Her research, her education research has 00:01:16.000 --> 00:01:21.000 focused on 00:01:21.000 --> 00:01:25.000 pedogogically effective design and use of interactive s 00:01:25.000 --> 00:01:27.000 imulations, sustainable course reform, students beliefs about 00:01:27.000 --> 00:01:31.000 science and institutional change. Thank you for joining 00:01:31.000 --> 00:01:33.000 us today. We look forward to your keynote presentation. 00:01:33.000 --> 00:01:37.000 » KATHY PERKINS, Ph.D.: Thank you very much. It's great to 00:01:37.000 --> 00:01:40.000 be here and have the chance to talk with everyone. Let me go 00:01:40.000 --> 00:01:52.000 ahead and start by sharing my screen. 00:01:52.000 --> 00:01:54.000 So can everybody hear me okay? We're good. 00:01:54.000 --> 00:01:57.000 » STEPHANIE VENDETTI: Sounds good. 00:01:57.000 --> 00:02:02.000 » KATHY PERKINS, Ph.D.: Okay. All right. So today I'm going 00:02:02.000 --> 00:02:06.000 to focus on a new effort we have around expanding what 00:02:06.000 --> 00:02:12.000 simulations can do and how they can fit into the 00:02:12.000 --> 00:02:14.000 education ecosystem. And also can inspire new ways of doing 00:02:14.000 --> 00:02:16.000 research and 00:02:16.000 --> 00:02:20.000 new opportunities around researching interactive 00:02:20.000 --> 00:02:24.000 simulations. I'm going to start with a super brief 00:02:24.000 --> 00:02:27.000 introduction. I think most people are familiar with the 00:02:27.000 --> 00:02:31.000 basics much PhET since we've been around for a while. Our 00:02:31.000 --> 00:02:35.000 project that started in 2002 and we have both a product 00:02:35.000 --> 00:02:37.000 development side where we're designing and building and 00:02:37.000 --> 00:02:41.000 disseminating interactive simulations. And then our 00:02:41.000 --> 00:02:45.000 group has a research side where we're looking at the 00:02:45.000 --> 00:02:48.000 design of assimilation specifically but looking how 00:02:48.000 --> 00:02:55.000 simulations can be integrated into the classroom and how 00:02:55.000 --> 00:02:58.000 that can impact students, student interactivities, 00:02:58.000 --> 00:03:00.000 student teacher interaction and interaction between the 00:03:00.000 --> 00:03:05.000 students and the content itself. I want to give you 00:03:05.000 --> 00:03:11.000 examples of where we're going when we redesign simulations 00:03:11.000 --> 00:03:15.000 from the original legacy version into html5. We have 00:03:15.000 --> 00:03:20.000 been engaged in this process since 2013. And we did a lot 00:03:20.000 --> 00:03:25.000 of learning between 2002 and 2013. So we've been 00:03:25.000 --> 00:03:29.000 implementing that l earning as we redesign and redevelop the 00:03:29.000 --> 00:03:33.000 simulation. So for those not familiar with the 00:03:33.000 --> 00:03:37.000 simulations, PhET simulations have lots of different options 00:03:37.000 --> 00:03:43.000 to do. They're kind of an open exploratory space where 00:03:43.000 --> 00:03:51.000 you can do things that you 00:03:51.000 --> 00:03:53.000 engage next -- in experimentation. In many of 00:03:53.000 --> 00:04:00.000 our new 00:04:00.000 --> 00:04:03.000 simulations we have these additional measure and 00:04:03.000 --> 00:04:05.000 representations. For instance here we have a new measuring 00:04:05.000 --> 00:04:09.000 tool where you can go ahead and measure energy at 00:04:09.000 --> 00:04:13.000 different points along the track. And we have a whole 00:04:13.000 --> 00:04:18.000 new screen that is focused on graphing. And here you can 00:04:18.000 --> 00:04:23.000 look at two different -- sorry . Something is in the way. 00:04:23.000 --> 00:04:27.000 Close it. Two different kinds of graphs, either energy 00:04:27.000 --> 00:04:33.000 versus position or energy versus time. 00:04:33.000 --> 00:04:37.000 The other simulation I wanted to show you was the natural 00:04:37.000 --> 00:04:39.000 selection simulation. So while we d on't -- we haven't 00:04:39.000 --> 00:04:44.000 had 00:04:44.000 --> 00:04:47.000 a big graph -- a big grant in biology, we do have some 00:04:47.000 --> 00:04:49.000 specific simulations for biology. And I just wanted to 00:04:49.000 --> 00:04:54.000 welcome the 00:04:54.000 --> 00:04:59.000 biologists, biology education researchers to explore these 00:04:59.000 --> 00:05:01.000 types of simulations. In this one you can decide what 00:05:01.000 --> 00:05:06.000 mutation -- 00:05:06.000 --> 00:05:10.000 whether you quantity the brown mutation to be dominant or 00:05:10.000 --> 00:05:12.000 recessive and you can see how it changes over the course of 00:05:12.000 --> 00:05:17.000 different generations and you can look at different sorts of 00:05:17.000 --> 00:05:19.000 things, like the pedigree of this particular bunny who had 00:05:19.000 --> 00:05:23.000 a mutation. You can look at the proportion across 00:05:23.000 --> 00:05:27.000 different g enerations or the population graph again taking 00:05:27.000 --> 00:05:30.000 out a data probe now and being able to take data along the 00:05:30.000 --> 00:05:38.000 way. You can add limited food so the bunny population doesn' 00:05:38.000 --> 00:05:44.000 t get out of control. And you can go on to the -- on to the 00:05:44.000 --> 00:05:48.000 next -- on to the next stream where you then have additional 00:05:48.000 --> 00:05:52.000 mutations you can play with. Around ears which basically 00:05:52.000 --> 00:05:54.000 has no impact. And teeth which is a selector for tough 00:05:54.000 --> 00:06:00.000 food. 00:06:00.000 --> 00:06:04.000 So we are continually -- continuing to bring simulation 00:06:04.000 --> 00:06:08.000 to stage 5 as we have resources. The goal is really 00:06:08.000 --> 00:06:11.000 to provide a tool that 00:06:11.000 --> 00:06:16.000 can Simultaneous aaddress multiple 00:06:16.000 --> 00:06:19.000 goal. Goals around experimentation, data 00:06:19.000 --> 00:06:23.000 collection, questioning, reasoning, modeling, those 00:06:23.000 --> 00:06:25.000 sorts of things. Create a tool that students find e 00:06:25.000 --> 00:06:29.000 njoyable, engagible, understandable. Really 00:06:29.000 --> 00:06:33.000 working on that sort 00:06:33.000 --> 00:06:37.000 of affective goal and something that can be 00:06:37.000 --> 00:06:41.000 disseminated freely and easily basically around the world. 00:06:41.000 --> 00:06:46.000 Today we have 158 00:06:46.000 --> 00:06:51.000 simulations. 85 in html5 right 00:06:51.000 --> 00:06:56.000 now. 2,0 00sim-based lessons have been added to the 00:06:56.000 --> 00:06:59.000 database. If you have some, I encourage you to add them to 00:06:59.000 --> 00:07:03.000 the collection. They're all open education resources. 00:07:03.000 --> 00:07:07.000 Translated into 90 languages and run online or off line. 00:07:07.000 --> 00:07:13.000 This is the usage curve. I didn't have a chance to update 00:07:13.000 --> 00:07:20.000 it now. Now we're up to 200 million simulation uses per 00:07:20.000 --> 00:07:24.000 year. That's really about double from last year. 00:07:24.000 --> 00:07:31.000 Primarily due to COVID. We've had expansions over the years 00:07:31.000 --> 00:07:36.000 into different content areas. And we do hope to continue to 00:07:36.000 --> 00:07:40.000 expand our collection. We've also had technology 00:07:40.000 --> 00:07:45.000 advancements. So one of the first was making the sim 00:07:45.000 --> 00:07:49.000 translatable and then h tml5 and then working on creating 00:07:49.000 --> 00:07:54.000 additional accessibility parameters around description 00:07:54.000 --> 00:07:58.000 and I'll show what you we're also doing with sort of PhET 00:07:58.000 --> 00:08:01.000 io project. They do have global impact right now. We' 00:08:01.000 --> 00:08:06.000 re 50% international. And I just wanted to show you sort 00:08:06.000 --> 00:08:08.000 of some of what happened when COVID hit last March. This 00:08:08.000 --> 00:08:14.000 was in France. 00:08:14.000 --> 00:08:17.000 Orange is last year. Blue is this year. And we definitely 00:08:17.000 --> 00:08:20.000 saw a huge uptake of the simulations as the students 00:08:20.000 --> 00:08:28.000 moved to remote 00:08:28.000 --> 00:08:29.000 learning. Which we -- we were pretty excited to see that as 00:08:29.000 --> 00:08:32.000 an indication that the simulations were helping to 00:08:32.000 --> 00:08:34.000 keep students learning science and math during this 00:08:34.000 --> 00:08:41.000 challenging time. 00:08:41.000 --> 00:08:47.000 We've also had a surge in sort of research that uses the PhET 00:08:47.000 --> 00:08:49.000 simulation. So this is a plot from Google scholar of a 00:08:49.000 --> 00:08:52.000 search of PhET simulation education articles. These 00:08:52.000 --> 00:08:57.000 aren't all in research journals. But what we have 00:08:57.000 --> 00:09:01.000 seen is a growth since we moved to html5 and especially 00:09:01.000 --> 00:09:07.000 a growth around -- in international research. And 00:09:07.000 --> 00:09:12.000 we see the research really probing many different things. 00:09:12.000 --> 00:09:16.000 It probes student learning but it also probes 00:09:16.000 --> 00:09:19.000 problem-solving, affective experiences around science and 00:09:19.000 --> 00:09:25.000 math. It really 00:09:25.000 --> 00:09:30.000 ranges. And one of the goals of the new initiative is to 00:09:30.000 --> 00:09:35.000 expand requesters and research that you can engage in 00:09:35.000 --> 00:09:40.000 PhET simulation by providing additional flexibility of the 00:09:40.000 --> 00:09:46.000 s imulations and research data out. So I want to move to 00:09:46.000 --> 00:09:51.000 focusing on introducing you to these next generation PhET 00:09:51.000 --> 00:09:55.000 simulations. You have seep the h tml5 ones on our website 00:09:55.000 --> 00:10:00.000 . But I want to give you a bit of a history. We really 00:10:00.000 --> 00:10:05.000 wanted to envision these next generation PhET simulations as 00:10:05.000 --> 00:10:08.000 being driven by what we already knew about how PhET 00:10:08.000 --> 00:10:12.000 simulations worked, how they were designed, how teachers 00:10:12.000 --> 00:10:15.000 were leveraging them. But we wanted them to go beyond what 00:10:15.000 --> 00:10:19.000 they were able to do back when they were just java and flash. 00:10:19.000 --> 00:10:23.000 And so we also 00:10:23.000 --> 00:10:27.000 engaged and looked at the community needs. So w e -- 00:10:27.000 --> 00:10:33.000 back in 2014, we held a stakeholders meeting that 00:10:33.000 --> 00:10:37.000 included educators, learning, designers, assessment 00:10:37.000 --> 00:10:41.000 professionals, in the learning s ciences. Inclusive design 00:10:41.000 --> 00:10:45.000 and accessibility experts and technology companies. We 00:10:45.000 --> 00:10:51.000 brought them together and really envisioned sort of the 00:10:51.000 --> 00:10:56.000 c apabilities that we wanted to bake into the new 00:10:56.000 --> 00:11:00.000 simulations. So we have -- well, we have two different 00:11:00.000 --> 00:11:07.000 styles of simulations. We have the PhET simulations you 00:11:07.000 --> 00:11:07.000 find on our open accessible website. So these -- 00:11:07.000 --> 00:11:10.000 » Kathy. » KATHY PERKINS, Ph.D.: Yeah. 00:11:10.000 --> 00:11:17.000 » I'm sorry. There is a gray 00:11:17.000 --> 00:11:19.000 bar across your slide at the top right. Where your mouse 00:11:19.000 --> 00:11:20.000 is right now. Now it is gone. Is that. 00:11:20.000 --> 00:11:24.000 » KATHY PERKINS, Ph.D.: Yeah. Maybe somebody -- I don't know 00:11:24.000 --> 00:11:27.000 why it is there. I mean I know sometimes the Zoom -- it 00:11:27.000 --> 00:11:29.000 is a Zoom -- » TOM HELIKAR: Okay. 00:11:29.000 --> 00:11:35.000 » KATHY PERKINS, Ph.D.: It is a Zoom thing. 00:11:35.000 --> 00:11:41.000 I don't know how to turn it off. It is the one that says 00:11:41.000 --> 00:11:43.000 admit. When people are in the waiting room. 00:11:43.000 --> 00:11:44.000 » TOM HELIKAR: Oh, okay. Stephanie. 00:11:44.000 --> 00:11:46.000 » KATHY PERKINS, Ph.D.: Do you know how to not have that pop 00:11:46.000 --> 00:11:48.000 up for me? 00:11:48.000 --> 00:11:50.000 » STEPHANIE VENDETTI: You 00:11:50.000 --> 00:11:55.000 should be able 00:11:55.000 --> 00:11:58.000 -- there's a an arrow to close that notification box. It 00:11:58.000 --> 00:11:59.000 should be up at the top. 00:11:59.000 --> 00:12:04.000 » KATHY PERKINS, Ph.D.: Well, 00:12:04.000 --> 00:12:10.000 I -- I closed 00:12:10.000 --> 00:12:11.000 it. » TOM HELIKAR: Is that a full 00:12:11.000 --> 00:12:15.000 screen option. » KATHY PERKINS, Ph.D.: Well, 00:12:15.000 --> 00:12:18.000 I'll keep closing it when it pops up there. 00:12:18.000 --> 00:12:19.000 » STEPHANIE VENDETTI: Can you X out of it. There we go. 00:12:19.000 --> 00:12:23.000 That's better. » KATHY PERKINS, Ph.D.: Yes. 00:12:23.000 --> 00:12:26.000 So please tell me if it reappears. Last night I was 00:12:26.000 --> 00:12:27.000 on Zoom and that grid box wasn 't appearing. 00:12:27.000 --> 00:12:31.000 » TOM HELIKAR: Thank you. » KATHY PERKINS, Ph.D.: I 00:12:31.000 --> 00:12:42.000 thought it was gone. Thanks for letting me know. 00:12:42.000 --> 00:12:46.000 So -- with these next generation PhET simulations we 00:12:46.000 --> 00:12:51.000 have the html5 goal of bringing these simulations to 00:12:51.000 --> 00:12:53.000 be usable on all devices including phones and tablets, 00:12:53.000 --> 00:12:58.000 translatable and highly embeddable which is a project 00:12:58.000 --> 00:13:00.000 in progress. And those are the 85 -- 87sims that you see 00:13:00.000 --> 00:13:03.000 on our 00:13:03.000 --> 00:13:09.000 website. 00:13:09.000 --> 00:13:12.000 We want to customize the 00:13:12.000 --> 00:13:18.000 simulations. Set the starting stage so you can match it to 00:13:18.000 --> 00:13:27.000 your learning or instructional moment. Design a test feature 00:13:27.000 --> 00:13:57.000 interoperable. So it can talk with any education technology 00:14:00.000 --> 00:14:00.000 . You know, being able to use it in assessments. To make itthat you have. You can set the state from the technology.You can pull the back-end data out of the simulation. Into 00:14:00.000 --> 00:14:00.000 that technology. So you can really start to think about how to create a full learning environment and sort of watch what students are doing in that learning environment. So 00:14:00.000 --> 00:14:03.000 you don't just know how they answered your questions, but you actually can see inside what they're doing with the simulations 00:14:03.000 --> 00:14:08.000 themselves. So I want to give you a little bit of a tour of 00:14:08.000 --> 00:14:10.000 what this new technology looks like. Looks 00:14:10.000 --> 00:14:15.000 like. It 00:14:15.000 --> 00:14:19.000 is called PhET io. And you can customize the simulations. 00:14:19.000 --> 00:14:22.000 So I'm going to start with a basic demo. You can make it 00:14:22.000 --> 00:14:25.000 just one screen like other simulations. You can hide or 00:14:25.000 --> 00:14:28.000 show controls. Let's say you don't want students to see 00:14:28.000 --> 00:14:31.000 this full-time view. Just on the beam view. You can 00:14:31.000 --> 00:14:36.000 completely hide those controls . You can set the starting 00:14:36.000 --> 00:14:39.000 state if want them to start with the filter on. And you 00:14:39.000 --> 00:14:42.000 can modify labels. We have gone beyond 00:14:42.000 --> 00:14:47.000 -- this is just a demo. I want to give you a look at 00:14:47.000 --> 00:14:54.000 what it actually looks like. This is natural selection in 00:14:54.000 --> 00:14:58.000 our new PhET io studio. So this is -- exposes the inner 00:14:58.000 --> 00:15:01.000 workings of the instructal designer at a level that they 00:15:01.000 --> 00:15:05.000 can actually -- I know it looks like a lot. But you can 00:15:05.000 --> 00:15:08.000 actually pretty easily come to be acquainted with this. So 00:15:08.000 --> 00:15:13.000 each screen has the model and the view. The view is 00:15:13.000 --> 00:15:19.000 anything that is visible. So let's say you want to create a 00:15:19.000 --> 00:15:24.000 situation in this simulation where you want 00:15:24.000 --> 00:15:28.000 to start with no wolves. So you would go to the 00:15:28.000 --> 00:15:31.000 environmental factors panel. Go to the wolves check box and 00:15:31.000 --> 00:15:34.000 just -- you can just make the wolves check box invisible. 00:15:34.000 --> 00:15:37.000 So now that is your setting. Let's say, you know, you want 00:15:37.000 --> 00:15:45.000 to have this population graph but you don't want students to 00:15:45.000 --> 00:15:51.000 get distracted by those other graphs. You can just then go 00:15:51.000 --> 00:15:56.000 to the graphs, the choice radio button and make that 00:15:56.000 --> 00:16:02.000 entire thing invisible. You can preset the mutation to 00:16:02.000 --> 00:16:06.000 have a brown dominant mutation . And then you can launch the 00:16:06.000 --> 00:16:10.000 simulation in that specific mode. As you embed in your 00:16:10.000 --> 00:16:12.000 activity, it would come up in this setting that you had 00:16:12.000 --> 00:16:17.000 configured. And then the students would be able to look 00:16:17.000 --> 00:16:20.000 to use it in that. And this could be embedded as an eye 00:16:20.000 --> 00:16:33.000 frame in a full activity. 00:16:33.000 --> 00:16:37.000 So that is sort of the full customization integration. We 00:16:37.000 --> 00:16:43.000 allow data to be pulled in and out. And I'm just going to 00:16:43.000 --> 00:16:48.000 kind of show you an example of this virtual lab. So you can 00:16:48.000 --> 00:16:51.000 customize again just on the one. Here you can shake in. 00:16:51.000 --> 00:16:55.000 And this is monitoring the student to see if they can 00:16:55.000 --> 00:17:00.000 make a saturated solution. And when they do make a 00:17:00.000 --> 00:17:03.000 saturated solution, the external eye frame can detect 00:17:03.000 --> 00:17:08.000 that and give them feedback. So these new features allow 00:17:08.000 --> 00:17:13.000 you to create -- to monitor what students are doing and 00:17:13.000 --> 00:17:19.000 explore the role of feedback in that learning process. 00:17:19.000 --> 00:17:22.000 And then you can record data out. So you can build -- this 00:17:22.000 --> 00:17:27.000 is an external data. Record button. Every time that I 00:17:27.000 --> 00:17:31.000 press it, it goes back to the sim and says give me the data 00:17:31.000 --> 00:17:34.000 and then the external website just puts it into a table. So 00:17:34.000 --> 00:17:38.000 you can record out data. You can record out actually the 00:17:38.000 --> 00:17:42.000 entire state. So I can revisit any one of these 00:17:42.000 --> 00:17:44.000 previous data points I got and push it back into the -- into 00:17:44.000 --> 00:17:52.000 the simulation. 00:17:52.000 --> 00:18:03.000 And you can also, if you want to get into the 00:18:03.000 --> 00:18:07.000 details, log back into it. So -- sorry. I keep closing that 00:18:07.000 --> 00:18:12.000 . So every event or interaction the students do is 00:18:12.000 --> 00:18:15.000 logged. Through a -- it is called a json log file. But 00:18:15.000 --> 00:18:20.000 you can imagine it generates a whole lot of data as you 00:18:20.000 --> 00:18:23.000 detect what students do. You can also capture the states 00:18:23.000 --> 00:18:26.000 which is -- tells you what the state of the simulation is but 00:18:26.000 --> 00:18:29.000 it doesn't tell you how the students got there. And you 00:18:29.000 --> 00:18:34.000 can capture input events. So where the mouse is, what the 00:18:34.000 --> 00:18:39.000 mouse is doing. And right now these mouse events are being 00:18:39.000 --> 00:18:40.000 played back into that lower copy of the simulation on the 00:18:40.000 --> 00:18:47.000 lower right. 00:18:47.000 --> 00:18:51.000 So that's the suite of capabilities that were 00:18:51.000 --> 00:18:58.000 envisioned by this group of cross 00:18:58.000 --> 00:19:00.000 disciplinary sector stakeholders. And we are now 00:19:00.000 --> 00:19:02.000 essentially on the verge of realizing many -- many of 00:19:02.000 --> 00:19:06.000 those 00:19:06.000 --> 00:19:10.000 features accessibility is in the work. It is a challenging 00:19:10.000 --> 00:19:14.000 -- very challenging. And then we've also realized a lot of 00:19:14.000 --> 00:19:21.000 these more advanced f eatures. And we're working on growing 00:19:21.000 --> 00:19:23.000 the sim collection we have in that. So by -- when 00:19:23.000 --> 00:19:28.000 you embed 00:19:28.000 --> 00:19:31.000 simulation now into a learning environment, you can think of 00:19:31.000 --> 00:19:35.000 the new opportunities for l earning and research. And so 00:19:35.000 --> 00:19:41.000 I mentioned sort of you can monitor what students are 00:19:41.000 --> 00:19:43.000 doing, think about the role of feedback in activities or 00:19:43.000 --> 00:19:46.000 assessments. You can probe -- you can think about what can 00:19:46.000 --> 00:19:48.000 you measure now that you couldn't measure before. 00:19:48.000 --> 00:19:52.000 Because you can 00:19:52.000 --> 00:19:55.000 actually see students action and you can explore that back 00:19:55.000 --> 00:19:59.000 end data and probe what -- you know, what patterns of 00:19:59.000 --> 00:20:03.000 interaction lead to productive learning or what patterns of 00:20:03.000 --> 00:20:06.000 interaction indicate that students, you know, are 00:20:06.000 --> 00:20:09.000 proficient at this sort of science practice or that sort 00:20:09.000 --> 00:20:12.000 of science practice. And it provides a lot of 00:20:12.000 --> 00:20:17.000 instructional opportunity for 00:20:17.000 --> 00:20:21.000 designing, you know, new challenges for students by 00:20:21.000 --> 00:20:24.000 setting the simulation up in a particular state. So this is 00:20:24.000 --> 00:20:32.000 just an 00:20:32.000 --> 00:20:38.000 example. Andy garvin had built out this nice sim-based 00:20:38.000 --> 00:20:41.000 lab that included content and process goals where he had the 00:20:41.000 --> 00:20:46.000 hooke's law simulation and the students had it in a 00:20:46.000 --> 00:20:48.000 spreadsheet. You can customize the sim, design a 00:20:48.000 --> 00:20:53.000 instructional wrapper. You could follow students' actions 00:20:53.000 --> 00:20:56.000 . You canned provide adaptive feedback and assess on both 00:20:56.000 --> 00:21:00.000 process and content by embedding this in some sort of 00:21:00.000 --> 00:21:06.000 learning platform. So I want to give you some research 00:21:06.000 --> 00:21:10.000 examples now of collaborators that have been 00:21:10.000 --> 00:21:15.000 using these new 00:21:15.000 --> 00:21:19.000 capabilities of PhET io simulations. This is sort of 00:21:19.000 --> 00:21:26.000 -- was inspired before we fully got PhET io. But we 00:21:26.000 --> 00:21:31.000 were logging back in event stream. So Julia Chamberlain 00:21:31.000 --> 00:21:37.000 was a postdoc with PhET. And with this study wily 00:21:37.000 --> 00:21:39.000 we looked at the effective guidance, different levels of 00:21:39.000 --> 00:21:44.000 guidance on the way that college students interacted 00:21:44.000 --> 00:21:50.000 with the simulations. So they were engaging the s imulations 00:21:50.000 --> 00:21:54.000 in their lab sections. And we recorded out a log of event 00:21:54.000 --> 00:21:58.000 streams of user interactions and then analyzed that log. 00:21:58.000 --> 00:22:02.000 We went from sort of light guidance where we asked 00:22:02.000 --> 00:22:06.000 students to explore the sim, make sure that we investigate 00:22:06.000 --> 00:22:10.000 all of the factors, to moderate guidance where we had 00:22:10.000 --> 00:22:13.000 a little bit more direction, compare strong and weak, 00:22:13.000 --> 00:22:17.000 describe all of the ways that they're similar and different. 00:22:17.000 --> 00:22:22.000 But still pretty open. Ask then have a guidance where 00:22:22.000 --> 00:22:28.000 students were more told what to interact with and what to 00:22:28.000 --> 00:22:31.000 do within the simulation. And we -- we definitely saw an 00:22:31.000 --> 00:22:35.000 impact on students -- the level of student interaction 00:22:35.000 --> 00:22:39.000 with the simulation. So the students that had the light 00:22:39.000 --> 00:22:42.000 guidance had the most interaction with the 00:22:42.000 --> 00:22:46.000 simulation. And students with heavy guidance had the least. 00:22:46.000 --> 00:22:50.000 The students with the light guidance also explored almost 00:22:50.000 --> 00:22:55.000 100% of the features within the simulation on their own. 00:22:55.000 --> 00:22:58.000 The moderate guidance as well. And the heavy guidance was 00:22:58.000 --> 00:23:03.000 impacted. If you didn't tell them to explore it, they did 00:23:03.000 --> 00:23:10.000 not explore it. That told us a lot about the influence of 00:23:10.000 --> 00:23:16.000 heavy guidance on how students interact with simulations. 00:23:16.000 --> 00:23:20.000 Ido roll is a scientist and has gone deeper into this 00:23:20.000 --> 00:23:24.000 direction and used learning analytics in his research. So 00:23:24.000 --> 00:23:29.000 this is a study that he did with 100 first-year college 00:23:29.000 --> 00:23:34.000 students in the introductory physics courses. Again, with 00:23:34.000 --> 00:23:38.000 two different levels of scaffolding. We call it 00:23:38.000 --> 00:23:42.000 guidance. He calls it scaffolding. Basically the 00:23:42.000 --> 00:23:50.000 same type of -- it is very similar. 00:23:50.000 --> 00:23:54.000 He logged 100 hours of data from these student users. And 00:23:54.000 --> 00:23:58.000 that was 130,000 a ctions. Just to give you a sense of 00:23:58.000 --> 00:24:04.000 how much data is generated by a simulation. 00:24:04.000 --> 00:24:09.000 The study was structured as a pretest to understand their 00:24:09.000 --> 00:24:12.000 knowledge about circuits. And then two c onditions. 00:24:12.000 --> 00:24:18.000 Students first -- in this activity. One, they either 00:24:18.000 --> 00:24:22.000 did an unstructured activity or a directive activity. 00:24:22.000 --> 00:24:26.000 And then in activity 2, all of the students did an 00:24:26.000 --> 00:24:29.000 unstructured activity. And then the post test measured 00:24:29.000 --> 00:24:33.000 both their understanding of activity one and their 00:24:33.000 --> 00:24:37.000 understanding of activity two. Unstructured activity was 00:24:37.000 --> 00:24:43.000 similar to what I showed you about Julia's. You know, 00:24:43.000 --> 00:24:47.000 pretty open. Used the simulation to explore how 00:24:47.000 --> 00:24:50.000 voltage current and 00:24:50.000 --> 00:24:52.000 brightness depend on the lightbulbs. Structured 00:24:52.000 --> 00:24:57.000 similar learning goals but as you can see from the look of 00:24:57.000 --> 00:24:59.000 this, it was much more structured. Had explicit data 00:24:59.000 --> 00:25:06.000 tables. They were asked to build specific configurations 00:25:06.000 --> 00:25:11.000 of lightbulb s. He looks at a variety of things. I'm going 00:25:11.000 --> 00:25:16.000 to give you a sense. It is a very deep study. I encourage 00:25:16.000 --> 00:25:18.000 you to go to look at this paper. It was in 00:25:18.000 --> 00:25:25.000 instructional science if you want more i nformation. First 00:25:25.000 --> 00:25:31.000 on the impact of learning, he did see sort of the impact of 00:25:31.000 --> 00:25:38.000 learning on low -- this was selected for the students that 00:25:38.000 --> 00:25:40.000 had low preknowledge. So under the directive case, 00:25:40.000 --> 00:25:47.000 those students did learn but they actually learned a little 00:25:47.000 --> 00:25:50.000 bit less than the students that were -- came in 00:25:50.000 --> 00:25:55.000 as preknowledge but had structured activity. And they 00:25:55.000 --> 00:25:58.000 all ended up the same. But essentially the bottom line is 00:25:58.000 --> 00:26:02.000 that -- you know, the unstructured activity was 00:26:02.000 --> 00:26:07.000 probably addressing many more of the kind of process goals 00:26:07.000 --> 00:26:12.000 that you would have in -- in science learning and they 00:26:12.000 --> 00:26:16.000 learned just as much. He explored whether the back-end 00:26:16.000 --> 00:26:20.000 data could indeed predict learning from the simulations. 00:26:20.000 --> 00:26:27.000 And he found that after about five minutes, the log -- you 00:26:27.000 --> 00:26:31.000 know, if you did files f ile -- you could predict learning. 00:26:31.000 --> 00:26:34.000 This was the number of interactions observed. The 00:26:34.000 --> 00:26:38.000 examples of three sorts of models. He studied many 00:26:38.000 --> 00:26:44.000 different model combinations of the back end data. But you 00:26:44.000 --> 00:26:49.000 can see at about 10 minutes, he was, you know, 65 00:26:49.000 --> 00:26:52.000 to 70% accurate. Not 10 minutes. Sorry. 10% of the 00:26:52.000 --> 00:26:55.000 interaction. He could have a pretty good accuracy of 00:26:55.000 --> 00:26:58.000 whether they were going to be a high learner or a low 00:26:58.000 --> 00:27:03.000 learner. And by the end of that interact -- those 00:27:03.000 --> 00:27:06.000 interaction events, it was an even higher accuracy whether - 00:27:06.000 --> 00:27:11.000 - he could predict if they were low learner or high 00:27:11.000 --> 00:27:18.000 learner. He also investigated what productive students do. 00:27:18.000 --> 00:27:23.000 So he kind of looked at three different types of things. 00:27:23.000 --> 00:27:26.000 Constructing a circuit. Pausing. Not doing anything. 00:27:26.000 --> 00:27:33.000 Just pausing 00:27:33.000 --> 00:27:38.000 and thinking. Sort of implied in the pause. Testing a 00:27:38.000 --> 00:27:42.000 circuit with one resistor, testing a circuit with two 00:27:42.000 --> 00:27:48.000 resistors or multiple resistors. So the beginning 00:27:48.000 --> 00:27:52.000 of the interaction, so this is the -- you know, the first 240 00:27:52.000 --> 00:27:55.000 seconds. So students are constructing. Then they go 00:27:55.000 --> 00:28:00.000 test. Then they pause. Then they test. Then they 00:28:00.000 --> 00:28:05.000 construct some more. Then they test something more 00:28:05.000 --> 00:28:09.000 complicated. A resistor with two circuits -- a circuit with 00:28:09.000 --> 00:28:12.000 two resistors. Then they pause and build more complex 00:28:12.000 --> 00:28:16.000 tests. Pause. Construct. They're going back and forth 00:28:16.000 --> 00:28:22.000 through these different sort of actions. 00:28:22.000 --> 00:28:27.000 But then the pattern sort of changes. They start actually 00:28:27.000 --> 00:28:31.000 pausing more. So they construct. They do another 00:28:31.000 --> 00:28:34.000 test on this two resistor circuit. They pause, test 00:28:34.000 --> 00:28:38.000 again. They pause. They think. And they keep kind of 00:28:38.000 --> 00:28:45.000 going through this pattern. But with sort of longer pauses 00:28:45.000 --> 00:28:49.000 and thinking. And you can see here that they really focus on 00:28:49.000 --> 00:28:54.000 this -- they test a lot with this two -- two resistor 00:28:54.000 --> 00:28:58.000 circuit. So when he looked at patterns in the data of 00:28:58.000 --> 00:29:03.000 unproductive learners, he found that these unproductive 00:29:03.000 --> 00:29:09.000 learners often were testing with circuits that had lots of 00:29:09.000 --> 00:29:14.000 resistors. More than two resistors. Ask that 00:29:14.000 --> 00:29:19.000 . And the pauses came before the construct. Either before 00:29:19.000 --> 00:29:21.000 or after construct. They didn 't pause after test. Pause 00:29:21.000 --> 00:29:24.000 before after construct. When he looked at productive 00:29:24.000 --> 00:29:28.000 learners, he saw a different pattern. So where they did a 00:29:28.000 --> 00:29:33.000 lot of testing of two resistor circuits and pausing after 00:29:33.000 --> 00:29:37.000 that. Those are the kind of p atterns that he saw when he 00:29:37.000 --> 00:29:41.000 looked at the back end data. And he was able to pull that 00:29:41.000 --> 00:29:45.000 out and that was how, you 00:29:45.000 --> 00:29:47.000 know, the way -- the signatures that allowed him to 00:29:47.000 --> 00:29:57.000 understand learning. 00:29:57.000 --> 00:29:59.000 So -- so we have kind of looked at ido's example shows 00:29:59.000 --> 00:30:01.000 us how we can look at the impact of learning. We can 00:30:01.000 --> 00:30:04.000 look at the impact on prediction of learning. And 00:30:04.000 --> 00:30:10.000 we can dive into sort of what the back end data is doing. 00:30:10.000 --> 00:30:14.000 And now I want to move on to sort of what these new tools 00:30:14.000 --> 00:30:16.000 might be able to do in terms of advancing assessment and 00:30:16.000 --> 00:30:25.000 things that we can a ssess. 00:30:25.000 --> 00:30:33.000 So shima was a graduate 00:30:33.000 --> 00:30:37.000 student in Carl 00:30:37.000 --> 00:30:41.000 wieman's group. Now she is at Stanford. This is a research 00:30:41.000 --> 00:30:44.000 tool actually right now. We want to build it out for full 00:30:44.000 --> 00:30:48.000 use but it is a research tool. How we can use the 00:30:48.000 --> 00:30:54.000 construction kit and specifically this black box 00:30:54.000 --> 00:30:57.000 version to assess knowledge and practices give us insight 00:30:57.000 --> 00:31:03.000 into students problem solving practices. So I want to give 00:31:03.000 --> 00:31:07.000 you a sense of how different students can be when they're 00:31:07.000 --> 00:31:13.000 interacting with this sort of tool. So we're going to take 00:31:13.000 --> 00:31:19.000 a look at some videos. Oh. I think I need to unshare. 00:31:19.000 --> 00:31:21.000 Sorry. I think I did not share the video. Oh, I did. 00:31:21.000 --> 00:31:28.000 Okay. Never mind. 00:31:28.000 --> 00:31:38.000 So this is 00:31:38.000 --> 00:31:54.000 Ellen. I can't find my -- s orry. 00:31:54.000 --> 00:32:04.000 » All right. 00:32:04.000 --> 00:32:15.000 So let's see if the top and the 00:32:15.000 --> 00:32:17.000 right -- the top and the right -- terminals are connected 00:32:17.000 --> 00:32:32.000 electrically. 00:32:32.000 --> 00:32:36.000 Let's see. It looks l ike -- oh, it is -- is that -- so 00:32:36.000 --> 00:32:40.000 there is current flowing in there. 00:32:40.000 --> 00:32:45.000 Okay. 00:32:45.000 --> 00:32:50.000 Does it not show the electrons moving anymore? Okay. That's 00:32:50.000 --> 00:32:56.000 fine. Let's 00:32:56.000 --> 00:33:01.000 see if these are connecting. 00:33:01.000 --> 00:33:12.000 0 amps. Okay. 00:33:12.000 --> 00:33:20.000 What about -- the bottom and the right. 00:33:20.000 --> 00:33:25.000 Don't seem to be electrically connected. So these seem to 00:33:25.000 --> 00:33:31.000 be though. Yeah. 00:33:31.000 --> 00:33:34.000 And voltameter. So the battery -- 00:33:34.000 --> 00:33:39.000 » KATHY PERKINS, Ph.D.: So as you can see Ellen is being 00:33:39.000 --> 00:33:44.000 super systematic about how she is going through this. This 00:33:44.000 --> 00:33:48.000 process of exploration. So now let's do a 00:33:48.000 --> 00:33:53.000 comparison with Gail and Josh. 00:33:53.000 --> 00:33:57.000 » Nothing is connected, I 00:33:57.000 --> 00:33:59.000 guess. So I think -- okay. Let's start with a wire. 00:33:59.000 --> 00:34:04.000 » Should we connect this. » Yeah. Connect a wire from 00:34:04.000 --> 00:34:07.000 here to here. And see if there is a current. There's 00:34:07.000 --> 00:34:09.000 no current. » No. Do that for each of 00:34:09.000 --> 00:34:13.000 them. » Oh, how about -- I don't 00:34:13.000 --> 00:34:16.000 know if this is real, but try to connect it to all -- like 00:34:16.000 --> 00:34:21.000 just make it go all the way around. 00:34:21.000 --> 00:34:26.000 Like -- » Like one? 00:34:26.000 --> 00:34:30.000 » So get a wire. Another wire . And go from there to there. 00:34:30.000 --> 00:34:32.000 And another one from there to there. See what I'm saying. 00:34:32.000 --> 00:34:34.000 » Got it. 00:34:34.000 --> 00:34:37.000 » Okay. And I'm guessing 00:34:37.000 --> 00:34:40.000 there is still not going to be any current. Okay. Great. 00:34:40.000 --> 00:34:42.000 So there is no current. That means there is no battery in 00:34:42.000 --> 00:34:44.000 there. » No battery. 00:34:44.000 --> 00:34:47.000 » So okay. 00:34:47.000 --> 00:34:50.000 » Some kind of resistor. 00:34:50.000 --> 00:34:53.000 » Yeah. How about you get a battery and you attach it 00:34:53.000 --> 00:34:57.000 anywhere really. It could be anywhere. I think it means 00:34:57.000 --> 00:35:00.000 there is no battery. I could be wrong. 00:35:00.000 --> 00:35:07.000 » Yeah. 00:35:07.000 --> 00:35:13.000 » Oh, okay. That means that there's no resistor in there 00:35:13.000 --> 00:35:17.000 either. » Yeah. Well, no because -- 00:35:17.000 --> 00:35:20.000 because -- yeah. 00:35:20.000 --> 00:35:27.000 » KATHY PERKINS, Ph.D.: Okay. 00:35:27.000 --> 00:35:31.000 So hopefully you saw this sort of task can really provide 00:35:31.000 --> 00:35:35.000 insight into how students are thinking about problem solving 00:35:35.000 --> 00:35:39.000 and the way that they go about it. If we were to keep 00:35:39.000 --> 00:35:46.000 watching this, we would see actually the problem solving 00:35:46.000 --> 00:35:53.000 approaches of this team evolve to be more systematic as time 00:35:53.000 --> 00:35:56.000 went along. So the other kind of interesting fact about 00:35:56.000 --> 00:35:59.000 simulations or these sorts of tools is that there can be l 00:35:59.000 --> 00:36:07.000 earning and assessment all at the same time. 00:36:07.000 --> 00:36:13.000 So 00:36:13.000 --> 00:36:17.000 shima coded these interactions from these 00:36:17.000 --> 00:36:20.000 interviews and looked at different experimentation 00:36:20.000 --> 00:36:26.000 practices and reflective practices and looked how they 00:36:26.000 --> 00:36:28.000 were displayed within the interview. This -- this 00:36:28.000 --> 00:36:35.000 research is being 00:36:35.000 --> 00:36:41.000 extended to see what the back-end data can tell us. 00:36:41.000 --> 00:36:48.000 So that's ongoing work in Carl 's lab to now look at the -- 00:36:48.000 --> 00:36:54.000 at the l earning analytics and which of these sort of expert 00:36:54.000 --> 00:36:57.000 and novice practices would -- are able to be kind of done at 00:36:57.000 --> 00:37:03.000 scale by looking at the back end data. 00:37:03.000 --> 00:37:08.000 I'm going to leave time for questions. So I want to be 00:37:08.000 --> 00:37:12.000 sure though to thank the funders of the project, in 00:37:12.000 --> 00:37:19.000 particular the Moore foundation has really been a 00:37:19.000 --> 00:37:22.000 great supporter of this latest work on PhET io. And the new 00:37:22.000 --> 00:37:27.000 capabilities, new kinds of research that we're engaged in 00:37:27.000 --> 00:37:30.000 . And we have some new funding from the 00:37:30.000 --> 00:37:36.000 yidan prize that will help us continue in this direction. I 00:37:36.000 --> 00:37:40.000 want to invite you to find PhET. Use the 00:37:40.000 --> 00:37:44.000 simulations. Contribute your lessons. You can support PhET 00:37:44.000 --> 00:37:50.000 by downloading the PhET app on your phone. You can also e- 00:37:50.000 --> 00:37:54.000 mail us. We like to get input from everybody in the 00:37:54.000 --> 00:37:59.000 community if you have a tweak to a simulation that you think 00:37:59.000 --> 00:38:03.000 would be good 00:38:03.000 --> 00:38:05.000 p edogically. If you want to engage in collaborations. 00:38:05.000 --> 00:38:07.000 Specifically we will be -- I think we will have an 00:38:07.000 --> 00:38:14.000 opportunity 00:38:14.000 --> 00:38:16.000 to sort of expand our research collaborations around PhET io. 00:38:16.000 --> 00:38:22.000 If 00:38:22.000 --> 00:38:25.000 that inspires you to look more at around simulation based 00:38:25.000 --> 00:38:36.000 learning or learning 00:38:36.000 --> 00:38:40.000 analytics back in -- anything, please do reach out to me. 00:38:40.000 --> 00:38:46.000 And let's talk about that. With that, I'm going to end. 00:38:46.000 --> 00:38:50.000 » TOM HELIKAR: Thank you very 00:38:50.000 --> 00:38:52.000 much for a great presentation and sharing your work with us. 00:38:52.000 --> 00:38:57.000 So we have 7 minutes at least for some questions. And there 00:38:57.000 --> 00:39:03.000 are a few in the chat box. So I'll start translating them. 00:39:03.000 --> 00:39:07.000 So first question , are you planning to hold workshops for 00:39:07.000 --> 00:39:12.000 faculty interested in developing or modifying 00:39:12.000 --> 00:39:17.000 simulations? » KATHY PERKINS, Ph.D.: I -- 00:39:17.000 --> 00:39:24.000 we don't usually hold workshops for faculty to -- to 00:39:24.000 --> 00:39:28.000 develop their own simulations. But we are -- we definitely 00:39:28.000 --> 00:39:32.000 are interested in engaging in faculty that want to build out 00:39:32.000 --> 00:39:37.000 new kinds of simulations and happy to write research 00:39:37.000 --> 00:39:41.000 proposals around that. Usually that requires funding. 00:39:41.000 --> 00:39:47.000 So I would love to build out a new suite of biology 00:39:47.000 --> 00:39:52.000 simulations and engage with -- you know, a biology education 00:39:52.000 --> 00:39:59.000 researcher to -- in a proposal if they're interested in that 00:39:59.000 --> 00:40:04.000 sort of work. You know, same with data sciences or 00:40:04.000 --> 00:40:08.000 university based mathematics. Happy to do that sort of work. 00:40:08.000 --> 00:40:13.000 Building a new simulation for us is a big endeavor. And it 00:40:13.000 --> 00:40:13.000 costs about $85,000 at cost. So it is not cheap. 00:40:13.000 --> 00:40:19.000 » TOM HELIKAR: Okay. Thank 00:40:19.000 --> 00:40:23.000 you. Another question from 00:40:23.000 --> 00:40:25.000 Nicole. Is PhET io available in different languages? 00:40:25.000 --> 00:40:28.000 » KATHY PERKINS, Ph.D.: PhET io will be available in 00:40:28.000 --> 00:40:31.000 different languages, 00:40:31.000 --> 00:40:38.000 yeah. So PhET io right now we 're still -- it is still 00:40:38.000 --> 00:40:42.000 emerging. And it is -- it's a basis of a sustainable 00:40:42.000 --> 00:40:47.000 business model that we have. To leverage PhET io you have 00:40:47.000 --> 00:40:52.000 to have back end data collection, environment to 00:40:52.000 --> 00:40:56.000 hook up to PhET io. So we are working with some partners to 00:40:56.000 --> 00:41:00.000 leverage PhET io. I think recently 00:41:00.000 --> 00:41:04.000 p ivot interactive afoundationed they're a 00:41:04.000 --> 00:41:07.000 partner with us. So you will see some within pivot 00:41:07.000 --> 00:41:11.000 interactive. We want to have a program for researchers 00:41:11.000 --> 00:41:19.000 where researchers can engage with PhET io to do research, 00:41:19.000 --> 00:41:21.000 you know, without -- without any cost. But -- but it still 00:41:21.000 --> 00:41:24.000 -- it does require a license or just a simple arrangement 00:41:24.000 --> 00:41:30.000 with a researcher. » TOM HELIKAR: Okay. Thank 00:41:30.000 --> 00:41:34.000 you. A question from Keith. He says, math guy question. 00:41:34.000 --> 00:41:37.000 Mathematically a lot of these tests can be represented by 00:41:37.000 --> 00:41:41.000 ordinary differential equations. Do any of these 00:41:41.000 --> 00:41:45.000 simulations have an option to generate an ODE or a system of 00:41:45.000 --> 00:41:47.000 ODEs to represent the situation? It would be really 00:41:47.000 --> 00:41:50.000 neat for students to be able to make changes to the system 00:41:50.000 --> 00:41:52.000 and see how those changes influence the equations or 00:41:52.000 --> 00:41:53.000 vice versa. 00:41:53.000 --> 00:41:58.000 » KATHY PERKINS, Ph.D.: So I 00:41:58.000 --> 00:42:03.000 think that would be possible with PhET io. Because PhET io 00:42:03.000 --> 00:42:09.000 hooks into the model. So you could give students -- you 00:42:09.000 --> 00:42:15.000 could represent the ODEs that are representing the model. 00:42:15.000 --> 00:42:19.000 And give students hooks into it. And basically use the 00:42:19.000 --> 00:42:22.000 PhET io hook into the model t o -- you would build sort of 00:42:22.000 --> 00:42:25.000 the external representation of what the ODE looked like and 00:42:25.000 --> 00:42:28.000 create a hook into the model, push that into the simulation 00:42:28.000 --> 00:42:31.000 and it would respond to the changes that the student did 00:42:31.000 --> 00:42:32.000 in the ODE. 00:42:32.000 --> 00:42:39.000 » TOM HELIKAR: Great. Thank 00:42:39.000 --> 00:42:42.000 you. A question from Rebecca. If guidance leads to less 00:42:42.000 --> 00:42:46.000 exploration or learning but certain exploration steps are 00:42:46.000 --> 00:42:49.000 associated with learning success, which does an 00:42:49.000 --> 00:42:52.000 instructor go 00:42:52.000 --> 00:42:57.000 with? » KATHY PERKINS, Ph.D.: I 00:42:57.000 --> 00:43:00.000 think you should go with what -- you should 00:43:00.000 --> 00:43:06.000 -- instruction is the center on the learning goals that you 00:43:06.000 --> 00:43:10.000 have. So I think if processes and practices are important in 00:43:10.000 --> 00:43:16.000 your learning goals, you should say 00:43:16.000 --> 00:43:19.000 pretty open and how -- how to instruct students to learn w 00:43:19.000 --> 00:43:26.000 ith -- to -- like what kind of instructional tasks you give 00:43:26.000 --> 00:43:28.000 students. But I think there's opportunities there to sort of 00:43:28.000 --> 00:43:32.000 go meta- 00:43:32.000 --> 00:43:44.000 with your students and reflect on different ways to interact 00:43:44.000 --> 00:44:14.000 with the simulation or a tool and kind of help advance their 00:44:18.000 --> 00:44:18.000 overall exploratory practices and show some of those relationships. Like this, you know, is it easier to understand this or this. So you can go more explicit on 00:44:18.000 --> 00:44:18.000 sort of, you know, get feedback from your students, what did you learn and then go explicit on the learning. Butwe a re -- that is sort of the goal of one of our current 00:44:18.000 --> 00:44:20.000 research projects with Carl is to understand how to help teach expert problem solving strategies. So maybe check 00:44:20.000 --> 00:44:29.000 back in a couple years and we will have some more 00:44:29.000 --> 00:44:34.000 insight. I think -- so far his group and working with 00:44:34.000 --> 00:44:38.000 Natasha Holmes they have discovered that asking the 00:44:38.000 --> 00:44:41.000 students the decisions is a powerful learning moment. 00:44:41.000 --> 00:44:43.000 Decisions about what to measure, things like that. 00:44:43.000 --> 00:44:48.000 » TOM HELIKAR: Thank you. One 00:44:48.000 --> 00:44:52.000 more -- I think we have time for one more question. Have 00:44:52.000 --> 00:44:56.000 you looked at how well these reflective and experimental 00:44:56.000 --> 00:44:59.000 practices that students have been using in the simulation 00:44:59.000 --> 00:45:03.000 translate into an actual laboratory 00:45:03.000 --> 00:45:09.000 setting? » KATHY PERKINS, Ph.D.: We 00:45:09.000 --> 00:45:14.000 have -- a long time ago early in PhET we did a research 00:45:14.000 --> 00:45:18.000 study that provided insight into that. I wouldn't say we 00:45:18.000 --> 00:45:23.000 systematically studied that question. But we did early on 00:45:23.000 --> 00:45:27.000 it was like 2004, 2005, we did a study where we had students 00:45:27.000 --> 00:45:31.000 use the circuit construction kit in lab. And they either 00:45:31.000 --> 00:45:36.000 used the simulation or they used the real world equipment 00:45:36.000 --> 00:45:39.000 and did sort of parallel similar activities for the two 00:45:39.000 --> 00:45:42.000 groups. And then everybody got a learning challenge with 00:45:42.000 --> 00:45:47.000 the real world equipment at the end. And the students 00:45:47.000 --> 00:45:51.000 that had learned with the simulation were able to both 00:45:51.000 --> 00:45:54.000 build -- build the circuit faster or as fast as the 00:45:54.000 --> 00:45:57.000 students that had learned with the real equipment. So they 00:45:57.000 --> 00:46:00.000 didn't really have a challenge with building with real 00:46:00.000 --> 00:46:05.000 equipment. Then they actually generated better explanations 00:46:05.000 --> 00:46:09.000 about how the real equipment was used -- was working. So 00:46:09.000 --> 00:46:12.000 somewhere they had perhaps a better model of how circuits 00:46:12.000 --> 00:46:15.000 worked. And that really helped them when they turned 00:46:15.000 --> 00:46:21.000 to the real equipment. So that provides a little insight 00:46:21.000 --> 00:46:22.000 . But I would say it is probably really content 00:46:22.000 --> 00:46:24.000 specific and sim 00:46:24.000 --> 00:46:30.000 specific. » TOM HELIKAR: Okay. Thank 00:46:30.000 --> 00:46:33.000 you so much again. I'm sorry we can't get to all of the 00:46:33.000 --> 00:46:36.000 questions. Thank you so much, Dr. Perkins, for being here 00:46:36.000 --> 00:46:37.000 with us. And thank you for sharing your work.