Category Archives: Evaluation

Encouraging Visitor Reflection around STEM

Posted on by | Leave a reply

Do visitors use STEM reasoning when describing their work in a build-and-test exhibit? This is one of the first research questions we’re investigating as part of the Cyberlab grant, besides whether or not we can make this technology integration work. As with many other parts of this grant, we’re designing the exhibit around the ability to ask and answer this question, so Laura and I are working on designing a video reflection booth for visitors to tell us about what happened to the structures they build and knock down in the tsunami tank. Using footage from the overhead camera, visitors will be able to review what happens, and hopefully tell us about why they created what they did, whether or not they expected it to survive or fail, and how the actual result fit or didn’t match what they hoped for.

We have a couple of video review and share your thoughts examples we drew from; The Utah Museum of Natural History has an earthquake shake table where you build and test a structure and then can review footage of it going through the simulated quake. The California Science Center’s traveling exhibit Goosebumps: the Science of Fear also allows visitors to view video of expressions of fear from themselves and other visitors filmed while they are “falling”. However, we want to take these a step farther and add the visitor reflection piece, and then allow visitors to choose to share their reflections with other visitors as well.

As often happens, we find ourselves with a lot of creative ways to implement this, and ideas for layer upon layer of interactivity that may ultimately complicate things, so we have to rein our ideas in a bit to start with a (relatively) simple interaction to see if the opportunity to reflect is fundamentally appealing to visitors. Especially when one of our options is around $12K – no need to go spending money without some basic questions answered. Will visitors be too shy to record anything, too unclear about the instructions to record anything meaningful, or just interested in mooning/flipping off/making silly faces at the camera? Will they be too protective of their thoughts to share them with researchers? Will they remain at the build-and-test part forever and be uninterested in even viewing the replay of what happened to their structures? Avoiding getting ahead of ourselves and designing something fancy before we’ve answered these basic questions is what makes prototyping so valuable. So our original design will need some testing with probably a simple camera setup and some mockups of how the program will work for visitors to give us feedback before we go any farther with the guts of the software design. And then eventually, we might have an exhibit that allows us to investigate our ultimate research question.

Changing the Rules?

Posted on by | Leave a reply

For those of you just joining us, I’m developing a game called Deme for my master’s project. It’s a tactical game that models an ecosystem, and it’s meant primarily for adults. I’m studying how people understand the game’s mechanics in relation to the real world, in an effort to better understand games as learning and meaning-making tools.

I stumbled across Roll20, quite by accident, while reading the PA Report. What I like about Roll20 is the fact that your table session can be shared as a link (apparently—I haven’t started digging yet as I only found out about it a few hours ago). Also, each token can be assigned a hit counter. Damage tracking is something of a hassle in Deme’s current incarnation.

I’ll have more to report after I play around with this for a while. Moving the game from one incarnation and environment to another has forced me to think of it as a system, rather than a product. I want Deme to be portable, and a robust system can be used with just about any tabletop, real or virtual. For an example of a game system, see Wizards of the Coast’s d20 System. The d20 System happens to be a handy model for quantizing events and behaviors—handy enough to inform the data collection framework for our observation systems in the Visitor Center.

Of course, Deme cannot be run single-player as a tabletop game. That’s a double-edged sword. A tabletop game (even a virtual one) is an immediate social experience. A single-player game is a social experience too, but it’s an asynchronous interaction between the developer(s) and the player. I rather like the tabletop approach because each species has a literal voice. The unearthly torrent of resulting qualitative data may be tough to sort out, but I think that’s a good problem to have so long as I know what I’m looking for.

At this phase, the tabletop version is still officially—as much as I can make something official—just a pilot product. I don’t know if it will become something more, but I feel like it deserves a shot.

Wordle You Say? Or, how to use word clouds to look at your data

Posted on by | 2 Replies

I have been shuffling through data from the Exploratorium’s scientist-in-residence (SIR) project and I started thinking about what data (and the kinds of ways data) can or should be shared on a blog.  For now, I am going to share a few word clouds of raw data.  These do not illustrate full sentences nor can you tell which participant said what.

Each of these word clouds was based off of a survey question that I wrote and administered.

Visitors to the exhibition space were asked, upon leaving, “What would you tell a friend this space was about?”  The word cloud below contains data from the March residency, which focused on severe storm science (with scientists from NOAA’s National Severe Storm Lab).

 

The Exploratorium Explainers were an integral part of this project.  At the end of the second year I asked all of the Explainers, the Lead Explainers, and the Explainer managers to voluntarily complete the online survey.

Here is how Explainer managers responded to “Describe the impacts of this project on the scientists.”

 

While the Explainer survey was quite long and there is a lot of rich data there, I want to focus on their thoughts about the iPad.  The iPad was incorporated into the exhibition space as a mediating tool (as specified in the grant proposal).  I asked the Explainers “Where and how do you think the iPad was incorporated throughout the project?”  Their response…

 

 

So, what can we gain from word clouds?  It is certainly one way to look at raw data.  Thoughts?

 

Making Waves: Bringing Tsunamis to Washington DC – Part 1

Posted on by | Leave a reply

OSU ran three outreach activities at the 46th annual Smithsonian Folklife Festival, and we took the chance to evaluate the Wave Lab’s Mini-Flume wave tank activity, a related but different activity to the wave tanks in the HMSC Visitor Center.

Three activities were selected by the Smithsonian Folklife committee to best represent the diversity of research conducted at OSU, as well as the University’s commitment to sustainable solutions and family education: Tech Wizards, Surimi School, and the O.H. Hinsdale Wave Lab’s Mini-Flume activity. Tech Wizards was set up in the Family Activities area of Folklife, and Surimi School and the Mini-Flume activity shared a tent in the Sustainable Solutions area.

Given the anticipated number of visitors to the festival, and my presence as the project research assistant, we decided it would be a great opportunity to see how well people thought the activity worked, what they might learn, and what they liked or didn’t – core questions in an evaluation. The activity was led by Alicia Lyman-Holt, EOT director at the O.H. Hinsdale Wave Lab, and I developed and spearheaded the evaluation. To make the activity and evaluation happen, we also brought four undergraduate volunteers from OSU and two from Howard University in D.C, plus both the OSU Alumni Association and the festival supplied volunteers on an as-needed basis. We also wanted to try out data collection using iPads and survey software we’re working with in the FCL Lab.

Due to the sheer numbers of people we thought would be there, as well as the divided attentions of everyone, we decided to go with a straightforward survey. We ended up only collecting a small number of what we anticipated due to extreme heat, personnel, and divided attention of visitors – after they spent a lot of time with the activity, they weren’t always interested in sticking around even for a short survey.

I’m currently working on data analysis. Stay tuned for more information on the evaluation, the process, and to learn how we did on the other side of the continent.

So much data, so little time.

Posted on by | 4 Replies

Ok, I guess I am following suit and forgot to post on Friday! I don’t have quite as good of an excuse as Katie. Instead of prepping for conferences I was recovering from a vacation.

I thought it might be nice to provide an update about the Exploratorium project, where NOAA scientists are embedded on the museum floor with the Explainers (Exploratorium front-line staff consisting of young adults). I have collected so much data for this project I am beginning to feel overwhelmed.

Here’s the data that I have collected:
– Formal Interviews with each of the four groups of scientists, both before and after their experience.
– Informal interviews with all of the scientists. These were done in the time walking back to the hotel or when grabbing lunch. Both great times to collect data!
– Interviews with the two Explainer managers plus a survey with open- and closed-ended questions at the end of year 2.
– Interviews with each of the lead Explainers, 8 total. Also, lead Explainers during year 2 completed a survey with open- and closed-ended questions.
– Pre- mid- and post- data for what Explainers think atmospheric sciences is and what atmospheric scientists do. This was not done during the first year topic of ocean sciences.
– I also provided an optional survey for all Explainers so they could share their thoughts and opinions about the project. This provided a reflection opportunity for the Explainers that were not lead Explainers during the project.
– Visitor surveys about their experience in the scientists’ installation. During year 2 these were collected in both paper form and using survey software on the iPad.
– Field notes during meetings and time on the museum floor. During year 2 the field notes were taken on the iPad using survey software.
– And lastly…personal daily reflections.

So the question is “now what?” This data provides opportunities for triangulation but where does one start? I’m spending my final month of summer trying to figure that out.

Hopefully my next blog post will showcase my progress and some findings.

Some last thoughts (for now?) from Diana

Posted on by | Leave a reply

Our Summer Scholars’ time is drawing to a close at the end of the month, so we’re hearing some final words, at least for the moment, from some of them, starting with Diana:

“These past weeks have been filled with things that I never thought would happen and have surprised me in the most spectacular ways.  First, I went on vacation to Vancouver, BC and Seattle, WA which was a memorable experience.  I was able to see behind the scenes tanks and animals at the Vancouver Aquarium and even got to see Leonardo Da Vinci’s works in person as well as King Tutankhamen’s burial chamber items.  This vacation was a nice break from the craziness of the visitor’s center and refreshed me for another few weeks as an education intern.  The moment I returned many volunteers and other workers at Hatfield were asking about my vacation.  Even this little thing made me feel fully welcomed into the Hatfield family.  One of the first things that occurred when I returned was that many unannounced summer camps came into the visitor’s center which is always an experience.  Yet, one of the most progressive things I did was creating new signs for me shoreline erosion tank.  This time one of my mentors Mark Farley and I created 2 different signs compared to one long sign.  One sign said “The Erosion Problem” with photos of me showing how to use the paddle to create waves and see the erosion of the sand.  The second sign said “The Erosion Solution” and gave the visitors a chance to try 1 of 3 different protective strategies for beach/shoreline erosion.  These new larger signs seem to be working well for now.  I can already see a difference in the behavior of children and families when they come to wave tank; instead of sand castle building, they actually read the sings and follow the directions.

The visitor’s center also had some crazy moments.  We had Micro A and Micro B tanks overflow into the VC overnight and leave a lake in the surrounding area an inch deep.  That lake was an interesting mess to clean, but created a wonderful learning moment.  I was able to watch the aquarists and learn how to put on new filter bags as well as rework the tanks.  I was challenged to follow the pipes and figure out where the water went such as the outflow and inflow pipes.  Other crazy moments that occurred were people trying to put their whole hand inside of anemones or trying to crawl inside of the touch tank to touch the different fish.  While all of this was going on I also got to have some spectacular moments in the VC.

These spectacular moments occurred when the Aquarists took me under their wing and showed me some impressive things.  First, I got to see a fish necropsy which was highly informative and taught me new dissections skills.  I was also taught how to kill invasive coral apitasia with lemon juice.  I was able to inject a few micrometers of lemon juice into each invasive apitasia, which kills it almost instantaneously.  The apitasia tries as a defensive mechanism to spit its own guts out, but the lemon juice is too acidic.  I also learned through this process how to siphon a tank and change out the water while balancing the acidity in the water with baking soda, thus making the seawater more neutral.  Yet, the most spectacular thing I learned with the aquarists was how to feed all the animals in the Hatfield Marine Science Center.  I learned how and what to feed each animal except the octopus in the visitor’s center, which took a long time but was completely worth it.  The amount of knowledge I learned during that time was amazing and I will not forget anytime soon.  This entire summer has been a learning experience, but definitely a fun one that I shall remember for the rest of my life.”