Katie Stofer « The Free-Choice Learning Laboratory at HMSC

About Katie Stofer

Research Assistant Professor, STEM Education and Outreach, University of Florida PhD, Oregon State University Free-Choice Learning Lab

The funny thing about having the money to revise an exhibit that’s already installed is that, well, there’s already a version out there that people are using, so it sometimes falls to lower priority. Even if you know there are a lot of things that could be better about it. Even if, as I said, the money is in hand to update it. That’s another thing about outreach being still sort of the afterthought of scientific grants; even when the scientists have to report on their grant progress, if the outreach effort isn’t in that report, well, the grant agencies aren’t always so concerned about that.

So we’re trying to revise our salmon fisheries exhibit, and we have a concept, but we have to constantly remind ourselves to make progress on it. It’s an item on my list that is “Important, but not Urgent,” (one of those Seven Habits of Highly Effective People things), and it keeps being shoved out for the Urgent but Not Important and even Not Urgent, Not Important (but way more interesting!) things. I think it’s like revising a paper; sometimes, the work it takes to come up with the ideas in the first place is far more interesting than more nitpicky revisions. And, again, a lot less urgent. So, we’re setting interim milestones to make progress: 1) we have our visualization collaborator working on new images, 2) we have text to re-organize and re-write, and 3) we have a basic logic about the new version that we’ve sent to the developers so they can write the automated data collection tool that records what a user does and when and for how long. So, we feel confident in progress since we’ve farmed out 1 and 3, but that still leaves #2 for us to chip away at. And sometimes, that’s what it takes. A little bit of time here, a little bit there, and eventually, there’s a lot less to get done and the task seems less overwhelming. Maybe the blog will help keep us accountable for getting this exhibit done by … the end of the summer?

We heard recently that our developer contractors have decided they have to abandon their efforts to make the first facial recognition system they investigated work. It was a tough call; they had put a lot of effort into it, thinking many times if they could just tweak this and alter that, they would get better performance than 60%. Alas, they finally decided it was not going to happen, at least without a ridiculous amount of further effort for the eventual reward. So, they are taking a different tack, starting over, almost, though they have lots of lessons learned from the first go-round.

I think this indecision about when it makes sense to try and fix the leaking ship vs. abandon ship and find another is a great parallel with exhibit development. Sometimes, you have a great idea that you try with visitors, and it flops. You get some good data, though, and see a way you can try it again. You make your changes. It flops again, though maybe not quite as spectacularly. Just enough better to give you hope. And so on … until you have to decide to cut bait and either redesign something for that task entirely or, if you’re working with a larger exhibition, find another piece to satisfy whatever learning or other goals you had in mind for the failed piece.

In either situation, it’s pretty heartbreaking to let go of all that investment. When I first started working in prototyping, this happened to our team designing the Making Models exhibition at the Museum of Science, Boston. As an intern, I hadn’t invested anything in the failed prototype, but I could see the struggle in the rest of the team, and it made such an impression that I recall it all these years later. Ultimately, the final exhibit looks rather different from what I remember, but its success is also a testament to the power of letting go. Hopefully, we’ll eventually experience that success with our facial recognition setups!

Our actual eyetracker is a bit backordered, so we’ve got a rental for the moment. It’s astoundingly unassuming looking, just (as they picture on their web site) basically a small black bar at the bottom of a 22” monitor, plus the laptop to run the programs. When I took it out of the box, it fires up the operating system and there are the icons just sitting on the desktop, with a little warning that we shouldn’t mess with any settings, install a firewall or anti-virus software for risk of messing up the primary function. They have branded the screen with a little decal from their company, but otherwise, it’s just a laptop with an attached monitor.

The actual getting started is a bit complicated. I’m usually the one to pooh-pooh the need for “readme” documents, but I would have liked one here to tell me which program is which. That’s the thing – the software is powerful, but it has a bit of a steep learning curve. The “quick start” guide has several steps before you even think about calibrating a subject. We got stuck on the requirement to get Ethernet hooked up since we tried to set up in a tech closet and OSU otherwise has pretty widespread wireless coverage. Harrison had to run a 50’ cable from Mark’s office down the hallway to the closet.

Looks like the next step is some pretty intense work understanding how to set up an experiment in a different software program. This is where a “test” experiment just to start learning how to use the system would be good. That’s the sort of icon I need in the middle of the desktop. It reminds me of my first job as a research assistant, where I was registering brain images to a standard. The researchers had written a program to rotate the images to line up and then match certain features to the standard to stretch or compact the images as necessary, but there was no manual or quick start. My supervisor had to show me all the steps, what button did what, which order, etc. It was a fairly routine process, but it was all kept in someone’s head until I wrote it down. The pdfs here are a great start, but there still seems to be a step missing. Stay tuned!

Prototyping describes the process of creating a first-version exhibit, then testing it out with visitors, and redesigning. Often, we iterate this several times, depending on monetary and time budgets. It’s usually a fruitful way to find out not only what buttons confuse people, but also what they enjoy playing with and what great ideas totally bomb with users.

The problem with prototyping, as with many data collection processes, is that you have to ask the right questions to get useful answers. We are currently re-developing an interactive about how scientists use ocean data to make predictions about salmon populations for future harvests. The first round surveys revealed some areas of content confusion and some areas of usability confusion. Usability confusion is easy to re-work usually, but content confusion is harder to resolve, especially if your survey questions were confusing to the visitors.

This was unfortunately the case with the survey I made up, despite a few rounds of re-working it with colleagues. The survey had multiple-choice questions which were fairly straightforward, but it was the open-ended questions that tripped people up, making the results a bit harder to interpret and know what to do with. The moral of the story? Prototype (a.k.a. pilot) your survey, too!

Friday we continued our perfect technology quest, this time focusing on audio. While we actually want the cameras to capture the video in an overlapping manner, so that we can track visitors from one spot to another and be able to see their faces no matter what angle they face, it turns out that the audio is a different matter. Due to the acoustics in the Center, if we’re not careful, a mic at the front desk will pick up voices 25 feet away at the wave tank, not only muddling the audio we want to hear from the front desk, but also perhaps turning on extra cameras and recording unrelated video.

In order to localize the audio to particular people and in order to understand speech clearly, we’ll use so-called near field recording (up-close to the speaker rather than capturing a whole room). We’ll also need to input multiple mics into certain cameras in order to have audio coverage with minimal wiring in the way of exhibits. Beyond that, though, was the question of what kind of pickup pattern we need – whether the mic records audio straight in front of it, in front and behind, or all around, for example.

With help from audio technicians from the main campus who were out to work the retirement of one NOAA research vessel and the welcoming of another, we discussed the ins-and-outs of particular shapes of recording areas. Probably our best bet in most cases will be a carotid, or heart-shaped, mic, which gets mostly what’s in front of the mic, but not in a straight line, and some of what’s behind the mic. The exact sizes of the patterns can often be tuned, which in our case again will be crucial as we begin to determine how visitors use particular exhibits, where they stand when they talk to one another, and especially how they might move up and down as they interact with people of different ages and heights.

As usual, one of our biggest challenges is trying to retrofit this recording equipment into an already built space, and a space built with weird angles, less-than-optimal acoustics, somewhat unpredictable speaker locations, and often loud but inconsistent ambient noise such as the 65-decibel running water in the touch pools. But hey, that’s why we’re trying it, to see if it’s even possible and beyond possible, helpful to our research.

Harrison used an interesting choice of phrase in his last post: “time-tested.” I was just thinking as I watched the video they produced, including Bill’s dissection, that I don’t know what we’ve done to rigorously evaluate our live programming at Hatfield. But it is just this sort of “time-tested” program that our research initiatives are truly trying to sort out and put to the test. Time has proven its popularity, data is necessary to prove its worth as a learning tool. A very quick survey of the research literature doesn’t turn up much, though some science theater programming was the subject of older studies. Live tours are another related program that could be ripe for investigation.

We all know, as humans who recognize emotions in others, how much visitors enjoy these sorts of programs and science shows of all types. However, we don’t always apply standards to our observations, such as measuring specific variables to answer specific questions. We have a general sense of “positive affect” in our visitors, but we don’t have any data in the form of examples of quotes or interviews with visitors to back up our thoughts. Yet.

A good example of another need for this was in a recent dissertation defense here at OSU. Nancy Staus’ research looked at learning from a live program, and she interviewed visitors after watching a program at a science center. She found, however, that the presenter of the program had a lot of influence on the learning simply by the way they presented the program: visitors recalled more topics and more facts about each topic when the presentation was more interactive than scripted. She wasn’t initially interested in differences of this sort, but because she’d collected this sort of data on the presentations, she was able to locate a probable cause for a discrepancy she noted. So while this wasn’t the focus of her research (she was actually interested in the role of emotion in mediating learning), it pointed to the need for data to not only back up claims, but also to lead to explanations for surprising results and open areas for further study.

That’s what we’re working for: that rigorously examining these and all sorts of other learning opportunities becomes an integral part of the “time-honored tradition.”

The Free-Choice Learning Laboratory at HMSC

Informal science education research

Author Archives: Katie Stofer

About Katie Stofer

Swimming Upstream

The End of the Road?

The eyetracker is here! … well, sort of.

The Problem with Prototyping

Moving in the right direction

“Time-tested” traditions

Contact Info