I want to talk today about what many of us here have alluded to in other posts: the approval (and beyond) process of conducting ethical human research. What grew out of really really unethical primarily medical research on humans many years ago now has evolved into something that can take up a great deal of your research time, especially on a large, long-duration grant such as ours. Many people (including me, until recently) thought of this process as primarily something to be done up-front: get approval, then sort of forgotten about except for the actual gaining of consent as you go and unless you significantly change your research questions or process. Wrong! It’s a much more constant, living thing.

We at the Visitor Center have several things that make us a weird case for our Institutional Review Board office at the university. First, even though it is generally educational research that we do, as part of the Science and Mathematics Education program, our research sites (the Visitor Center and other community-based locations) are not typically “approved educational research settings” such as classrooms. Classrooms have been so frequently used over the years that they have a more streamlined approval process unless you’re introducing a radically different type of experiment. Second, we’re a place where we have several types of visitor populations: the general public, OSU student groups, and K-12 school and camp groups, who each have different levels of privacy expectations, requirements for attending (public: none, OSU school groups: may be part of a grade), and thus different levels and forms of obtaining consent to do research required. Plus, we’re trying to video record our entire population, so getting signatures from 150,000+ visitors per year just isn’t feasible. However, some of the research we’re doing will be our typical video recording that is more in-depth than just the anonymized overall timing and tracking and visitor recognition from exhibit to exhibit.

What this means is a whole stack of IRB protocols that someone has to manage. At current count, I am managing four: one for my thesis, one for eyetracking in the Visitor Center for looking at posters and such, one for a side project involving concept mapping, and one for the general overarching video recording for the VC. The first three have been approved and the last one is in the middle of several rounds of negotiation on signage, etc., as I’ve mentioned before. Next up we need to write a protocol for the wave tank video reflections, and one for groundtruthing the video-recording-to-automatic-timing-tracking-and-face-recognition data collection. In the meantime, the concept mapping protocol has been open for a year and needs to be closed. My thesis protocol has bee approved nearly as long, went through several deviations in which I did things out of order or without getting updated approval from IRB, and now itself soon needs to be renewed. Plus, we already have revisions to the video recording protocol staff once the original approval happens. Thank goodness the eyetracking protocol is already in place and in a sweet spot time-wise (not needing renewal very soon), as we have to collect some data around eyetracking and our Magic Planet for an upcoming conference, though I did have to check it thoroughly to make sure what we want to do in this case falls under what’s been approved.

On the positive side, though, we have a fabulous IRB office that is willing to work with us as we break new ground in visitor research. Among them, us, and the OSU legal team we are crafting a strategy that we hope will be useful to other informal learning institutions as they proceed with their own research. Without their cooperation, though, very little of our grand plan would be able to be realized. Funders are starting to realize this, too, and before they make a final award for a grant they require proof that you’ve discussed the basics of your project at least with your IRB office and they’re on board.

As the lab considers how to encourage STEM reflection around the tsunami tank, this recent post from Nina Simon at Museum 2.0 reminds us what a difference the choice of a single word can make in visitor reflection:

“While the lists look the same on the surface (and bear in mind that the one on the left has been on display for 3 weeks longer than the one on the right), the content is subtly different. Both these lists are interesting, but the “we” list invites spectators into the experience a bit more than the “I” list.”

So as we go forward, the choice not only of the physical booth set up (i.e. allowing privacy or open to spectators), but also the specific wording can influence how our visitors choose to focus or not on the task we’re trying to investigate, and how broad or specific/personal their reflections might be. Hopefully, we’ll be able to do some testing of several supposedly equivalent prompts as Simon suggests in an earlier post as well as more “traditional” iterative prototyping.

One of the great things about being in graduate school is the variety of experiences that are available in the competition for funding. Each one offers unique opportunities for growth and learning, but some are certainly more challenging than others. I’m currently working on a project that utilizes my skills in web design, but the requirements of the project are beyond what I was formerly able to perform. The past few weeks have been full of learning and expanding and lots of trial and error. I finally found a few useful printed books (especially the Drupal Bible) and with their help I’ve been more successful in building the website with the functionality I envisioned. There is still quite a ways to go, and it would be easier if I had direct access to the servers, but I’m still proud of the work I’ve been able to do and look forward to adding “web development” to my Curriculum Vitae.

(Since the website is still under quite a bit of construction, I have chosen not to release the URL at this point.)

It seems that a convenience sample really is the only way to go for my project at this stage. I have long entertained the notion that some kind of randomization would work to my benefit in some abstract, cosmic way. The problem is, I’m developing a product for an established audience. As much as I’d like to reach out and get new audiences interested, that will have to come later.

That sounds harsh, which is probably why I hadn’t actually considered it until recently. In reality, it could work toward my larger goal of bringing in new audience members by streamlining the development process.

I’ve discovered that non-gamers tend to get hung up on things that aren’t actually unique to Deme, but are rather common game elements with which they’re not familiar. Imagine trying to design a dashboard GPS system, then discovering that a fair number of your testers aren’t familiar with internal combustion engines and doubt they will ever catch on. I need people who can already drive.

Games—electronic, tabletop or otherwise—come with a vast array of cultural norms and assumptions. Remember the first time you played a videogame wherein the “Jump” button—the button that was just simply always “Jump” on your console of choice—did something other than jump?* It was like somebody sewed your arms where your legs were supposed to be, wasn’t it? It was somehow offensive, because the game designers had violated a set of cultural norms by mapping the buttons “wrong.” There’s often a subtle ergonomic reason that button is usually the “Jump” button, but it has just as much to do with user expectations.

In non-Deme news, we’re all excited to welcome our new Senior Aquarist, Colleen Newberg. She comes to us from Baltimore, but used to work next door at the Oregon Coast Aquarium. I learned last week that she is a Virginian, leaving Sid as the lone Yankee on our husbandry team. We’ve got some interesting things in the works, and Collen has been remarkably cool-headed amidst a torrent of exhibit ideas, new and changing protocols and plumbing eldritch and uncanny.

 

*I’ve personally observed that button-mapping has become less standardized as controllers have become more complex. I could be wrong, though—my gameplay habits do not constitute a large representative sample. Trigger buttons, of course, would be an exception.

Do visitors use STEM reasoning when describing their work in a build-and-test exhibit? This is one of the first research questions we’re investigating as part of the Cyberlab grant, besides whether or not we can make this technology integration work. As with many other parts of this grant, we’re designing the exhibit around the ability to ask and answer this question, so Laura and I are working on designing a video reflection booth for visitors to tell us about what happened to the structures they build and knock down in the tsunami tank. Using footage from the overhead camera, visitors will be able to review what happens, and hopefully tell us about why they created what they did, whether or not they expected it to survive or fail, and how the actual result fit or didn’t match what they hoped for.

We have a couple of video review and share your thoughts examples we drew from; The Utah Museum of Natural History has an earthquake shake table where you build and test a structure and then can review footage of it going through the simulated quake. The California Science Center’s traveling exhibit Goosebumps: the Science of Fear also allows visitors to view video of expressions of fear from themselves and other visitors filmed while they are “falling”. However, we want to take these a step farther and add the visitor reflection piece, and then allow visitors to choose to share their reflections with other visitors as well.

As often happens, we find ourselves with a lot of creative ways to implement this, and ideas for layer upon layer of interactivity that may ultimately complicate things, so we have to rein our ideas in a bit to start with a (relatively) simple interaction to see if the opportunity to reflect is fundamentally appealing to visitors. Especially when one of our options is around $12K – no need to go spending money without some basic questions answered. Will visitors be too shy to record anything, too unclear about the instructions to record anything meaningful, or just interested in mooning/flipping off/making silly faces at the camera? Will they be too protective of their thoughts to share them with researchers? Will they remain at the build-and-test part forever and be uninterested in even viewing the replay of what happened to their structures? Avoiding getting ahead of ourselves and designing something fancy before we’ve answered these basic questions is what makes prototyping so valuable. So our original design will need some testing with probably a simple camera setup and some mockups of how the program will work for visitors to give us feedback before we go any farther with the guts of the software design. And then eventually, we might have an exhibit that allows us to investigate our ultimate research question.

For those of you just joining us, I’m developing a game called Deme for my master’s project. It’s a tactical game that models an ecosystem, and it’s meant primarily for adults. I’m studying how people understand the game’s mechanics in relation to the real world, in an effort to better understand games as learning and meaning-making tools.

I stumbled across Roll20, quite by accident, while reading the PA Report. What I like about Roll20 is the fact that your table session can be shared as a link (apparently—I haven’t started digging yet as I only found out about it a few hours ago). Also, each token can be assigned a hit counter. Damage tracking is something of a hassle in Deme’s current incarnation.

I’ll have more to report after I play around with this for a while. Moving the game from one incarnation and environment to another has forced me to think of it as a system, rather than a product. I want Deme to be portable, and a robust system can be used with just about any tabletop, real or virtual. For an example of a game system, see Wizards of the Coast’s d20 System. The d20 System happens to be a handy model for quantizing events and behaviors—handy enough to inform the data collection framework for our observation systems in the Visitor Center.

Of course, Deme cannot be run single-player as a tabletop game. That’s a double-edged sword. A tabletop game (even a virtual one) is an immediate social experience. A single-player game is a social experience too, but it’s an asynchronous interaction between the developer(s) and the player. I rather like the tabletop approach because each species has a literal voice. The unearthly torrent of resulting qualitative data may be tough to sort out, but I think that’s a good problem to have so long as I know what I’m looking for.

At this phase, the tabletop version is still officially—as much as I can make something official—just a pilot product. I don’t know if it will become something more, but I feel like it deserves a shot.