Tag Archives: evaluation

Preparing for a different type of tsunami

Posted on by | Leave a reply

Another of our interns, (actually, they are all Oregon Sea Grant Summer Scholars), Nick, lets us know what it’s like trying to prepare for the “visitor tsunami” that’s bound to occur when we get our third wave tank all set, based somewhat on the inundation he gets as an on-floor interpreter:

“Working as a docent for the front desk and touch pools has provided abundant opportunity to interact with the public and I seem to learn as much information as I provide. Visitors ask so many interesting questions and also ask about local marine events: how is the Sea Turtle doing that washed up on the shore, inquiries about the tsunami debris and recently the Brown Pelican crisis at the Yaquina Head seabird colony. Visitors also bring in some unusual items and ask for help identifying them; one man brought in an orca tooth that he had discovered eroding out of a cliff.

Among my favorite duties is acting as the guide for the daily tour of the Yaquina Bay estuary describing the marine plants and animals of the bay. Participants especially like discovering the tiny crabs that are often living under the very rocks they are standing on. It is pretty rewarding and members of the tour group have often told me that after taking the tour, they now want to become marine biologists.  I have also been helping with the Ocean Quest multimedia presentation in the auditorium. We have been working out the bugs in the presentation and it is finally at a point where we are happy with it.

Our main project involves working on three wave tank exhibits. Brian has been working with the wave energy exhibit, designing an experimental “wave power” device that looks like a futuristic mechanical snake. Diana has been working with the erosion tank and has had to be vigilant in order to prevent the “sandy beach” from becoming a mess from enthusiastic children. My project has been working with our tsunami tank. I have been working on designing ideal tsunami proof structures as well as showing buildings that will not be able to survive the wave. Using Legos as building materials, I have attempted to construct scale models of different building to see if the various designs are demolished or not with the wave tank.

The tsunami project has had some problems associated with it. For starters we have had endless computer glitches and malfunctions that often make it difficult just to run the machine. We have also been experimenting with different lengths of continental slope (represented by an acrylic ramp). Additionally, we found that Legos seem to stick together really, really well….sometimes so well that buildings that should be demolished are still left standing!  We have had to resort to sanding the individual bricks so that they do not stick together as well and will better represent actual building materials. We are hopeful that these problems will be fixed within a few weeks when we plan to open the tank to the public. The educational intent of the display is to challenge visitors to construct a building that can stand up to a tsunami wave. We are confident it has the makings of a fun and interesting exhibit and hope it will be very popular with our visitors!”

 

Using the Looxies at PAM

Posted on by | Leave a reply

So last week I posted about the evaluation project underway at Portland Art Museum (PAM) and wanted to give a few more details about how we are using the looxcie cameras.

 

Looxcies are basically bluetooth headsets, just like the ones regularly seen used with cell phones, but with a built in camera. I am currently using them as part of my research emcompassing docent-visitor interactions, and decided to use them as a data collection tool because of their ability to generate a good quality “visitor-eye-view” of the museum experience. I personally feel their potential as a research/evaluation tool in informal settings are endless, and had some wonderful conversations with other education professionals at the National Marine Educators Association conference in Anchorage, AK recently about where some other possibilities could lie – including as part of professional development practice for educators and exhibit development.

At PAM, the looxices will be used to capture that view when visitors interact with exhibition pieces, specifically those related to their Museum Stories and Conversation About Art video-based programs. Here, fitting visitors with looxcies will enable us to capture the interactions and conversations visitors have about the art on display as they visit the museum. The video data gained here can then be analyzed for repeating themes around what and how visitors talk about art in the museum setting.

During our meeting with Jess Park and Ally Schultz at PAM, we created some test footage to help with training other museum staff for the evaluation procedures. In the clip below, Jess and Ally are looking at and discussing some sculpture pieces, and were both wearing looxcies to give them a sense of how they feel to the user. This particular clip is from Ally’s perspective, and you’ll notice even Shawn and I have a go a butting in and talking about art with them!

What’s exciting about working with the looxcies, and with video observations in general, is how much detail you can capture about the visitor experience, down to what they are specifically looking at, how long they look at it, and even if they nod their head in agreement with the person they are conversing with. Multimodal discourse eat your heart out!

 

Conversations About Conversations at Portland Art Museum

Posted on by | 1 Reply

Yesterday Shawn and I met with Jess Park at Portland Art Museum (PAM) about an exciting new evaluation project utilizing our looxcie cameras. We had some great conversation about how to capture visitor conversation and interactions in relation to PAM’s Museum Stories and Conversations About Art video-based program. The project will be one the first official evaluation partnerships we have developed under the flag of FCL lab!

PAM has developed these video-based experiences for visitors in order to deepen visitors’ engagement with objects, with each other, and with the museum.  Museum Stories features short video presentations of museum staff talking about specific objects in the collection that have some personal meaning for them. All videos are available on touch screen computers in one gallery of the museum, which also houses the areas where the stories are recorded as well as some of the objects from the museum featured in the stories.  These videos are also available on-line.  Conversations about Art is a series of short videos featuring conversations among experts focused on particular objects in the museum’s collection.  These are available on hand-held devices provided by the museum, as downloads to visitors’ personal hand-held devices, and on the museum website. PAM is now looking to expand the program, and wishes to document some of the predicted and unexpected impacts and outcomes of these projects for visitors. The evaluation will recruit visitors to wear the looxcie cameras during their visit to the pertinent exhibits, including that of object stories. We will likely also be interviewing some of the experts/artists involved in creating the  videos.

We spent time going over the looxcie technologies and how best to recruit visitors in the Art Museum space. We also created some test clips to help the PAM folks working on the evaluation better understand the potential of the video data collection process. I will post a follow up next week with some more details about how we’re using the looxcies.

Shawn and I come back from PAM feeling like the A-Team – we love it when an evaluation plan comes together.

Rejection OR How I learned to stop worrying and love the research process

Posted on by | Leave a reply

Rejection. It’s an inevitable part of recruiting human subjects to fill out your survey or try out your exhibit prototype. It’s also hard not to take it personally, but visitors have often paid to attend your venue today and may or may not be willing to sacrifice some of their leisure time to improve your exhibit.

 

[Full disclosure: this blog post is 745 words long and will take you approximately 5-10 minutes to read. You might get tired as you read it, or feel your eyes strain from reading on the computer screen, but we won’t inject you with any medications. You might learn something, but we can’t pay you.]

 

First, you have to decide beforehand which visitors you’re going to ask – is it every third visitor? What if they’re in a group? Which direction will they approach from? Then you have to get their attention. You’re standing there in your uniform, and they may not make eye contact, figuring you’re just there to answer questions. Sometimes rejection is as simple as a visitor not meeting your eye or not stopping when you greet them.

You don’t want to interrupt them while they’re looking at an exhibit, but they may turn and go a different direction before you get a chance to invite them to help you. How far do you chase them once you’ve identified them as your target group? What if they’re going to the restrooms, or leaving the museum from there? When I was asking people to complete surveys about our global data display exhibit, they were basically on their way out the door of the Visitor Center, and I was standing in their way.

 

If you get their attention, then you have to explain the study and not scare them off by making it sound like a test, with right or wrong answers, even when you have right and wrong answers. You also have to make sure that you don’t take too much of their time.

 

Then there are the visitors who leave in the middle of the experiment, deciding they didn’t know what they were getting into, or being drawn away by another group member.

 

Oh, you’re still there? This isn’t too long? It’s not lunchtime, planetarium show time or time to leave for the day? I’ll continue.

 

If you have an IRB or other informed consent document, this can be another hurdle. If you’re not careful about what you emphasize, visitors could focus on the “Risks” section that you must tell them about. In exhibit evaluation and research, this is often only fatigue or discomfort when someone feels they don’t know the right answer (despite assurances that no one is judging them). But of course, you have to be thorough and make sure they do understand the risks and benefits, who will see the information they give and how it will be used. Luckily, we don’t often need to collect personal information, even signatures, if we’re not using audio or video recording.

 

Then there is the problem of children. We want to assess the visit with the true types of groups that we see, that is, mostly families or mixed adult-child groups. However, anyone under 18 needs to have consent given by a parent. Unfortunately, a grandparent, aunt, uncle, sister or brother doesn’t count, so you have to throw out those groups as well. Even if a parent is present, you have to make sure that you can explain the research to the youngest visitor you have permission to study (usually about 8 years old) and even worse, explain the assent process to him or her without scaring them off. As our IRB office puts it, consent is a process, a conversation, not just a form.

 

So who knows if we’re really truly getting a representative sample of our visitors? That’s definitely a question about sampling theory. Luckily for us at Hatfield, we’re working with our campus IRB office to try and create less-restrictive consent situations, as when we don’t have to get a signed consent form if that’s the only identifying information we ask visitors to provide. Maybe we’ll be able to craft a situation where over-18 family members will be able to provide consent for their younger relatives if a parent didn’t travel with them that day. Luckily, as this progresses, you’ll be able to follow it on our blog.

 

Wow, you’ve read this far? Thank you so much, and enjoy the rest of your visit.