Author Archives: Katie Stofer

About Katie Stofer

Research Assistant Professor, STEM Education and Outreach, University of Florida PhD, Oregon State University Free-Choice Learning Lab

Rejection OR How I learned to stop worrying and love the research process

Posted on by | Leave a reply

Rejection. It’s an inevitable part of recruiting human subjects to fill out your survey or try out your exhibit prototype. It’s also hard not to take it personally, but visitors have often paid to attend your venue today and may or may not be willing to sacrifice some of their leisure time to improve your exhibit.

 

[Full disclosure: this blog post is 745 words long and will take you approximately 5-10 minutes to read. You might get tired as you read it, or feel your eyes strain from reading on the computer screen, but we won’t inject you with any medications. You might learn something, but we can’t pay you.]

 

First, you have to decide beforehand which visitors you’re going to ask – is it every third visitor? What if they’re in a group? Which direction will they approach from? Then you have to get their attention. You’re standing there in your uniform, and they may not make eye contact, figuring you’re just there to answer questions. Sometimes rejection is as simple as a visitor not meeting your eye or not stopping when you greet them.

You don’t want to interrupt them while they’re looking at an exhibit, but they may turn and go a different direction before you get a chance to invite them to help you. How far do you chase them once you’ve identified them as your target group? What if they’re going to the restrooms, or leaving the museum from there? When I was asking people to complete surveys about our global data display exhibit, they were basically on their way out the door of the Visitor Center, and I was standing in their way.

 

If you get their attention, then you have to explain the study and not scare them off by making it sound like a test, with right or wrong answers, even when you have right and wrong answers. You also have to make sure that you don’t take too much of their time.

 

Then there are the visitors who leave in the middle of the experiment, deciding they didn’t know what they were getting into, or being drawn away by another group member.

 

Oh, you’re still there? This isn’t too long? It’s not lunchtime, planetarium show time or time to leave for the day? I’ll continue.

 

If you have an IRB or other informed consent document, this can be another hurdle. If you’re not careful about what you emphasize, visitors could focus on the “Risks” section that you must tell them about. In exhibit evaluation and research, this is often only fatigue or discomfort when someone feels they don’t know the right answer (despite assurances that no one is judging them). But of course, you have to be thorough and make sure they do understand the risks and benefits, who will see the information they give and how it will be used. Luckily, we don’t often need to collect personal information, even signatures, if we’re not using audio or video recording.

 

Then there is the problem of children. We want to assess the visit with the true types of groups that we see, that is, mostly families or mixed adult-child groups. However, anyone under 18 needs to have consent given by a parent. Unfortunately, a grandparent, aunt, uncle, sister or brother doesn’t count, so you have to throw out those groups as well. Even if a parent is present, you have to make sure that you can explain the research to the youngest visitor you have permission to study (usually about 8 years old) and even worse, explain the assent process to him or her without scaring them off. As our IRB office puts it, consent is a process, a conversation, not just a form.

 

So who knows if we’re really truly getting a representative sample of our visitors? That’s definitely a question about sampling theory. Luckily for us at Hatfield, we’re working with our campus IRB office to try and create less-restrictive consent situations, as when we don’t have to get a signed consent form if that’s the only identifying information we ask visitors to provide. Maybe we’ll be able to craft a situation where over-18 family members will be able to provide consent for their younger relatives if a parent didn’t travel with them that day. Luckily, as this progresses, you’ll be able to follow it on our blog.

 

Wow, you’ve read this far? Thank you so much, and enjoy the rest of your visit.

 

 

Oceans ’12 Day 3

Posted on by | Leave a reply

Day 2’s sessions ended up focusing on the Communication side of Education/Outreach/Scientific Workforce, and I think that framing it that way drew a bigger audience. One presentation on how to create a video was very similar to Ari Daniel Shapiro’s Education talk on producing radio programs or podcasts the day before, with how-to’s, but the audience was much bigger. Is it that “education” and “outreach” are scarier terms than “communicating”? If so, we educators need to think about how to make education more “do-able” for scientists if we want them to do the education to at least some extent, rather than leaving it all to education professionals.

 

We wonder, however, why education and communication are separated? Perhaps we have slightly different goals, but perhaps not: communication may have a specific outcome in mind, such as motivating people to think a certain way or do a certain thing, where education might more broadly want learners to understand how science works.

 

One afternoon talk pointed out that between science and communication, at least, it depends on your audience. For example, scientists focus on what we don’t know, whereas policymakers need to know what science does know. So in communicating and educating, we have to decide whether we’re trying to convey what science is and how it works, or whether we’re trying to convey where science is at the moment.

 

Throughout the week COSEE is hosting a series on how to do education/communication of your science. These lunch workshops that have had about 100 participants, or roughly 2 percent of the conference attendees. Again, by a show of hands on Tuesday, many of those, however, were graduate students.

 

Last night there was a panel discussion on Bridging the Cultural Gap between Scientists and the Public. I overheard one scientist at the COSEE exhibitor booth pooh-pooh the need for him to attend the panel, as he basically said he knew there was a gap but that it was the public’s problem. COSEE staff made a valiant effort to convince him that he was actually a vital part of bridging the gap, but regardless, there was still a relatively small audience for the program. However, attendees seemed to skew a little bit more toward the early- to mid-career scientists than the other education sessions (perhaps because the grad students had all run off to get beer). We had mixed feelings about the presentation because we walked away a bit more confused about what we could do. The researchers on communication and communicators on the panel offered us ideas about what the communication breakdown was, but we didn’t get a chance to discuss many practical ideas.

 

The basic premises were that it’s not a problem of literacy, but of people tending to affiliate with groups and basically only attend to information that those groups agree with, in order to maintain that affiliation. The other presentation highlighted peculiarities of the journalistic process that complicated the communication picture, such as editors who focus on minor details to up the drama factor and sell their products. So on the one hand, we need to remove the threat that holding a position on a subject would automatically mean you’d no longer be part of a group that’s important to you, that is, we have to change the “meaning” that accompanies the facts. How to do this, however, is what remains for us to figure out.

 

 

 

 

Oceans ’12 Day 2

Posted on by | Leave a reply

Okay, so I started things off as a bit of a downer. Considering this is only the third Oceans conference to include education strands, it’s great that it’s being supported.

 

However, I wonder if a large scientific conference is the best place to sell outreach to scientists for a number of reasons. For one thing, the education research sessions basically competed with the scientific sessions, almost as if it was a parallel conference, scheduled at the same time so one physically could not attend both. Shawn noted that the evening and lunch workshops on outreach are often well attended, for example (at least by graduate students), but that doesn’t get our research out there.

 

For another thing, the education presentations focused a lot on specific program evaluation results, as I mentioned yesterday. In that way, they really were not speaking to scientists who were looking to get involved in outreach, at least beyond trying to make the case for it in terms of the personal fulfillment results and opportunities for increased funding. The sessions were by and large not aimed at delivering the skills to a broad audience that people could take back to whatever institution they worked with. The specificity of many projects showed more that such programs were possible and rewarding, without offering opportunities for people at other places ways to get involved. On the other hand, Ari Shapiro of Woods Hole (and often heard on NPR) gave many how-to examples for either partnering for general media publishing or do-it-yourself podcasting and multimedia presentations. The low-cost, do-it-yourself options of course appealed to the educators in the audience as well.

 

Nevertheless, for those of us that are going back to our institutions and hoping to help the scientists we work with there, there were several interesting findings from the sessions:

 

1) Scientists are still largely unaware of the work we do, especially that there is educational literature out there about what works.

2) All participants in these programs, educators, scientists, and the ostensible “audience” each play roles as both teachers/facilitators and learners at various times. Educators and public audiences both have frequent opportunities for reflecting on their experiences during programs. Most of this occurs via feedback to each other as both groups are fairly familiar with their roles in these situations. On the other hand, the scientists often lack such opportunities outside of program evaluations, to reflect on either of their roles, or even the fact that they play both of those roles during the experience. They probably also need tools to help them do that reflection.

3) There are a lot of great programs reaching maybe 50 teachers at a time. If each of those teachers reach say 200 students each per year, that’s still only 10,000 students, with perhaps a little more via “trickle down” to other teachers that the program teachers work with. In a country with maybe 100 million students, we have a lot of work to do. And we need a lot of money to do it. And we need evidence of these things working and ways to can scale them up wherever possible.

4) We have a bit of work to do even among our education community about the value of qualitative data and what it can tell you, including the fact that there are people out there that can help analyze that data if you have it.

5) We need more research that’s applicable to a lot of situations, not just evaluations of great projects.

 

I love being in an emerging field, but some days it’s not emerging fast enough.

 

Oceans ’12

Posted on by | Leave a reply

No, it’s not the George Clooney movie sequel. It’s a chance for us to blog near-real-time about our experiences at getting the word out about what we do to the scientists we’re trying to work with. Several of the lab members are attending the Oceans 2012 meeting in Salt Lake City this week. The meeting is a joint offering of The Oceanography Society, Americal Geophysical Union, and the Association for the Sciences of Limnology and Oceanography.

The meeting is a typical science conference: many many many presentations in all sorts of subdisciplines of ocean sciences, and about one session on education, outreach, communication, evaluation, and student engagement. So, we still have an uphill battle to reach those folks who need to do “broader impacts” for their grants or just as part of the greater good. That is, like recycling, there isn’t always a personal gain, but when you consider the bigger picture, the argument is that it’s the right thing to do (thanks, Jude).

In listening to the first set of education and evlauation presentations this morning, it struck me that the evaluations almost all noted that the majority of their participants in delivering outreach were graduate students. The good news is, the graduate students were very enthusiastic about the programs and found it rewarding both by learning new ways of doing teaching and outreach and also by improving their own research at some times. This bodes well for exciting this new generation of scientists to get involved in outreach.

The bad news, though, is by and large, the practicing scientists were still missing. Are they thinking that their grad students will “take care of it” for them? Do they still find it a hassle, unrewarding, time-consuming, and an activity that basically takes away from their own research? Are they afraid of kids? Have they tried it and had a bad response (probably because they got little guidance on how to do it in the first place)? This is a large cadre of professionals that we can’t afford to ignore, no matter how excited the next generation is. We can’t let them, or ourselves as outreach and education professionals and researchers, cop out.

So, the question remains, how do we reach this population? I’m hoping to talk up our program and work as the week progresses, but this venue is challenging, to say the least, with only a few general mixers and the sheer number of simultaneous presentations.

Dare we hope in 2014 for a plenary talk about education and outreach for *all* scientists? Or educational presentations that aren’t competing with the scientific ones? Or scientists that start to attend education and outreach conferences? A woman can dream …

Worst Customer Response and Customer Service Ever: Nuance

Posted on by | 8 Replies

Wow, I thought I had seen lousy customer service, but Nuance is taking the cake. I have tried to contact the audio mining software company for the last two weeks, through their Enterprise sales web site (no response to my email), and on the phone to about 4 different phone numbers (no response to my voice messages). Tuesday the 7th I spent about 3 hours on hold waiting for their Enterprise sales departments (including multiple calls where I’d been hung up on while on hold, and voice messages on this line asking them to contact me), after trying other departments and complaining. Their other departments, however, don’t even know that the product I’m interested in exists. They did try to contact the Enterprise sales department themselves, however, but even customer service couldn’t get through! The hold “music” ironically talks about how customers go online to complain about poor customer service, and how their products can help you keyword search those web sites. And their website tag line: “As a speech company, we put a premium on listening – get in touch with Nuance today!” Uh, no you don’t.

I finally got ahold of someone in the Enterprise sales Thursday the 9th after a similar hold experience and at least 30 minutes on hold that time. Claudia told me they needed to know my budget, which I didn’t actually know. She said they wouldn’t call us back if we didn’t have at least 5-10K. I told her sure, then we’d have that if that’s what we needed. She took my name, phone number, and email, as well as Mark’s. Now, another week later, no response.

I’ve been hung up on 3 times this morning. I tried the customer service department again, and now I’m on hold while they try to contact the Enterprise sales people for me. The hold music has switched back from the stuff that customer service plays to that of the Enterprise line, so I’m suspecting customer service is not getting back to me, either. (One hour later, no response). By the way, Nuance, this does not bode well for our confidence in your technical support or customer service in general.

So, I’m putting it out to the web: if anyone has an audio mining software solution to search several camera audio feeds for keywords (basically something that competes with Nuance’s Dragon Audio Mining SDK), please contact us through this blog. Let’s talk. It’s more than your competition is willing to do.

[Specifically, we need something speaker-independent and that processes files automatically].

What can I do to meet your needs, Katie?

Posted on by | Leave a reply

I’m back to the sales calls, this time for Video Management Systems, the back-end software that will coordinate all our cameras. This field seems more competitive than that of eye tracking, or maybe there is just more demand, as VMS is what runs your basic surveillance system you find anywhere from the convenience store to the casino. So people are scrambling for our business.

However, whenever we try to describe what we’re doing and what our needs are, we run into some problems. You want to record audio? Well, that’s illegal in surveillance systems (it’s ok for research as long as you get consent), so it’s not something we deal a lot with. Don’t mount your camera near a heating or cooling vent or it will drown out the video. The microphones on the cameras are poor, and by the way, it doesn’t sync correctly with the video – “it’s like watching a bad Godzilla movie,” said the engineer we spoke with this morning. You want to add criteria to flag video and grab certain pieces? Well, you can’t access the video stream because if you do, then it’s not forensically admissable and can’t be used in court (Ok, we just need an exported copy, we’re not going to prosecute anyone even if they chew gum in the Visitor Center). You want to record high-resolution images? Well, you can either buy a huge amount of storage or a huge amount of processing capability. Minor obstacles, really, but a lot of decision points, even more than eye trackers. Again, though, it’s a learning experience in itself, so hopefully we’re generating some data that will save someone else some time in the future.

The pricing and purchasing is a bit strange, too. The companies seem to all have “sales” teams, but many can’t actually sell anything more than the software, some don’t even sell their software directly. Instead, we have to deal then with retailers and sometimes “integrators” that can sell us hardware, too, or at least specify requirements for us. Then there’s the matter of cameras – we haven’t decided on those, either, and it’s becoming clear that we’ll have several different types of cameras. Juggling all these decisions at once is quite a trick, literally.

At least it’s a moderately amusing process; many of the sales folks are here or were visiting in the Northwest recently, and we’ve commiserated over the last week about all the rain/snow/ice that ground the area to a halt from Seattle to Eugene.