research « The Free-Choice Learning Laboratory at HMSC

Our climate change “exhibit” is rapidly losing its primacy as an exhibit on which we do research to instead becoming a research platform that we set up as an exhibit. The original plan was to design an exhibit on a multitouch table around climate change and research, among other things, how users interact and what stories they choose to tell as related to their “6 Americas” identity about climate change.

After Mark attended the ASTC conference, in talking with Ideum folks and others, we’ve decided what we really need to build is a research platform on the table, with exhibit content just as the vehicle for doing that research. That means instead of designing content and asking research questions about it, we’re taking the approach of proposing the research questions, then finding content to put on that allows us to investigate those questions. The good news is that a lot of content already exists.

So, with that in mind, we’re taking the tack now of identifying the research questions we’re interested in in order to build the appropriate tools for answering those questions. For example,

-How do people respond to the table, and what kinds of tools do we need to build so that they will respond, especially by creating their own narratives about the content?

-How can we extend the museum’s reach beyond the building itself, for example, by integrating the multitouch exhibit and handheld tools? What is the shelf life of interactions in the museum?

-What are the differences between the ways groups and individuals use the table, or the differences between the horizontal interactions of the table-based exhibit vs. the more traditional “vertical” interactions provided by other exhibits (did you play Ms. Pac Man differently when it was in the table version vs. the stand-up kiosk?)

-How can we help facilitate visualization understanding through simulations on the table where visitors can build comparisons and manipulate factors in the data to create their own images and animations?

What other questions with the multitouch table should we build research tools to answer?

Writing your dissertation seems like the perfect time to learn new software, no? As Laura mentioned, she’s starting to use NVivo for her analysis, and I’m doing the same. It’s a new program for our lab, but already it looks very powerful, combining multiple types of data within the same project. For me, that’s audio, video, and transcripts of course, but I’m also finding that I will be able to link the imagery that I used probably to particular parts of the transcript. That means that I will likely be able to connect those easily in the actual dissertation write up. For me, that could prove incredibly useful as I have so many images that are virtually the same, yet subtlely different, what with the topic and level of scaffolding varying just slightly. I don’t think describing the “levels” of scaffolding in words will be quite the same. It may mean a lot of color images for my dissertation printing, though. Hm, another thing to figure out!

I’m also diving into using the new eyetracking tools, which are also powerful for that analysis, but still tricky in terms of managing licenses across computers when I’m trying to collect data in one place and analyze it in another. We’re certainly epitomizing free-choice learning in that sense, learning in an on-demand fashion to use tools that we want to learn about in order to accomplish specific tasks. One could just wish we had had real data to use these tools with before (or money to purchase them – NVivo and StudioCode, another powerful coding tool for on-the-fly video coding, are not cheap). Between that and the IRB process, I’m realizing this dissertation process is even more broadly about all the associated stuff that comes with doing research (not to mention budgeting, scheduling, grant proposing …) than it is about even the final project and particular findings themselves. I’m sure someone told me this in the beginning, but it’s one of those you don’t believe it until you see it sorts of things.

What “else” have you learned through your research process?

Happy new year everyone!

After all the fun and frivolities of the holiday season, I am left with not only the feeling that I probably shouldn’t have munched all those cookies and candies, but also the grave realization that crunch time for my dissertation has commenced. I’d like to have it completed by Spring and, just like Katie, I’ve hit the analysis phase of my research and am desperately trying not to fall into the pit of never-ending data. All those current and former graduate students out there, I’m sure you can relate to this – all those wonderful hours, weeks and months I have to look forward to of frantically trying to make sense of the vast pool of data I have spent the last year planning for and collecting.

Source: scienceblogs.com via Free-Choice on Pinterest

But fear not! ’tis qualitative data sir! And seeing as I have really enjoyed working with my participants and collecting data so far, I am going to attempt to enjoy discovering the outcomes of all my hard work. To me, the beauty of working with qualitative data is developing the pictures of the answers to the questions that initiated the research in the first place. It’s a jigsaw puzzle with only knowing a rough idea of what the image might look like at the end – you slowly keep adding the pieces until that image comes clear. I’m looking forward to seeing that image.

So what do I have to analyze? Well, namely ~20 interviews with docents, ~75 docent observations, ~100 visitor surveys and 2 focus groups (which will hopefully take place in the next couple of weeks). I will be using the research analysis tool, Nvivo, which will aid me in cross-analyzing the different forms of data using a thematic coding approach – analyzing for reoccuring themes within each data set. What I’m particularly psyched about is getting into the video analysis of the participant observations, whereby I’m finally going to get the chance to unpack some of that docent practice I’ve been harping on about for the last two years. Here, I’ll be taking a little multimodal discourse analysis and a little activity theory to break down docent-visitor interaction and interpretative strategies observed.

Right now, the enthusiasm is high! Let’s see how long I can keep it up 🙂 It’s Kilimanjaro, but there’s no turning back now.

And now it comes to this: thesis data analysis. I am doing both qualitative analysis of the interviews and quantitative analysis for the eye-tracking, mostly. However, I will also quantify some of the interview coding and “qualify” the eye-tracking data, mainly while I analyze the paths and orders in which people view the images.

So now the questions become, what exactly am I looking for, and how do I find evidence of it? I have some hypotheses, but they are pretty general at this point. I know that I’m looking for differences between the experts and the non-experts, and among the levels of scaffolding for the non-experts in particular. For the interviews, that means I expect experts will 1) have more correct answers than the non-experts, 2) have different answers from the non-experts about how they know the answers they give, 3) be able to answer all my questions about the images, and 4) have basically similar meaning-making across all levels of scaffolding. This means I have a general idea of where to start coding, but I imagine my code book will change significantly as I go.

With the eye-tracking data, I’ll also be trying to build the model as I go, especially as this analysis is new to our lab. With the help of a former graduate student in the Statistics department, I’ll be starting at the most general differences, again whether the number of fixations (as defined by a minimum dwell time in a maximum diameter area) differ significantly: 1) between experts and non-experts overall with all topics included and all images, 2) between supposedly-maximally-different unscaffolded vs. fully-scaffolded images but with both populations included, and 3) experts looking at unscaffolded vs. non-experts looking at fully-scaffolded images. At this point, I think that there should be significant differences in cases 1 and 2, but hope that, if significant, at least the value of the difference should be smaller in 3, indicating that the non-experts are indeed moving closer to the patterns of experts when given scaffolding. However, this may not reveal itself in the eye-tracking as the populations could make similar meaning as reflected in the interviews but not have the same patterns of eye-movements; that is, it’s possible that the non-experts might be less efficient than experts but still eventually arrive at a better answer with scaffolding than without.

As for the parameters of the eye-tracking, the standard minimum dwell time for a fixation included in our software is 80 ms, and the maximum diameter is 100 pixels, but again, we have no standard for this in the lab so we’ll play around with this and see if results hold up over smaller dwell times or at least smaller diameters, or if they appear. My images are only 800×600 pixels, so a minimal diameter of 1/6th to 1/8th of the image seems rather large. Some of this will be mitigated by the use of areas of interest drawn in the image, where the distance between areas could dictate a smaller minimum diameter, but at this point, all of this remains to be seen and to some extent, the analysis will be very exploratory.

That’s the plan at the moment; what are your thoughts, questions, and/or suggestions?

Or at least across the globe, for now. One of the major goals of this project is building a platform that is mobile, both around the science center and beyond. So as I travel this holiday season, I’ll be testing some of these tools on the road, as we prepare for visiting scholars. We want the scholars to be able to come to work for about a month and set the system up as they like for capturing the interactions that provide the data they’re interested in. Then we want them to have the ability to log in to the system from their home institutions, continuing to collect and analyze data from home. The first step in testing that lies with those of us who are living in Corvallis and commuting to the center in Newport only a couple times a week.

To that end, we’re starting with a couple more PC laptops, one for the eye-tracker analysis software, and one more devoted to the higher-processing needs of the surveillance system. The video analysis from afar is mostly a matter of getting the servers set up on our end, as the client software is free to install on an unlimited number of machines. But, as I described in earlier posts (here and here), we’ve been re-arranging cameras, installing more servers (we’re now up to one master and two slaves, with the one master dedicated to serving the clients, and each slave handling about half the cameras), and trying to test out the data-grabbing abilities from afar. Our partner in New Zealand had us extend the data recording time after the motion sensors decide there’s nothing going on in order to try and fix frame drop problems during the export. We’re also installing a honking lot more ethernet capability in the next week or so to hopefully handle our bandwidth better. I’ll be testing the video export on the road myself this week.

Then there’s the eye-tracker. It’s a different case, as it has proprietary data analysis software that has a per-user license. We have two, so that I can analyze my thesis data separately from any data collection that may now take place at the center, such as what I’m testing for an upcoming conference presentation on eye-tracking in museums. It’s not really that the eye-tracker itself is heavy, but with the laptop and all the associated cords, it gets cumbersome to go back and forth all the time, and I’d rather not have the responsibility of moving that $30K equipment any more than I have to (I don’t think it’s covered under my renter’s insurance for the nights it would be stored there in between campuses). So I’ve been working on setting up the software on the other new analysis laptop. Now I’m running into license issues, though I think otherwise the actual data transfer from one system to another is ok (except my files are pretty big – 2GB of data – just enough that it’s been a manual, rather than web-based, transfer so far).

And with that, I’m off to start that “eye-tracking … across the universe” (with apologies to the writers of the original Star Trek parody).

When thinking about creating outreach for a public audience, who should the target audience be? What types of questions can you ask yourself to help determine this information? If is ok to knowingly exclude certain age groups when you are designing an outreach activity? What setting is best for my outreach setting? How many entry or exist points should my activity have? Should there be a take-away thing or just a take-away message? How long should the outreach activity run? How long will people stay once my activity is completed? What types of materials are ok to use with a public audience? For example is there anything I should avoid like peanuts? Am I allowed to touch the people doing the activity to help them put something on to complete the activity? What types of things need to be watched in between each activity to avoid spreading germs? How much information should I “give away” about the topic being presented? What type of questions should I ask the participants in regards to the activity or information around the activity? How much assumed knowledge can I assume the audience has about the topic? Where do I find this information out? What are some creditable resources for creating research based educational activities?

These are some of the questions that I was asked today during a Pre-college Program outreach meeting by another graduate student who works with me on OSU’s Bioenergy Program. Part of our output for this grant is to create and deliver outreach activities around Bioenergy. We plan on utilizing the connections among SMILE, Pre-college Programs and Hatfield Marine Science Center since there are already outreach opportunities that exist within these structures. As we were meeting, it dawned on me that someone who has not ever been asked to create an outreach activity as part of their job may see this task as overwhelming. As we worked through the questions, activities and specific audience needs of the scheduled upcoming outreach, it was both rewarding and refreshing to hear the ideas and thoughts of someone new to the field of outreach.

What are some questions you have when creating outreach? What are some suggestions about creating outreach to the general public verse middle school students verse high school students? Do you have any good resources you can share? What are your thoughts?

The Free-Choice Learning Laboratory at HMSC

Informal science education research

Tag Archives: research

Exhibit or research platform?

Thesising

The pit of never-ending data

Analyze this

Data collecting … across the universe*

Creating Outreach Activities

Contact Info