Research « The Free-Choice Learning Laboratory at HMSC

Both Laura and I defend next week, which is why the blog has been a little quiet of late. So, hopefully, it’s the end of our dissertations, and the beginning (or really, continuations) of careers working to create fun and engaging science learning opportunities for all. We both came into the program with a lot of years of actually doing outreach, with a little bit of experience in designing programs and even less in evaluating them. Now we’re set to leave with a great set of tools to maximize these programs and hopefully share the ideas we’ve learned with the broader field as we go.

So that’s set us to thinking about where we go from here. Now I have to build a broader research project that maybe builds off of the dissertation, but the dissertation was so self-contained, and relatively concrete in a way, that the idea of being able to do multiple things again is a bit daunting. I’m almost not sure where to begin! I will have some structure, of course, provided by the grant funding I get, and the partnerships I join. However, it’s important to think about what I want to achieve before I worry about the tools with which to do it – as always, start with the outcomes and work backwards.

It’s fortunate, then, that the lab group has started to discuss our broader research interests with the hopes of finding where they intersect in order to guide future discussions. We’ve been using prezi, creating frames for each sort of focus, then intending to “code” these frames by grouping those with similar topics and ideas. For example, one of my interests at this point is everyday scientist adults keeping current with professional science research developments, for purposes of using that information in their own personal and societal decisions, or simply for keeping tabs on how tax dollars are put to work, or for any other purpose they so desire. So, I’m interested in the hows, whens, and whys of everyday scientists accessing professional science information. This means I overlap with others in the groups working with museum exhibits, but also with people interested in public dialogue events, and in general, the affordances and constraints around learning in these ways.

As the leader of the group, Shawn has mentioned that this has been an exercise he’s used to think about his broader research goals as well, simply writing down his areas of focus, looking back at what he’s done over the past few years, and looking forward to where he wants to go. It also helps him to see what’s matched with his previous plans, and how circumstances or opportunities have changed those plans. I’m grateful to have this fortuitously-timed example of long-term goal setting and building a broader agenda, especially in such a small field where it’s likely that this is the largest group of collaborators in one place that I’ll have for a while. Hopefully, though, I’ll have my own graduate students before too long and maybe even other colleagues who focus on outside-of-school learning as well.

What sorts of tools do you use for figuring out long-term, broad, and somewhat abstract research goals?

In light of the recent posts discussing Positivism vs Interpretivism, Grounded theory approach, and the challenge of thinking about epistemology and ontology, I decided to use this post to continue the debate and share a few things I have been thinking about and doing, that I hope will help me making sense of the paradigmatic views and theoretical approaches that may eventually be a part of my research.

Research design has been a challenging task nonetheless very meaningful process to me, because I am having the chance to dig deep inside into who I am and the personal values, beliefs and goals I carry with me. To start such reflection I referred back to writing exercises, a piece that I remember was topic of the first lab meetings I participated as a member of the group, and that inspired me to find ways to apply different kinds if exercises to research design. As a result of that and of the ongoing advanced qualitative class I am taking right now, my computer file folder entitled “Memos” is growing very quickly as I go through the process of writing my proposal and thinking about my research design.

I am using many forms of memos. I got myself a research “journal” that I am using to register the “brilliant” ideas I come across one way or another during this process, not only ideas for research goals/ methods/ questions, etc… but also epiphanies on concepts, theories and how I am making sense of them as they apply to my research. I am carrying it with me everywhere I go because, believe me, ideas pop up unexpectedly in very strange situations. The goal is not to loose track of my thought process as it evolves into a conceptual framework for my research. Saying it bluntly, I want to be able to say clearly why I choose the approach that I choose for my design and how I justify it.

To start this search for my own clarification about where in the world of qualitative research I sit in, which I assumed would inform my methodological choices, I wrote my first memo as a class exercise – a “Researcher Identity Memo”. It may sound very “elementary” to some of you, but I saw this exercise as opening the doors of my own path through understanding why I seat where I seat right now, how I came to be here, and where I can potentially go. The memo was a reflective exercise about past experiences in life, upbringing, values and beliefs that I may see connected to the topic of research I choose to investigate, how would I predict that as facilitating or imposing challenge to my work as a researcher. This turned out to be 6 pages document that brought out 3 personas in me that equally influenced my decisions. The educator, the scientist, and a concerned citizen of this world. The synergies between the values, beliefs, experiences, goals and interests of each got me to decided on my research topic (family “affordances” to learning at the touch-tank exhibit at HMSC).

This actually made me rethink my research goals to identify personal, practical and intellectual interests as they combine to answer the “so what?” of my research idea. In fact, “the evolution of my research questions” is another ongoing memo I am working at as my questions emerge, evolve, change, etc. I also have a mini notebook on a key chain attached to my wallet for when those revealing moments happen as I have dialogues with other professional like yourselves or want to write a quick reference to look at later. I think the practice of writing these memos is helping me untangled bits of theoretical debates that I am slowly making sense of , and that are helping me se where I sit.

Now if you are not too fan of writing, if you avoid writing exercises like the plague, Laura suggests to use alternatives ways of registering this moments. She told me she used her phone to record a voice memo the other days. How you do it is not the key issue, but I think it is important that you find a way that works for you that you can register the evolution of your thought process. Going through a few conversations with Shawn during our weekly meetings, he articulated an approach he thinks I seating on right now for my research. he bursted out these big words together that I am still trying to work trough but that emerged smoothly and almost instantly out of his mind. He called it “Neo-Kantian Post-positivist and Probabilistic Theory of Truth”. I hope he wasn’t tricking me :). Here is the way I see where I stand right now in my less eloquent philosophical terms:

1. Departing from axiological views, I am interested in explanations and descriptions of real meaningful events, why and how questions.

2. Therefore, I am moving from “data to theory”, through inductive questioning

3. As for what is the nature of reality? (ontology), I think I compromise in between objectivity and subjectivity, is there a possible inter-objectivity or inter-subjectivity?

4. As for what counts as reality? (Epistemology), I tend to associate with Social-Cosntructivism.

So, I using the following schema as a wall decoration in my research room:

Epistemology – Social-Constructivism; Theoretical perspective/ Approach – Interpretivism; Suited Methodology: Grounded Theory.

However, I see myself as open to new topics, ideas. So I am adopting a paradigm but it does not necessarily mean that I will completely oppose combining aspects of other paradigms. I read in a piece of literature once that “sometimes we need a little constructivism, and sometimes we need a little realism”. While I oppose to think radically about it, I do think that it is important to use existing theories critically, and if you are to be critical you are open to testing (hermeneutics). Here is were I seat in conflict between objectivity and subjectivity, qualitative and quantitative values, and that is why I intend to use mixed methods

I don’t know if this links perfectly to the definition of the approach Shawm saw me taking, But boy, I am happy to be going through this discovery process right now, and memos are really helping me along the way.

Susan

When you have a new idea in a field so steeped in tradition as science or education, as a newcomer, how can you encourage discussion, at the very least, while still presenting yourself as a professional member of your new field? This was at the heart of some discussion that came up this weekend after Shawn and I presented his “Better Presentations” workshop. The HMSC graduate student organization, HsO, was hosting the annual exchange with the University of Oregon’s Oregon Institute of Marine Biology grad students, who work at the UO satellite campus in Charleston, Oregon, a ways south on the coast from Newport.

The heart of Shawn’s presentation is built around learning research that suggests better ways to build your visuals to accompany your professional presentation. For most of the audience, that was slides or posters for scientific research talks at conferences, as part of proposal defenses, or just with one’s own research group. Shawn suggests ways to break out of what has become a pretty standard default: slides crowded with bullet points, at-best illegible and at-worst incomprehensible figures, and in general, too much content crammed onto single slides and into the overall presentation.

The students were eager to hear about the research foundations of his suggestions, but then raised a concern: how far could they go in pushing the envelope without jeopardizing their entry into the field? That is, if they used a Prezi instead of a PowerPoint, would they be dismissed as using a stunt and their research work overlooked, perhaps in front of influential members of their discipline? Or, if they don’t put every step of their methodology on their poster and a potential employer comes by when they aren’t there, how will that employer know how innovative their work is?

Personally, my reaction was to think: do you want to work with these people if that’s their stance? However, I’m in the enviable position of having seen my results work – I have a job offer that really values the sort of maverick thinking (at least to some traditional science educators) that our free-choice/informal approach offers. In retrospect, that’s how I view the lack of response I got from numerous other places I applied to – I wouldn’t have wanted to work with them anyway if they didn’t value what I could bring to the table. I might have thought quite differently if I were still searching for a position at this point.

For the grad student, especially, it struck me that it’s a tough row to hoe. On the one hand, you’re new to the field, eager, and probably brimming with new ideas. On the other, you have to carefully fit those ideas into the traditional structure in order to secure funding and professional advancement. However, how do you compromise without compromising too far and losing that part of you which, as a researcher, tells you to look at the research for guidance?

It occurred to me that I will have to deal with this as I go into my new position which relies on grant funding after the first year. I am thinking about what my research agenda will be, ideally, and how I may or may not have to bend that based on what funding is available. One of my main sources of funding will likely be through helping scientists do their broader impacts and outreach projects, and building my research into those. How able I am to pick and choose projects to fit my agenda as well as theirs remains to be seen, but this conversation brought me around to thinking about that reality.

As Shawn emphasized in the beginning of the talk, the best outreach (and honestly, probably the best project in any discipline, be it science, or business, or government assistance) is designed with the goals and outcomes in mind first, then picking the tools and manner of achieving those goals only afterwards. We sometimes lament the amazing number of very traditional outreach programs that center around a classroom visit, for example, and wonder if we can ever convince the scientists we partner with that there are new, research-based ways of doing things (see Laura’s post on the problems some of our potential partners have with our ways of doing research). I will be fortunate, indeed, if I find partners for funding that believe the same, or at least are willing to listen to what may be a new idea, at least about outreach.

I have been coding my qualitative interview data all in one big fell swoop, trying to get everything done for the graduation deadline. It feels almost like a class project that I’ve put off, as usual, longer than I should have. In having a conversation with another grad student, about timelines, and how I’ve been sitting on this data since oh, November or so (at least a good chunk of it), we speculated about why we don’t tackle it in smaller chunks. One reason for me, I’m sure, is just general fear of failure or whatever drives my general procrastinating and perfectionist tendencies (remember, the best dissertation is a DONE dissertation – we’re not here to save the world with this one project).

However, another reason occurs to me as well; I collected all the data myself and I wonder if I was too close to it in the process of collecting it? I certainly had to prioritize finishing collecting it, considering the struggles I had to get subjects to participate, and delays with IRB, etc. But I wonder if it’s actually been better to leave it all for a while and come back to it. I guess if I had really done the interview coding before the eye-tracking, I might have shaped the eye-tracking interviews a bit differently, but I think the main adjustments I made based on the interviews were sufficient without coding (i.e. I recognized how much the experts were just seeing that the images were all the same and I couldn’t come up with difficult enough tasks for them, really). The other reason to have coded the interviews first would have been to separate my interviewees into high- and low-performing, if the data proved to be that way, so that I could invite sub-groups for the eye-tracking. But I ended up, again due to recruitment issues, just getting whoever I could from my interview population to come back. And now, I’m not really sure there’s any high- or low-performers among the novices anyway – they each seem to have their strengths and weaknesses at this task.

Other fun with coding: I have a mix of basically closed-ended questions that I am scoring with a rubric for correctness, and then open-ended “how do you know” semi-clinical interview questions. Since I eventually repeated some of these questions for the various versions of the scaffolded images, my subjects started to conflate their answers and parsing these things apart is truly a pleasure (NOT). And, I’m up to some 120 codes, and keeping those all in mind as I go is just nuts. Of course, I have just done the first pass, and as I created codes as I went through, I have to turn around and re-code for those particular ones on the ones I coded before I created them, but I still am stressing as to whether I’m finding everything in every transcript, especially the sort of obscure codes. I have one that I’ve dubbed “Santa” because two of my subjects referred to knowing the poles of Earth are cold because they learned that Santa lives at the North Pole where it’s cold. So I’m now wondering if there were any other evidences of non-science reasoning that I missed. I don’t think this is a huge problem; I am fairly confident my coding is thorough, but I’m also at that stage of crisis where I’m not sure any of this is good enough as I draw closer to my defense!

Other fun facts: I also find myself agonizing over what to call codes, when the description is more important. And it’s also a very humbling look at how badly I (feel like I) conducted the interviews. For one thing, I asked all the wrong questions, as it turns out – what I expected people would struggle with, they didn’t really, and I didn’t have good questions ready to probe for what they did struggle with. Sigh. I guess that’s for the next experiment.

The good stuff: I do have a lot of good data about people’s expectations of the images and the topics, especially when there are misunderstandings. This will be important as we design new products for outreach, both the images themselves and the supporting info that must go alongside. I also sorta thought I knew a lot about this data going into the coding, but number of new codes with each subject is surprising, and gratifying that maybe I did get some information out of this task after all. Finally, I’m learning that this is an exercise in throwing stuff out, too – I was overly ambitious in my proposal about all the questions I could answer, and I collected a lot more data than I can use at the moment. So, as is a typical part of the research process, I have to choose what fits the story I need to tell to get the dissertation (or paper, or presentation) done for the moment, and leave the rest aside for now. That’s what all those papers post-dissertation are for, I guess!

What are your adventures with/fears about coding or data analysis? (besides putting it off to the last minute, which I don’t recommend).

While we don’t yet have the formal guest researcher program up and running, we did have a visit from our collaborator Jarrett Geenan this week. He’s working with Sigrid Norris on multimodal discourse analysis, and he was in the U.S. for an applied linguistics conference, so he “stopped by” the Pacific Northwest on his way back from Dallas to New Zealand. Turns out his undergraduate and graduate work so far in English and linguistics is remarkably similar to Shawn’s. Several of the grad students working with Shawn managed to have lunch with him last week, and talk about our different research projects, and life as a grad student in the States vs. Canada (where he’s from), England (Laura’s homeland), and New Zealand.

We also had a chance to chat about the video cameras. He’s still been having difficulty downloading anything useful, as things just come in fits and starts. We’re not sure how the best way to go about diagnosing the issues will be (barring a trip for one of us to be there in person), but maybe we can get the Milestone folks on a screenshare or something. In the meantime, it led us to a discussion of what might be a larger issue, that of just collecting data all the time and overtaxing the system unnecessarily. It came up with the school groups – is it really that important to just have the cameras on constantly to get a proper, useful longitudinal record? We’re starting to think no, of course, and the problems Jarrett is having makes it more likely that we will think about just turning the cameras on when the VC is open using a scheduling function.

The other advantage is that this will give us like 16-18 hours a day to actually process the video data, too, if we can parse it so that the automated analysis that needs to be done to allow the customization of exhibits can be done in real-time. That would leave anything else, such as group association, speech analysis, and the other higher-order stuff for the overnight processing. We’ll have to work with our programmers to see about that.

In other news, it’s looking highly likely that I’ll be working on the system doing my own research when I graduate later this spring, so hopefully I’ll be able to provide that insider perspective having worked on it (extensively!) in person at Hatfield and then going away to finish up the research at my (new) home institution. That and Jarrett’s visit in person may be the kick-start we need to really get this into shape for new short-term visiting scholars.

Last month I wrote about Literacy in the 21^st Century and the wonderful new project evaluation I’m working on, Project SEAL. I first want to share a blog post that the Model Classroom team wrote about their time with the Project SEAL teachers during the professional development in February. http://www.modelclassroom.org/blog/2013/03/projectsealoregonpd-intro.html. It has a wonderful synopsis of the two days as well as some teacher reflections.

Since the February professional development, I have turned my attention to the family literacy nights. I have never attended a family literacy night. They were not part of my K-12 experience and I have never heard of or seen them as a researcher/evaluator. The Project SEAL team told me that literacy nights can differ greatly and they did not have standards for the schools to follow for these events. This presented some troubles with me as an evaluator. How can you standardize an evaluation tool for something that looks different each time?

After having some conversations with the Project SEAL team, we decided on a short and sweet survey. Something parents would be willing to fill out throughout the night and something that would focus on literacy, ocean science resource use, as well as structure of the event. We hope that these literacy nights 1) lead to families checking out ocean-related books (purchased for the libraries through the grant), 2) give parents an opportunity to see technology that is being incorporated into literacy (the grant also bought a classroom set of iPad mini’s for each school), and 3) give teachers and students time to present on learning experiences they’ve had with the iPads and new reading material available in the library. Here are the questions on the Family Literacy Night survey.

1) What was your (or your child’s) favorite part of this Family Literacy Night?

2) What went well during this Family Literacy Night?

3) What suggestions for improvement do you have for future Family Literacy Nights?

4)What did you hope to take away from tonight’s Family Literacy Night? (check all that apply)

More activities and games to do at home

Information on what is being done in my child’s classroom

Information on assessment in reading and writing

Information about how children learn to read and write

Information on how to work with the school and my child’s teacher

New resources available in the library

Ways to use technology with my child at home

How my child’s class has been using library resources

5)You or your child have checked out ocean science resources to read together at home.

6) Your child presented or talked about a class project at this Family Literacy Night.

7) You learned what you wanted to learn tonight. Agree / Neutral / Disagree

8)Tonight I gained new information about ocean science resources available to my child through his/her school library. Agree / Neutral / Disagree

Hopefully the data can be useful in proving the effectiveness of this project but also give the schools some ideas for future family literacy nights.

The Free-Choice Learning Laboratory at HMSC

Informal science education research

Category Archives: Research

The end (I hope) and the beginning

Memos Memos Everywhere

Drawing the line: Pushing the field forward or being dragged along with it

Monkey See, Monkey Don’t

Visiting Scholars

Evaluating Family Literacy Nights

Contact Info