Tag Archives: data analysis

Wading through a pool of data

Posted on by | Leave a reply

Another task within my Master’s research is checked off.  I have reviewed and noted the conversations and behaviors from video footage of 25 family groups using our Ideum multi-touch table exhibit.  As I went through the footage, it was fascinating to see the similarities and differences in how groups used the exhibit, talked (or not) while using the table, and the elements of the exhibit that appeared most attractive to them.  I will be analyzing the engagement and learning strategies data along with the group interview responses that I collected post-use of the exhibit.  I am so thankful for the infrastructure that we have installed as part of the Cyber Lab.  The video recordings that include audio of the group conversation has been a great way to examine the data beyond field notes from the date of observation.  Quite a bit of data to make sense of!

This quarter I have had the opportunity to take the Socio-Cultural Dimensions of Learning course from Dr. Lynn Dierking at Oregon State University.  It has aligned perfectly with where I am at in my project.  Topics in the course have included how we learn through our interactions and observations of others, how culture influences learning, and how institutions scaling from families, to museums, up through society plays a role in the learning experience.  Family learning in museum spaces has been one topic that we have focused on, particularly how different members of a multi-generational group make meaning of exhibits and content that is personal and relevant to their experience.  An element that I have taken an interest in is the roles that the family members take, whether it is the adult or child, and who “takes charge” of the interaction with the exhibit.  For example, is it primarily the adult showing the child how to use it, or explaining what the information means?  In what ways do they make connections to what the child already knows?  These questions relate to my observations with the touch table.  I have seen evidence of the child taking charge of the interaction and showing the adult, and I’m curious to investigate what strategies the child uses to “show” the adult what to do.  I wonder if it is because this generation is often around touch surfaces with their personal electronics, that they feel comfortable taking on that role to “teach” the adult.

I have also appreciated the opportunity to interact with other students in the socio-cultural dimensions class that are located around the country, many who also work in free choice learning venues.  Several students have shared teaching and learning strategies that they use to interact and engage with their visitors.  The course has inspired me to think more about the transition from theory to practice, by applying what we are studying to improving the learner experience.  We can conduct this research, but until it is applied and shared, it seems anticlimactic (at least to me!).  I hope from my research of watching the natural behaviors of families using the multi-touch table, I can provide recommendations for ways to improve content to facilitate the behaviors they are already expressing.  The technology is a tool that is being used to share science content, so what meaning are the users making of that information?  Research has been done on the overall usability of large scale touch surfaces in public spaces, but how does that connect to learning in a space where individuals have choice and control over their experience with the technology?  It is not so much as HOW they use it, but what do they gain as a result of the interaction on a personal or social level?  The beauty of research – you look to answer some questions and come up with more!

Reflections as I start “18th” Grade

Posted on by | Leave a reply

I started “18th” grade this past week, also known as the second year of my two year program at OSU.  The beginning of a new academic year is a great time to reflect and I’ve been thinking about my evolution as a graduate student and on the work we have accomplished in the Cyberlab thus far.  Since my first posts from last year, much of what I wrote about being patient in the process still rings true.  Iteration and refinement help to direct the course.  As I have made progress in my own research study, I still have to be patient as the project unfolds as some unique results may appear that I might otherwise miss.  Looking forward to where I might be next September is exciting too.  It is unknown at this time, but thinking about all of the potential opportunities…who knows!

I am proud to say that I have transitioned into the analysis phase of my Master’s research.  I have some results from my interviews of the families that used the touch table, but more will be following as I start to review the videos.  One challenge has been to develop a strategy for analyzing the video of families using the table.  This is something I have not done before.  There are some resources for analyzing video in non-school settings, so I am referencing that heavily.  One book that has been particularly helpful is Video Research in the Learning Sciences (Goldman, Pea, Barron, and Derry, 2007).  This is the most comprehensive source with theoretical and methodological guidance I have seen, especially with connections to filming observations in an informal science setting.  As family behavior and interactions in a museum setting has been studied (Falk, Dierking, Ash, to name a few), we have a better idea of the types of behaviors that take place in this environment.  I am interested in the degree to which they are occurring around the touch table.  We know parents may read content on signage aloud, point, question, recall past events…but to what extent is this happening with technology that is not commonly seen (at least scaled to a table on a daily basis)?  I’m going to approach this on a spectrum or scale of low to high levels of the presence of behaviors.  Using a rubric as a way to score the interactions, something done to assess teacher facilitation in the classroom, I believe this is a way to put a “measure” on the adult and child interactions.  From the results, we may have a better idea of what the quality of interaction with touch tables looks like in a science center, allowing us to point to specific areas to improve content that affords these behaviors on a deeper level.

This quarter I also started taking the free-choice learning series through the College of Education.  It is perfect timing as I work through my research project.  I am gaining knowledge and a better understanding of what learning is and the context to which it takes place, and how we do not learn in isolation.  Our perspectives and experiences can be shaped by those around us, one reason for my interest in family learning behaviors.  The first course is “Personal Dimensions of Learning” and I appreciate the new resources to read about motivations and identity as related to self-driven learning.  As this is an Ecampus course, there are students from around the country doing incredible science education projects both in and outside of a formal classroom setting.  I am looking forward to getting to know them better as the quarter progresses.

Next post will recount my first experience at the annual meeting of the Association of Science and Technology Centers (ASTC).  I will be tweeting from Raleigh next week – follow me @East_JennyL.

What we say through play

Posted on by | Leave a reply

Super Mario moves like a machine.

He almost never turns around unless he must. He runs rightward. He jumps rightward. He crouches and slides under bricks without slowing. He acquires coins. He kills with fire and boot-heel. Still he runs. Rightward—ever rightward.

Finally, a difficult jump briefly halts his progress. Super Mario dies. For now.

My wife puts down the controller. It’s my turn, and Luigi’s. I went from the Atari 2600 straight to the Super Nintendo in my youth. The NES, while much-loved and present in my childhood memories, was not a major factor in my early development as a gamer.

Luigi looks terrified, and far from Super. He hesitates. He backtracks. He pauses. He approaches his first Goomba anxiously, and his jump is ragged and imprecise. The original Super Mario Bros. has somewhat drifty controls compared to its successors, and it always takes me some time to re-adjust. Too long.

Death comes quickly to Luigi. My wife finishes the game a few lives later, with Mario’s triumphant campaign only infrequently punctuated by Luigi’s fitful progress and inevitable tragedies.

Non-verbal communication among players is a big part of tabletop gaming, and I’ll be looking at that as I analyze interactios around my game Deme. However, as in the anecdote above, games—electronic or otherwise—come with their own non-verbal cues and even a body language of sorts. This can be more noticeable when players aren’t able to physically observe or interact with each other.

An arrangement of chess pieces could be interpreted as aggressive or defensive. A player’s confidence and skill can show in online games through movement and action. In these cases, with in-game actions—and sometimes movement—being limited and uniform, interactions come at least partially pre-coded for the researcher.

It’s an interesting phenomenon, and you can see it just about everywhere. I’m a big fan of playing games together with friends and family in the same room, but I’ve often been amazed by how much meaningful information I’ve exchanged online with fellow players I’ve never seen, and with whom I’ve never exhanged a typed or spoken word. Feints, counter-feints, acknowledgements, threats, camaraderie, humor—humans will find ways to communicate with any tools available. In online games, these tools may be anything from complex role-playing avatars to playing cards or two-dimensional spaceships.

Would anyone else like to share an anecdote or two about nonverbal communication within games? The novel ways people find to convey a message can often be just as interesting as the message itself.

Lining up more ducks: data management

Posted on by | Leave a reply

Lining up these ducks is definitely not sexy research work, yet it’s often very important, as Laura mentioned last week. Ideally, this would have been done from the beginning. It was to some extent, but mostly with what was new coming in rather than taking stock also of what we already have.

The same will probably be true of the data that the lab collects, but since we haven’t really started collecting any new data, we can hopefully be proactive about how we integrate the old and the new. We’re building on previous work, such as video analysis of the touch tanks, where a corpus of data already exists. That means we have (perhaps) an existing data management structure, or not, but we need to find out.

I recently attended a great workshop from the OSU Library on data management plans. For this grant, we promised not only to share our experiences with this project as we go, but also to share our data to the extent possible, especially with our visiting scholars. Funders are also requiring plans for storing and managing data, and there is also a larger push for much federally-funded data to be shared in open-access databases. Even without these prods, managing data saves time in the long run by keeping information organized and easy-to-access. One of the tools they recommend is DMP Tool, which has many templates from different funders to create the plans for grant applications in the first place.

Once you’ve been awarded the grant, planning file names and creating meta-data up-front can ensure things are easy to find, easy to share, and easy to manage. That’s where we are. As we start to plan exactly what research questions we pursue first, we’ll figure exactly what types of data we’ll be collecting. As we are working with Media Macros to set up analytic programs to parse the video data, we will be able to designate how we want our file names to be constructed for example to indicate the date and time and type of data, such as video, transcript, or demographics, and source such as particular camera. Having all of this in the file name will make the data consistent and sortable. We also have to build a consistent file folder structure and locations for storing data, whether it’s dropbox or external hard drives or some other sharing platform for works in progress as well as ultimate archival.

The naming scheme goes beyond that, too, to the secondary data that we analyze, such as filenames in transcript analysis software, excel spreadsheets, and even presentations and writing. One more important type of data is the metadata, which is data about the data. For us, that could mean: a) descriptions of how and when the data is collected, part of which is captured in the filename but which can be expanded upon, b) descriptions of the data collection equipment, such as the type of camera (3301 or 1034 or other models), frame rate, and resolution, and c) descriptions of the larger events that may have been going on at the science center on any given day, such as Shark Day, Whale Watch Week, or Home School Days, if they were occurring.

Of course, with a number of people working on the project now, students rotating on and off the project, and visiting scholars joining, we have to make sure that this structure is communicated and most importantly, used! I have to admit I was guilty of this during my dissertation – having a plan at the beginning but not necessarily sticking to it. We’ll have to regularly re-visit the plan and make sure things are still working for everyone, but hopefully having a plan at the outset will prevent a lot of work down the road with a disorganized set of raw data, working data, meta data, and final products.

Monkey See, Monkey Don’t

Posted on by | Leave a reply

I have been coding my qualitative interview data all in one big fell swoop, trying to get everything done for the graduation deadline. It feels almost like a class project that I’ve put off, as usual, longer than I should have. In having a conversation with another grad student, about timelines, and how I’ve been sitting on this data since oh, November or so (at least a good chunk of it), we speculated about why we don’t tackle it in smaller chunks. One reason for me, I’m sure, is just general fear of failure or whatever drives my general procrastinating and perfectionist tendencies (remember, the best dissertation is a DONE dissertation – we’re not here to save the world with this one project).

However, another reason occurs to me as well; I collected all the data myself and I wonder if I was too close to it in the process of collecting it? I certainly had to prioritize finishing collecting it, considering the struggles I had to get subjects to participate, and delays with IRB, etc. But I wonder if it’s actually been better to leave it all for a while and come back to it. I guess if I had really done the interview coding before the eye-tracking, I might have shaped the eye-tracking interviews a bit differently, but I think the main adjustments I made based on the interviews were sufficient without coding (i.e. I recognized how much the experts were just seeing that the images were all the same and I couldn’t come up with difficult enough tasks for them, really). The other reason to have coded the interviews first would have been to separate my interviewees into high- and low-performing, if the data proved to be that way, so that I could invite sub-groups for the eye-tracking. But I ended up, again due to recruitment issues, just getting whoever I could from my interview population to come back. And now, I’m not really sure there’s any high- or low-performers among the novices anyway – they each seem to have their strengths and weaknesses at this task.

Other fun with coding: I have a mix of basically closed-ended questions that I am scoring with a rubric for correctness, and then open-ended “how do you know” semi-clinical interview questions. Since I eventually repeated some of these questions for the various versions of the scaffolded images, my subjects started to conflate their answers and parsing these things apart is truly a pleasure (NOT). And, I’m up to some 120 codes, and keeping those all in mind as I go is just nuts. Of course, I have just done the first pass, and as I created codes as I went through, I have to turn around and re-code for those particular ones on the ones I coded before I created them, but I still am stressing as to whether I’m finding everything in every transcript, especially the sort of obscure codes. I have one that I’ve dubbed “Santa” because two of my subjects referred to knowing the poles of Earth are cold because they learned that Santa lives at the North Pole where it’s cold. So I’m now wondering if there were any other evidences of non-science reasoning that I missed. I don’t think this is a huge problem; I am fairly confident my coding is thorough, but I’m also at that stage of crisis where I’m not sure any of this is good enough as I draw closer to my defense!

Other fun facts: I also find myself agonizing over what to call codes, when the description is more important. And it’s also a very humbling look at how badly I (feel like I) conducted the interviews. For one thing, I asked all the wrong questions, as it turns out – what I expected people would struggle with, they didn’t really, and I didn’t have good questions ready to probe for what they did struggle with. Sigh. I guess that’s for the next experiment.

The good stuff: I do have a lot of good data about people’s expectations of the images and the topics, especially when there are misunderstandings. This will be important as we design new products for outreach, both the images themselves and the supporting info that must go alongside. I also sorta thought I knew a lot about this data going into the coding, but number of new codes with each subject is surprising, and gratifying that maybe I did get some information out of this task after all. Finally, I’m learning that this is an exercise in throwing stuff out, too – I was overly ambitious in my proposal about all the questions I could answer, and I collected a lot more data than I can use at the moment. So, as is a typical part of the research process, I have to choose what fits the story I need to tell to get the dissertation (or paper, or presentation) done for the moment, and leave the rest aside for now. That’s what all those papers post-dissertation are for, I guess!

What are your adventures with/fears about coding or data analysis? (besides putting it off to the last minute, which I don’t recommend).

Entering the home stretch

Posted on by | 1 Reply

I have just about nailed down a defense date. That means I have about two months to wrap all this up (or warp it, as I originally typed) into a coherent, cohesive, narrative worthy of a doctoral degree. It’s amazing to me to think it might actually be done one of these days.

Of course, in research, there’s always more you can analyze about your data, so in reality, I have to make some choices about what goes in the dissertation and what has to remain for later analysis. For example, I “threw in” some plain world images into the eye-tracking as potential controls just to see how people might look at a world map without any data on it. Not that there really is such a thing; technically any image has some sort of data on it, as it is always representing something, even this one:

 

 

Here, the continents are darker grey than the ocean, so it’s a representation of the Earth’s current land and ocean distinctions.

I also included two “blue marble” images that are essentially images of Earth as if seen from space, without clouds and all in daylight simultaneously, one with the typical northern hemisphere “north-up” orientation, the other “south-up” as the world is often portrayed in Australia, for one. However, I probably don’t have time to analyze all of that right now, at least not and complete the dissertation on schedule. The best dissertation is a done dissertation, not one that is perfect, or answers every single question! If it did, what would the rest of my career be for?

So a big part of the research process is making tradeoffs between how much data to collect so that you do get enough to anticipate any problems you might incur and want to examine about your data, but not so much that you lose sight of your original, specific research questions and get mired in analysis forever. Thinking about what does and doesn’t fit in the particular framework I’ve laid out for analysis, too, is part of this. That means making smart choices about how to sufficiently answer your questions with the data you have and address major potential problems but letting go and letting some questions remain unanswered. At least for the moment. That’s a major task in front of me right now, with both my interview data and my eye-tracking data. At least I’ve finished collecting data for the dissertation. I think.

Let the countdown to defense begin …