Tag Archives: evaluation

Wordle You Say? Or, how to use word clouds to look at your data

Posted on by | 2 Replies

I have been shuffling through data from the Exploratorium’s scientist-in-residence (SIR) project and I started thinking about what data (and the kinds of ways data) can or should be shared on a blog.  For now, I am going to share a few word clouds of raw data.  These do not illustrate full sentences nor can you tell which participant said what.

Each of these word clouds was based off of a survey question that I wrote and administered.

Visitors to the exhibition space were asked, upon leaving, “What would you tell a friend this space was about?”  The word cloud below contains data from the March residency, which focused on severe storm science (with scientists from NOAA’s National Severe Storm Lab).

 

The Exploratorium Explainers were an integral part of this project.  At the end of the second year I asked all of the Explainers, the Lead Explainers, and the Explainer managers to voluntarily complete the online survey.

Here is how Explainer managers responded to “Describe the impacts of this project on the scientists.”

 

While the Explainer survey was quite long and there is a lot of rich data there, I want to focus on their thoughts about the iPad.  The iPad was incorporated into the exhibition space as a mediating tool (as specified in the grant proposal).  I asked the Explainers “Where and how do you think the iPad was incorporated throughout the project?”  Their response…

 

 

So, what can we gain from word clouds?  It is certainly one way to look at raw data.  Thoughts?

 

Making Waves: Bringing Tsunamis to Washington DC – Part 1

Posted on by | Leave a reply

OSU ran three outreach activities at the 46th annual Smithsonian Folklife Festival, and we took the chance to evaluate the Wave Lab’s Mini-Flume wave tank activity, a related but different activity to the wave tanks in the HMSC Visitor Center.

Three activities were selected by the Smithsonian Folklife committee to best represent the diversity of research conducted at OSU, as well as the University’s commitment to sustainable solutions and family education: Tech Wizards, Surimi School, and the O.H. Hinsdale Wave Lab’s Mini-Flume activity. Tech Wizards was set up in the Family Activities area of Folklife, and Surimi School and the Mini-Flume activity shared a tent in the Sustainable Solutions area.

Given the anticipated number of visitors to the festival, and my presence as the project research assistant, we decided it would be a great opportunity to see how well people thought the activity worked, what they might learn, and what they liked or didn’t – core questions in an evaluation. The activity was led by Alicia Lyman-Holt, EOT director at the O.H. Hinsdale Wave Lab, and I developed and spearheaded the evaluation. To make the activity and evaluation happen, we also brought four undergraduate volunteers from OSU and two from Howard University in D.C, plus both the OSU Alumni Association and the festival supplied volunteers on an as-needed basis. We also wanted to try out data collection using iPads and survey software we’re working with in the FCL Lab.

Due to the sheer numbers of people we thought would be there, as well as the divided attentions of everyone, we decided to go with a straightforward survey. We ended up only collecting a small number of what we anticipated due to extreme heat, personnel, and divided attention of visitors – after they spent a lot of time with the activity, they weren’t always interested in sticking around even for a short survey.

I’m currently working on data analysis. Stay tuned for more information on the evaluation, the process, and to learn how we did on the other side of the continent.

So much data, so little time.

Posted on by | 4 Replies

Ok, I guess I am following suit and forgot to post on Friday! I don’t have quite as good of an excuse as Katie. Instead of prepping for conferences I was recovering from a vacation.

I thought it might be nice to provide an update about the Exploratorium project, where NOAA scientists are embedded on the museum floor with the Explainers (Exploratorium front-line staff consisting of young adults). I have collected so much data for this project I am beginning to feel overwhelmed.

Here’s the data that I have collected:
– Formal Interviews with each of the four groups of scientists, both before and after their experience.
– Informal interviews with all of the scientists. These were done in the time walking back to the hotel or when grabbing lunch. Both great times to collect data!
– Interviews with the two Explainer managers plus a survey with open- and closed-ended questions at the end of year 2.
– Interviews with each of the lead Explainers, 8 total. Also, lead Explainers during year 2 completed a survey with open- and closed-ended questions.
– Pre- mid- and post- data for what Explainers think atmospheric sciences is and what atmospheric scientists do. This was not done during the first year topic of ocean sciences.
– I also provided an optional survey for all Explainers so they could share their thoughts and opinions about the project. This provided a reflection opportunity for the Explainers that were not lead Explainers during the project.
– Visitor surveys about their experience in the scientists’ installation. During year 2 these were collected in both paper form and using survey software on the iPad.
– Field notes during meetings and time on the museum floor. During year 2 the field notes were taken on the iPad using survey software.
– And lastly…personal daily reflections.

So the question is “now what?” This data provides opportunities for triangulation but where does one start? I’m spending my final month of summer trying to figure that out.

Hopefully my next blog post will showcase my progress and some findings.

Some last thoughts (for now?) from Diana

Posted on by | Leave a reply

Our Summer Scholars’ time is drawing to a close at the end of the month, so we’re hearing some final words, at least for the moment, from some of them, starting with Diana:

“These past weeks have been filled with things that I never thought would happen and have surprised me in the most spectacular ways.  First, I went on vacation to Vancouver, BC and Seattle, WA which was a memorable experience.  I was able to see behind the scenes tanks and animals at the Vancouver Aquarium and even got to see Leonardo Da Vinci’s works in person as well as King Tutankhamen’s burial chamber items.  This vacation was a nice break from the craziness of the visitor’s center and refreshed me for another few weeks as an education intern.  The moment I returned many volunteers and other workers at Hatfield were asking about my vacation.  Even this little thing made me feel fully welcomed into the Hatfield family.  One of the first things that occurred when I returned was that many unannounced summer camps came into the visitor’s center which is always an experience.  Yet, one of the most progressive things I did was creating new signs for me shoreline erosion tank.  This time one of my mentors Mark Farley and I created 2 different signs compared to one long sign.  One sign said “The Erosion Problem” with photos of me showing how to use the paddle to create waves and see the erosion of the sand.  The second sign said “The Erosion Solution” and gave the visitors a chance to try 1 of 3 different protective strategies for beach/shoreline erosion.  These new larger signs seem to be working well for now.  I can already see a difference in the behavior of children and families when they come to wave tank; instead of sand castle building, they actually read the sings and follow the directions.

The visitor’s center also had some crazy moments.  We had Micro A and Micro B tanks overflow into the VC overnight and leave a lake in the surrounding area an inch deep.  That lake was an interesting mess to clean, but created a wonderful learning moment.  I was able to watch the aquarists and learn how to put on new filter bags as well as rework the tanks.  I was challenged to follow the pipes and figure out where the water went such as the outflow and inflow pipes.  Other crazy moments that occurred were people trying to put their whole hand inside of anemones or trying to crawl inside of the touch tank to touch the different fish.  While all of this was going on I also got to have some spectacular moments in the VC.

These spectacular moments occurred when the Aquarists took me under their wing and showed me some impressive things.  First, I got to see a fish necropsy which was highly informative and taught me new dissections skills.  I was also taught how to kill invasive coral apitasia with lemon juice.  I was able to inject a few micrometers of lemon juice into each invasive apitasia, which kills it almost instantaneously.  The apitasia tries as a defensive mechanism to spit its own guts out, but the lemon juice is too acidic.  I also learned through this process how to siphon a tank and change out the water while balancing the acidity in the water with baking soda, thus making the seawater more neutral.  Yet, the most spectacular thing I learned with the aquarists was how to feed all the animals in the Hatfield Marine Science Center.  I learned how and what to feed each animal except the octopus in the visitor’s center, which took a long time but was completely worth it.  The amount of knowledge I learned during that time was amazing and I will not forget anytime soon.  This entire summer has been a learning experience, but definitely a fun one that I shall remember for the rest of my life.”

 

Preparing for crowd management – Tsunami Tank Exhibit

Posted on by | Leave a reply

This follows Nick’s post on “preparing for a different type of Tsunami”, when he discussed initial challenges of the tsunami tank exhibit, especially in terms of the Lego activity and resources used.  Nick pointed out some mechanical/ physical challenges already encountered during initial prototyping but nevertheless said he was confident that the exhibit will be fun, interesting, and popular among Hatfield visitors.

POPULAR without a doubt! I have done some observations and brought in some groups to test the Lego activity at the tank and already can tell you Nick, the tsunami tank will most certainly be very popular. As a consequence, challenges to the exhibit are not only related to the resources used in the activity as you pointed out and whether it works or not, but also brings up issues of crowd management, flow and accessibility to the tank area and interactions among visitors.

 

In sum, here are some main points that surfaced from my short prototype:

a) The Lego activity and concept for the tank seem to generally work, apart from a few glitches already being addressed such as computer malfunctions and the sanding of Lego blocks so that they don’t stick so strongly together causing poorly constructed structures to stay firm after a potentially strong tsunami wave.

b) There is a need for establishing some rules for building structures so that participants won’t just build a solid square block that will stand still no matter what. The rules during the prototype were that each participant gets a cup full of Legos and have to build a structure of whatever shape but that will not surpass eight blocks tall to survive a tsunami wave. However, a few malfunctions were observed. As an example, the cup idea did not work well as most groups will go to the activity table and search for parts they want to use that were not in their cups.

c) Groups, especially children will spend a long…long time at the tank, which is good and challenging at the same time since crowds accumulate around and things can get really chaotic pretty fast. Creating clearly defined stations for building structures, providing a set of steps to be followed (through a facilitator or signage) and reinforcing time management can address the issue. Although I am afraid there isn’t really a definite solution for that, and at some degree we will have to rely on the visitors themselves (especially parents) to make good judgments and facilitate the process.

d) Visitors have LOTS OF FUN, interact and participate in shared learning.  After all, isn’t that the important aspects to cultivate if we are trying to facilitate learning?

Other subsequent observations were also made when the tank was opened to the public for a day with no facilitator and all my initial speculations were confirmed that crowd management will pose a huge issue, and while some creative solutions are on the making, the exhibit will need constant prototyping through time and even after it is completely opened to the public in order to minimize the problem.  Should I even call it a problem in the very sense of the word? Maybe I should say it is a good problem to have.

The exhibit has all the potential to foster active prolonged engagement (APE) and promote meaningful interactions. Humphrey and Gutwill (2005) importantly point out that APE exhibits are empowering to visitors as they can take pleasure in “observing, playing, investigating, exploring, collaborating, searching and speculating”. That is what I just saw groups starting to do at the tsunami tank.

(Humphrey, T., Gutwill, J. P., & Exploratorium (Organization). (2005). Fostering active prolonged engagement: The art of creating APE exhibits. San Francisco: Exploratorium).

 

 

 

Wading in to visitor research

Posted on by | Leave a reply

Here’s an update from intern Julie Nance as she wades in to gathering data from the public:

“Last week I began front-end evaluation – talking to people out in the Visitor Center to get their opinions for the climate change exhibit.  I had them choose what case study they would want to learn more about, from a set of 14 pictures (species affected by global warming such as salmon, pteropods, etc).

 

I wrote down what everyone said and came up with some interesting trends, such as how the majority of women in their 20’s and 30’s as well as school age girls chose the emperor penguin over the rest.  This wasn’t a huge surprise given the options.
So the next round, I removed the penguin and turtle to force a harder choice, so many in that age group switched to the next most familiar and cute creature: the clown fish.  As my fellow intern Nick puts it, they’re only interested in “charismatic mega fauna”.
However, there were many people who chose things that were more local and meaningful to them personally.  My favorite comment I found funny was, “I chose Dungeness Crab, because I like to eat them, and I’m interested in keeping that going.”
The two most surprising comments were from gentlemen who were roughly age 60 to 70.  This demographic is kind of stereotyped as being very skeptical of climate change, and I will admit that I pegged them as probably being in that group.  One chose phytoplankton and said, “they are basic foodstuff.  They’re at the bottom  of the food chain, so that has effects all the way up.”  The other chose algae and said, “some people don’t believe [climate change] and think it’s ‘business as usual’, but I don’t think so.  Algae will probably be one of the first affected.”  Wooohoo!  You go dudes!  Thanks for changing my perception.

This week I’m preparing for the next phase of evaluation in which I will get into more detail with visitors about the exhibit itself, how it will work, and what types of resources they would want available on it.  A graphic designer, Alison, who works on projects for the VC is making a graphic for me to print and show to visitors while I talk to them about the exhibit.  This will really help them to visualize what I’m talking about.

The psychology behind talking to people and getting their opinions is staggering.  Every question, phrase, graphic, etc. I use goes through this complex interview process in my brain.  I wonder things such as, “how might people misinterpret this?”  “Will using this picture bias people’s responses?”  “Will using this phrase turn people away and change their answers?”  “If I color code, what are these colors going to make people think?”
That last one is big right now, along with positioning of things.  I am going to ask people to self-categorize into one of the groups from “Global Warming’s Six Americas,” but the graphic I was going to use from that study has different colors for each and I wonder if people will choose their favorite color, or think that one color is better or more desirable or think that I as the researcher want to lean them a certain direction based on the color or position.  If I lay out the 6 options top to bottom, it makes the top seem to be best and the bottom worst.  Colors- are warm or cool colors more acceptable and which do I appear to be favoring?  If I lay them left to right it might feel best to worst on a spectrum, or perhaps even political left wing/ right wing.  When you are doing research with human beings, whose thought patterns are so complex, you really can NOT control for every variable and you just have to do your best and realize that the results are influenced by many things.”