Recently, I came across a blog post by Daniel Green, 
who is the head of strategic media partnerships at the Bill and Melinda Gates Foundation. He coauthored this post with Mayur Patel, 
vice president of strategy and assessment at the Knight Foundation. I mention this because those two foundations have contributed $3.25 million in seed funding “…to advance a better understanding of audience engagement and media impact…”. They are undertaking an ambitious project to develop a rubric (of sorts) to determine “…how media influences the ways people think and act, and contributes to broader societal changes…”. Although it doesn’t specifically say, I include social media in the broad use of “media”. The blog post talks about broader agenda–that of informed and engaged communities. These foundations believe that an informed and engaged communities will strengthen “… democracy and civil society to helping address some of the world’s most challenging social problems.”
Or in other words, what difference is being made, which is something I wonder about all the time. (I’m an evaluator, after all, and I want to know what difference is made.)
Although there are strong media forces out there (NYTimes, NPR, BBC, the Guardian, among others), I wonder about the strength and effect of social media (FB, Twitter, LinkedIn, blogs, among others). Anecdotally, I can tell you that social media is everywhere and IS changing the way people think and act. I watch my now 17 y/o who uses the IM feature on her social media to communicate with her friends, set up study dates, find out homework assignments, not the phone like I did. I watch my now 20 y/o multitask–talk to me on Skype and read and respond to her FB entry. She uses IM as much as her sister. I know that social media was instrumental in the Arab spring. I know that major institutions have social media connections (FB, Twitter, LinkedIn, etc.). Social media is everywhere. And we have no good way to determine if it is making a difference and what that difference is.
For something so ubiquitous (social media), why is there no way to evaluate social media other than through the use of analytics? I’ve been asking that question since I first posted my query “Is this blog making a difference?” back in March 2012. Since I’ve been posting since December 2009, that gave me over 2 years from which to gather data. That is a luxury when it comes to programming, especially when many programs often are a few hours in duration and an evaluation is expected.
I hope that this project provides useful information for those of us who have come kicking and screaming to social media and have seen the light. Even though they are talking about the world of media, I’m hoping that they can come up with measures that address the social aspect of media. The technology provided IS useful; the question is what difference is it making?
Hi Molly,
I’m curious, what kind of difference are you trying to make?
Analytics are everywhere because they are easy to obtain quantitative datasets. Analytics are also meaningless without context.
For example, 100 unique pageviews could be great if your intended audience is only 100 strong. Now if you’re getting 10,000 hits it should signal you’re hitting at a different audience (or your intended audience is bigger than you expected).
Evaluating social media is like evaluating anything else. There are a lot of possible measures/data. First, figure out what you are trying to accomplish, then determine what data sources/measures are appropriate.
From your about page, “Although I’d hoped that this blog would be a place for conversation–ordinary, everyday conversation–you know, making comments and posing questions, it seems to be more a forum of me telling what I think at a given moment.”
Has your blog been a platform for conversations? (Measure: analyze conversations/comments/questions triggered by blog posts)
There are sites with tons of page hits that get very few comments. There are also sites with tons of comments and very few hits. If this is your goal, measure it.
Has it offered you a forum for telling what you think at a given moment?
(Measure: interview yourself?)
Do you have any audience goals? Are you trying to reach specific groups? I’m guessing evaluators, have you been successful? (Measure: evaluators commenting/asking questions due to blog)
Chris
Thanks, Chris, for your very thoughtful comments. I don’t know how many folks are evaluators (besides you) as most of the comments I get are marketing comments, directly or indirectly promoting a product. Then there are those in an alphabet I cannot read and then there are those which say they like my post, writing style, or content (typically those are posts which are generated by SEO (I’ve entered “is this blog making a difference(2) and the SE returned it first). I don’t know how the first post was searched because when I entered that phrase, the page didn’t show; when I added the question mark, one of my posts was on the second page.
So going backwards:
1. I don’t know if the folks commenting are evaluators.
2. I have too many cognitive biases to interview myself.
3. My blog is getting at least 10 comments a day (see above for types–most of which I do not answer–maybe if I did, I’d get more comments? Hmmm–do I really want to approve comments that are blatantly marketing?). I haven’t seen my analytic numbers, so I don’t know how many hits the page is getting.
4. This is one of the handful of conversations I’ve had over the course of the last three years (I started blogging in December 2009, almost weekly). I would have thought I would have generated some conversation when my posts are more political. Perhaps, comments all need a response…
5. This blog was initiated as an evaluation capacity building blog–I write for folks who know enough to say that their program, Or fill in the blank, needs to determine if it was effective (i.e., has merit, worth, value). So my intended audience is/was nascent evaluators; folks new to the field; folks who need more detail than they currently know. I take that parameter to mean, “I want to know if the information I’ve provided has been useful to you in the work you do/you’ve done.”
Perhaps I don’t have the right approach to determining this outcome. I don’t know. I just know that I feel like I’m writing to a virtual audience and don’t know if anyone is reading.
Thanks, again, Chris for your thoughtful comments. Something to think about.
go peacefully,
Molly.
Thanks Molly,
You have a lot of comment spam (like email spam) definitely don’t respond or accept. I don’t know who does your Wordrpess programming but I would suggest adding a plugin (either askimet or spam free wordpress). This should vastly limit the spam you receive.
From my experience, sparking conversation and pulling in traffic require two things, (1) great content styled for the web and (2) spreading the word about your great content.
Evaluating web content also has a lot to do with the structure of that content. Some structures are much easier to evaluate than others.
I’d suggest trying something new and direct. #5 in your list gives me an idea for a collaboration, any interest? The result would be a piece of content that is easier to evaluate. I have a few things coming up, but if you’re interested I’ll get in touch via email within the next couple of weeks.