Is it “working”? Let’s talk about metrics for mission-driven work

How do we know if the information we’re providing is having an impact? How do we know if our efforts are worth it? If our plan is “working“?

Those questions don’t come up much if your ultimate business success can be easily quantified (dollars, users, sales, reach, etc.). But what if your goals aren’t so easily measured?

In a web analytics training earlier this year, I was the only person in the room who didn’t have a clearcut goal for my website. Everyone else had a page that all other pages were driving users to — a “thank you for purchasing” or “thank you for confirming your subscription” type page. Journalists tend to just want more of all of it — more views, more new users, more return users, more time.

And what about all the offline work we do?

I’ve been doing a lot of work recently on measuring the impact of mission-driven information. A lot of that work has been with journalists, but some of it has been targeted more generally to nonprofits. A few months ago, I brainstormed a whole host of possible metrics, and I figured I’d go ahead and share that list here.

Here are a few things to think about before diving in:

• Think through the story of your work from beginning to end. What do you hope will happen? You have to be clear on your goals before the metrics will make much sense.
• Look for correlation, not just causation. Metrics are imperfect, and cause and effect aren’t always neat and tidy.
• Focus on metrics that will help you solve problems or be more efficient. You could measure many more things that will actually be useful, and your time could be consumed in measurement that won’t actually make you better at your job.

I hope this list helps some folks think through what can be measured. I also hope it doesn’t feel overwhelming. A lot of this likely won’t help you measure the work you’re doing, so don’t consider it to be prescriptive. But if you get a few ideas, and someone else gets a few other ideas, I’ll be happy.

I’d love to hear what you find useful, what you disagree with or what you’d add. Leave a comment, tweet at me or email me. Here we go …

Reach of content

• Website reach
• Facebook reach
• Print ad reach
• Number or percentage who open email newsletters
• Number who attend event, or percentage invited who attend event
• Audience growth — to specific content areas or overall
• Audience growth or involvement from key stakeholders or influencers. (Success sometimes depends on who you reach as much as how many you reach.)
• Percent of new visitors to content
• Penetration in identified community
• Audience growth in targeted geographic area
• Growth of audience in key ways (geographic, demographic)
• Exposure to new audience through community groups or partnerships

Website design

Consider how people find your content and change what’s not working. Use unique bit.ly links when helpful to track source of visit, such as:

• Home page links
• Links in house ads on website
• Links from print or broadcast
• Links inserted into a story by a journalist
• Links from a project landing page or to a project landing page

Payoff from specific outreach, sponsorships or marketing efforts

When asking community partners, community groups, loyal users or others to share links to your content, use unique bit.ly links that can be easily tracked by putting a + on the end of the link.

• Use specific bit.ly links for specific platforms (to compare efficiency of Facebook and Twitter, for example)
• Use specific bit.ly links for specific partnerships or groups
• Use specific bit.ly links for specific marketing efforts
• Track inbound visits from websites of partners and community groups
• Track outbound visits from your site or social platforms to those partners or groups
• Ideas, sources or contributions that emerge from specific partnerships, platforms or groups

Loyalty/commitment of users

• Visitors (number or percentage) who return during a set time period
• Number of likes/follows on social (or growth of likes/follows)
• Growth of active visits (visits that involve key metrics like commenting, sharing, page depth, time spent)
• Lowering of exit rates from landing pages and key content
• Involvement of key, identified stakeholders

Consider the percentage of users or event attendees (or the percentage of web visitors) who then:

• Like/follow social platforms
• Call into broadcast show
• Subscribe or sign up for membership
• Sign up for email newsletters
• Read email newsletter

Note: A low-tech way to offer email subscriptions is to embed a google form with signup info in related content, then once a week send those email addresses (which live in an automatic spreadsheet) an email.

Conversation around content: raw size versus relative activity

Consider:

• The number of comments, posts, submissions
• The number of unique commenters, posters, submitters
• The quality or tone of comments, posts, submissions

How many people (or what percentage of people) who saw the content:

• Talk about it on your site
• Share it on social media (try http://muckrack.com/whoshared/)
• Contribute thoughts/ideas/endorsement about it on social media
• Share it with specific groups or conversations (not just their general network)
• For Facebook pictures, number or percentage who tag themselves in the picture and/or share it
• Retweets, replies or hashtag use on key action-oriented tweets or topics (For social searching resources, see bit.ly/joysocialsearch.)

Invitations to specific conversation

How many people (or what percentage of people who were exposed to the invitation):

• Respond to a specific request from the newsroom on website or social platform
• Participate in a live chat (on your site, Reddit, Twitter, etc.)
• Ask a question of a specific staff member
• Use a specific hashtag to contribute to a conversation
• Become a repeat commenter/contributor

Internal involvement in conversation

If your newsroom or organization invests in online discussions, consider how to measure the success of those investments.

• Number of comments from staff
• Percentage of overall comments that are from staff
• Speed of staff in replying to comments
• Percentage of user comments that get responded to by staff
• Staff perceptions of conversation’s value
• Staff participation in live chats (on your site, Reddit, Twitter, etc.)

Qualitative feedback on content

• Comment cards at events (number of and value of)
• Connections made and fun had by users at events
• Email or other communication from users about effect of content on personal perspective
• Email or other communication from key stakeholders about effect of content on process/policy/conversation
• Feedback from focus group or advisory boards

Internal metrics

• Changes in newsroom/organizationsl culture (requests by staff to seek user feedback, perceived value by newsroom of user input, recognition by staff of outreach strategies)
• Increases in newsroom responsiveness (responsiveness to users’ phone calls, social posts or emails)
• Knowledge gained by newsroom about its audience (this is key — the value sometimes is in the internal knowledge of user preferences, even if a strategy or idea doesn’t resonate with users)
• Return on investment, for engagement and for measurement. Are the efforts getting more efficient with time? Is what you’re learning worth what you’re putting in? Are users responding to what you’re investing in?

User knowledge and intended action

It’s easy and free to embed google forms in or at the end of stories. This works for 20-question quizzes, but also for one-question polls. Did you learn what you hoped to learn from this story/post? Did this story reflect your view? Make it easy, then offer more ways to get involved for the people who participate.

• Participation in quizzes related to content (consider offering incentives for users to participate — that you pay for yourself or seek a sponsor for — or donating to a charity of the user’s choice for each completed quiz)
• Performance of users on quizzes (see what they know, then ask how much or how often they consume your content)
• Participation of committed users (by specific invitation) when asked to assess what they’ve learned

How else could you ask users, online and in person, about the effects of your coverage?

• Response (number and rate) to printed surveys at key meetings or gatherings
• Response (number and rate) to online surveys embedded in key content

Invite users to tell you if they:

• Consumed
• Learned more facts about a topic or perspective
• Have a better understanding about a topic of perspective
• Have changed beliefs or attitudes
• Have changed behaviors or plan to take action
• Have suggestions for future coverage

Diversity of voices or perspectives shared for content and agenda setting

A key measure for impact could be whether your efforts and priorities are shaped by users’ actual preferences, curiosities and needs. Also, whether you’ve captured their viewpoints: Did this story reflect a perspective you hold? Did this story help you understand a perspective less familiar to you?

• Number (or percentage) of sources/contributors from target groups
• Number (or percentage) of story ideas/suggestions/contributions from target groups
• Submissions in physical drop boxes for questions or story ideas at events or left at key events
• Contributions to an online or in-person poll: Are we on the right track? Do the perspectives shared here reflect your own?

External (community or organizational) change

When your goals are broad, spend some time talking about what metrics could help you track:

• Change to individuals, groups, organizations, systems, social and physical conditions
• Change in policies, processes, laws
• Change in quality or quantity of public discourse

Action taken related to newsroom/organizational resources

• Signups for subscriptions or members
• Donations
• Recruitment of friends
• New volunteers
• New advertisers
• New advertising dollars