Raise your hand if you’ve watched any World Cup matches this year. I don’t follow soccer in general, but must admit to being a bit of a fanatic when it comes to the World Cup. The timing of this year’s just-completed tournament has coincided with my thinking a lot about learning and evaluation for MeHAF’s new community-based initiatives,* and the foundation’s growth as a learning organization. I’m seeing lessons everywhere, even in my downtime as I watched the World Cup with my family.
Historically, we (the royal we: funders/nonprofits/social sector) have tended to focus on things (metrics and indicators) that are relatively easy to count – ones we think can best determine and/or demonstrate our hoped for impact. These emerge from within our current underperforming systems and are often framed within a linear concept of change: X causing Y resulting in Z. Is this the best way to assess the work – especially that of complex systems change – given traditional funding timeframes?
For example, in our educational system we often measure a student’s grade point average and SAT score as predictors of future success. However, recent research** indicates that the level of a student’s social-emotional intelligence might be a more accurate predictor. But how do we measure that?
Suppose we look at a soccer match as a system, and at the measurable indicators that might have been expected to predict the results of two World Cup matches played in the first round: shots on goal, fouls (the team with more fouls might indicate that it’s struggling to dictate the pace of the game and playing more defensively), corner kicks (the team with more corner kicks is most likely being more dynamic offensively deep in their opponent’s half of the field), offsides (the team with more offsides might indicate they’re a more active offensive team that takes chances, because their defense is strong and can weather a counter attack),time of possession, and saves.
21(8) Shots (on Goal) 8(7)
10 Fouls 12
7 Corner Kicks 3
5 Offsides 1
59% Time of Poss. 41%
3 Saves 4
Italy Costa Rica
10(6) Shots (on goal) 10(6)
10 Fouls 24
4 Corner kicks 5
11 Offsides 3
58% Time of Possession 42%
4 Saves 4
Measuring the system and its expected outcomes by these indicators you would expect that Ghana and Italy would have won or at the very least tied their opponents – the opposite actually occurred. In Match 1, USA beat Ghana 2-1, and in Match 2, Costa Rica beat Italy 1-0.
So how and why did a different result emerge? Maybe it’s because typical measures that are easy to count give us a snapshot of only part of a system rather than an understanding of the system in its entirety. They can neither determine nor describe success. The snapshot approach overlooks the relational and process elements of our work and the influential roles they play in supporting systems change.See Curtis Ogden, Dimensions of Network Success, Interaction Institute for Social Change blog, 9/19/12.
The relational aspect of social change is typically the least rewarded and tended to, but arguably may be the most critical. Given typical funding timeframes that don’t realistically align with the time horizon required to see long-term change, how can we measure increased trust or the other relational and process-oriented facets of the work in ways that make these factors more tangible products, and acceptable intermediate outcomes/indicators of potential long-term success? How do we adjust our focus from items that are easy to count to meaningful factors that help us better understand the how and why change occurs? These are the questions MeHAF is asking itself and its grantees.
*MeHAF’s community-based initiatives include our Thriving in Place, Improving Access to Quality Care, and Healthy Communities grant programs.
Thank you for contacting us!
Our group will review and follow up within 72 hours.
Thanks for your interest!