About a month ago a couple of my colleagues attending the ISCRAM 2014 Conference – an international conference on information systems for crisis response and management – reached out to me during one of the sessions. They informed me that the digital humanitarian attendees were asking for proof that the digital humanitarian responders had direct impact on decision makers during the emergency response in the Philippines in the aftermath of Typhoon Haiyan. I found this request difficult to answer. Here is why:
Over the past 10 years, I have seen OCHA and the humanitarian community create many products. Unfortunately, due to budget constraints, humanitarian organizations do not usually have the capacity to follow each product sufficiently to gauge their effect. This is one of the hindrances within the community in measuring a product’s impact. A second hindrance is that there is no identical parallel emergency not using the product for comparison purposes. The set of circumstances in each emergency is different. Thirdly all of the products, systems and organizations work together to produce an emergency response. It is difficult to measure one product in isolation.
In the Philippines, the Digital Humanitarian Network provided OCHA with 40% of the data for our first 3W (Who’s doing What Where) product [PDF]. Fortunately, I was able to put the 3W and other volunteer-developed products directly into the hands of the UN’s most senior humanitarian official, Valerie Amos, and spend a few seconds explaining them before she went into meetings and interviews. This opportunity is extremely rare. Most often, due to nature and circumstances of emergency response, an IMO (Information Management Officer) is not present when such a product is distributed.
I cannot know the exact impact that the 3W product had when it was handed out at donor meetings, distributed in inter-cluster coordination meetings, and shared to over 2,000 people on a mailing list. I can only believe that its use greatly contributed to the humanitarian response. It augmented our other products and arguably helped to paint a more complete picture for a range of decision makers. Providing proof that this was the case was not something we sought to measure. It seemed obvious.
However, in discussing with colleagues how to respond to the digital humanitarians’ request, I recalled something that I had read in Gary Klein’s Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making about pieces of information required to make a decision. After digging through my notes, I found what I was looking for: the author argues that experts typically use 5 cues to make a decision.
Even though only 5 cues will matter, experts tend to ask for more data than they will use. In an emergency, it is a constant stream of information from which decision makers need to take action. Experience helps responders develop patterns that let them size up situations quickly and accurately, therefore enabling them to judge what to pay attention to and what to ignore. They reserve their attention for the most important cues and aspects of a situation.
At the same time, we know that people can usually only hold 2 or 3 things in their short-term memory, meaning there is only a small chance that a given decision maker will be able to give you post-facto feedback on your product if they choose to ignore it in the stream of other cues. And, in a recently commissioned learning report by ACAPS entitled The Use of Evidence in Humanitarian Decision Making [PDF], the authors noted that “current processes of decision making tend to be undocumented and un-transparent. It is therefore hard to judge whether or how information and evidence have been used to inform them” while also noting that it is “common for experienced staff to base decisions mainly on past experiences, instinct, and assumptions – even in the face of contradicting evidence.” The result is that it is unfeasible to measure the impact of one specific product. There is a lot of information available, different experience has framed it differently for different decision makers, and it is impossible to ever know which combination of cues each decision maker used or ignored.
What is a cue exactly? For me, it is a range: from map products like the 3W, to verbal discussions with colleagues, to visual signs in the environment, to experience in past emergencies, to situation reports from different organizations, to assessments taken across the emergency, to political pressure from various governments. The challenge is how to figure out what cues are important to decision makers during emergencies and then focus our data and information efforts to this end.
Although there has been tons of research into decision making outside the humanitarian sector, the field of study is in no way complete or conclusive. The limited resources of humanitarian organizations to put towards studying decision-making (when compared to private industry), make understanding its intricacies even more difficult.
However, some work has started.
As difficult it is to answer their question, I am glad that the digital humanitarians are asking these hard questions. We need as many catalysts as we can get to help us find ways to understand decision-making and to make decision-making during emergency response more effective. Those affected by disasters depend on it.
One major challenge during a humanitarian response is simply getting organizations, governments, and private sector to share their data. I have heard an amazing range of excuses from the logical (security), to fear of others trying to find un-constructive or pointless correlations (like Spurious Correlations), to the rather absurd (“HQ has not told me to share”). Even groups who you think have to share often do not. In an interview with NPR recently (MP3), Graeme Wood noted that the different peacekeeping missions in the Central African Republic, one of the most severe crisis in the world right now, do not trust each other enough to share information or operational plans amongst themselves. There seems like an endless list of reasons that people find to not share data. So, how can we get people to share? I see it as a three-prong approach: technology, human, and policy.
Of course we can, will, and need to build technical applications to enable people to share. As Jonathon Morgan of Ushahidi noted in the blog post announcing Crisis.net, which aims to be the crisis data firehouse, we are in an era where data is “locked away in forgotten databases, undocumented APIs, or proprietary formats that hide data in pie charts and situation reports.” The Humanitarian Data eXchange team is working on a platform that will define exchange standards, accept data, curate valuable information, provide an analytical interface, and enable a common operating picture for humanitarians. Both are formidable efforts that I am following keenly. I have been around the humanitarian field long enough to realize that a technical solution will unfortunately not magically change everything. We will need to put effort into the human and policy aspects of the data sharing puzzle.
Bringing the human element to technical solutions will help. As Frog points out, any technical solution must listen to and integrate user needs while adding benefit and using data responsibly if we want people to be comfortable with sharing data. And, as Eduardo Jezierski suggested with a health commons concept and Patrick Meier notes in his #noshare concept, we need to treat data about individuals with utmost care. Any technical solution is going to need to incorporate such integrity at its core - it will need superior user permissioning around data access.
But, when reflecting on the above, I recognized that we still need more “human” effort than just building technology to meet user requirements. OCHA runs information management working groups (IMWG) in every major crisis to which they respond and this does help as it brings a human, in-person interface to a technical structure. Yet, having run several of these IMWGs in major emergencies, I know that it is still not enough. And, I am regularly asked by OCHA Information Management Officers how to get partners to open their data. So, I decided to start a list of creative examples of creative data sharing tactics that I have either experienced or read about in the past. While I cannot personally condone the following list, I found these examples to be quite creative. Many have added a social element, which I have advocated for in the past.
Beer-for-Data (Afghanistan): Visitors to a bar were given a free beer if they added their data to a central hard-drive. Data shared had to be ‘shareable’ with anyone. Any visitor could then take a copy of the 1 TB of data (photos, imagery, white papers, point data, etc). The concept spawned at least one spin-off in the US.
Facilities key (Pakistan earthquake 2005): Where the international responders set up their forward operations base, there was only one toilet. The key to that toilet was held in the central information center. In order to get the key, responders obviously had to visit the information center and share.
3W data not shared? No flight (Indonesia tsunami 2004/5). The most senior UN official in Banda Aceh instituted a policy that humanitarians could not fly on UN-HAS unless their 3W and contact information had been shared with the Humanitarian Information Center.
Contact details not shared? No flight (Sudan, 2005-07). The operators of UN-HAS used an openly available OCHA contact database to decide if someone was eligible to fly on any of their flights [this was neither agreed nor sanctioned by OCHA]. If the person’s details were not on the database, UN-HAS would not issue them a ticket.
Social center(s) (Pakistan 2009-11, DPRK 2000+, etc). Coordination agencies have set up social centers that attract responders to come together, meet each other, and (as a result) become more willing to share data.
Press coffee (Africa 2013+). In some of the long running African protracted emergencies, humanitarian press officers stopped holding press conferences. Rather they started something called “press coffee” where the journalists, public information & information management officers, spokesman and authorities hold casual events to talk about the humanitarian situation and share key information in both directions.
However, even if we can find creative ways to entice people to share - both carrot and stick approaches - we have not yet solved the overall policy challenge that stops information from flowing. There are some agreements within closed networks such as the Inter Agency Standing Committee’s Operational Guidance on Responsibilities of Sector/Cluster Leads and OCHA in Information Management [PDF], but these fall short on addressing the full community.
That is why I am working with Leandro Salazar-Liévano to draft the Open Humanitarianism concept. We want to leverage the open movement such that if a humanitarian organization or government (donor or recipient) wants to call itself an Open Humanitarian organization they will need to adhere to certain principles and behaviours, including an honest commitment to opening and sharing their data. When an organization has labeled itself as Open Humanitarian, I would expect no delays during any humanitarian crisis. And, if such an organization stalled at either the local or global level, then anyone could publicly “press” these organizations as Source recently did with some of the modern data journalism sites, like FiveThirtyEight, who promote open yet struggle to share their work. I want humanitarian organizations to strive towards being open, transparent, participatory, collaborative, and accountable.
I believe that if we can find the right combination of technology, human or social relationships, and overarching policy, we will soon find a more efficient and effective humanitarian system.
If you would like to share your data sharing stories or join Leandro and I in our Open Humanitarianism adventure, please comment below.
Last summer I had the fortunate pleasure to work with Annie Waldman for a few months. With her journalism background and enrollment in a Data Journalism masters, simply watching how she worked, how she searched for data and how quickly she could augment a story with that data was inspiring. It was at that time when I realized that the Data Journalism discipline would soon be impacting the humanitarian sector. Knowing that “facts tell and stories sell”, how can we use the increasing amount of humanitarian-related open data to augment good story telling (and information products). We need people who can find data in a range of locations and then be able to look at the pile and distill it down to interesting, impactful stories. In my day job, that got me thinking about how this would impact the work UN-OCHA's Information Management Officers do and how that work would change in the future to support improved data-driven storytelling.
When I was leading OCHA’s information management response to Typhoon Haiyan, I regularly helped clean and compile the Who’s doing What Where (3W) data in the early days of the emergency. But, after a few cycles, I realized that there was a lot of very interesting facts being lost as the data was summarized into matrices or maps and only the big numbers (e.g. number of people fed) were pulled out for official reports. We were not using the data to tell compelling stories or even relay facts in an interesting fashion to decision makers. As Nate Silver noted in The Signal and the Noise: Why So Many Predictions Fail-but Some Don’t, “The numbers have no way of speaking for themselves. We speak for them. We imbue them with meaning”. I needed to find a way to make the 3W data interesting. Late one evening, I started talking to the team about developing comparisons as a way to draw out interesting facts from the 3W. The interest factor “hit home” when we calculated which western city could be covered with all the tarpaulins that had been delivered to date. Although we did not release the details of that comparison, we started pulling interesting facts from the 3W and doing some comparisons. On 24-November we released our first 3W: Interesting Facts as part of the regular 3W package. Once we did that, people wanted more and it turned into its own product. And, as you see below, we were even able to include some interesting comparisons. The 3W Interesting Facts product quickly became the most popular product in our physical kiosk in Manila. Why? Because it shared small facts in a digestible, interesting format with a bit of storytelling effort.
Re-inspired with the Humanitarian Data Journalism concept, I arranged for Daria Kireeva to join me to do some research and work on the topic. We are going well beyond the interesting facts concept. We are focusing our initial efforts on augmenting early narrative UN reports from the Philippines with data, comparisons and trends. We chose the Philippines as I know the context and, in many respects, it is an easy case given that there was little security concern and there was a lot of data available. If the Philippines case proves difficult, we can imagine what it will be like in places like the Central African Republic where data is sparse and security is a major concern. However, our hope is that we will produce examples and materials that will help spark ideas and efforts throughout humanitarian organizations. We want humanitarian data to help produce even more compelling humanitarian stories. Stories that help decision makers to understand the situation and to compel both governments and individuals to respond with even bigger hearts than they already do today.
But, to do this, we know that data preparedness will be key. I do not mean just the regular operational data. I mean understanding the full range of data available that will or could be relevant to a given emergency. Why so much? So that when an emergency hits and you want to show a multi-year trend or a comparison against a similar emergency, you know where to go. You will need to be able to do it in a hurry. You will not have time to search through the Internet. Such work will not be easy. In Guiuan, Philippines I combined official humanitarian community data with data that was found on local Filipino news sites to create a bunkhouse graphic within minutes. So, I know that it can be done. But, how much better could have the product been if I had fast access to even more relevant data?
OCHA has recently started the Humanitarian Data eXchange project which will initially house over 150 baseline humanitarian indicators and eventually expand into a larger data repository. This site will become a great resource for humanitarian data. ACAPS has created data-rich preparedness packages on Bangladesh for Flash Floods and Landslides, River Flooding, Haor Flooding, Waterlogging, Cyclones, Cold Wave [PDF for cyclones without registration requirement]. Even beyond these, we will have to stop and recognize that there will be different data sources out there that must be accepted and used if we want to put together the most complete picture as possible for decision makers. Wikipedia and social media are obvious examples. But what if we stretched our thinking even more? What about collaborating with Data Brokers? Data Brokers are heavily active within the western world in efforts to collect a huge amount of personal data about us that we do not even realize. Is it much of a reach to imagine that such companies will eventually get involved in collecting data shortly after a crisis? Of course that could raise some serious privacy issues, but if we were to work collaboratively with them and ensure that they adhered to humanitarian principles, perhaps they would become a great, new data source to help tell constructive and compelling stories.
What will the result be? My preference in the short-term is to think that we can use data to augment our existing reporting and storytelling. But, I would also love to see a humanitarian data lab that includes capacity to write about humanitarian response from the data perspective just as the FiveThirtyEight Data Lab does about main stream media events. It could tackle hard questions like “Has Humanitarian Aid Been Effective in Sudan?” just as FiveThirtyEight wrote about increased disaster costs not being related to climate change. Now that would be a exciting challenge.
I think that the future holds an interesting time for the humanitarian sector, its data efforts, and its story telling. That is why I have enrolled in the (free) Doing Journalism with Data: First Steps, Skills and Tools course. Will you join and become a future data-powered humanitarian story teller?
I look forward to reading your data-driven humanitarian stories.
Last fall, I had the opportunity to attend a Global Solution Networks event in Brussels where I was part of a panel with Thomas Hale who spoke about the concepts of Orchestration and Gridlock. Although he talked about these topics in relation to climate change and other thematic issues, I started reflecting how they relate to humanitarian information management. First I read a paper entitled Orchestration and Transnational Climate Governance that he authored with Charles Roger. Then I read a book he co-authored entitled Gridlock: Why Global Cooperation is Failing when We Need It Most with David Held and Kevin Young. Although they are focused on specific topics, I found several parallels in humanitarian information management and the challenges that we face today. I also realized that some individuals are starting to “orchestrate” within our field, but we can do A LOT more as organizations.
The authors defined gridlock as “the growing gap between our need for global solutions and the flagging ability of multilateral institutions to meet that need. This represents a breakdown of global cooperation that we call gridlock” and outlined the paths to gridlock as having been: multipolarity, institutional inertia, harder problems, and institutional fragmentation. Sound familiar to any of you information management practitioners?
How big is the challenge becoming? In 1909 there were 37 intergovernmental organization and in 2011 there were 7,608. According to Yearbook of International Organizations, international nongovernmental organizations had increased from 832 in 1951 to a whopping 56,834 in 2011. Mancur Olson noted a truism in his book - The Logic of Collective Action - that “the larger the group, the less it will further its common interests”. Such numbers make it much clearer to me why coordination is extremely difficult, yet very critical. So, if we have grown this big, how does the formal humanitarian community find a way to move forward without completely grinding to a halt?
But you asking: “Are we really in gridlock?” Perhaps we have been there for so long that we don’t know the difference. When working in a political entity like the United Nations, a information management officer must respect the sovereign rights to a country. If a country does not want us to respond to an emergency, we cannot (we have to be asked). We have to be VERY careful how we draw borders on maps. Often we cannot release products publicly without fear of government reprisal. We often stick to official numbers released by the government even if other sources contradict them. But, with the technology available in today’s age, the barriers to entry into humanitarian information management have dropped drastically. It means that groups can be quickly created and be completely virtual while not having to respect an affected countries sovereignty (e.g. to help a local NGO if they want). The Digital Humanitarian Network helped a local NGO with inter-organizational coordination during floods in India where the UN was not asked to respond. ACAPS, a non-governmental organization, can undertake and release analysis about emergencies (like the Syria crisis) when the UN cannot do so publicly. Are these kind of groups the mechanisms to help us escape gridlock?
With such actors maturing quickly, international organizations face the possibility of orchestrating our way out of humanitarian information management gridlock. What is orchestration exactly? International Organizations “engage in orchestration when they enlist intermediary actors on a voluntary basis, by providing them with ideational and material support, to address target actors in pursuit of IGO governance goals. Orchestration is thus both indirect (because the IGO acts through intermediaries) and soft (because the IGO lacks control over intermediaries).” Essentially, I see it a matter of aligning the goals of different groups to achieve a set outcome. It is neither delegation nor simple outsourcing.
So, could the DHNetwork and other NGOs become the intermediaries for the international humanitarian information management responders? Absolutely. Of course we will face challenges related to authority, data security & privacy, and legal frameworks. But, several of the DHNetwork members are further ahead in thinking about and finding solutions for these issues than most formal humanitarian organization. Beyond helping the international responders, groups like the DHNetwork can bring the world to help with complex challenges we often only assign a handful of people. Not a bad idea - no?
In my experience of pushing in the direction of bringing more people (virtually) to help me with hard humanitarian information management problems, I have learned a couple things that we need to keep in mind. First, we cannot simply consider them volunteers and we must recognized that they will often give more than 100%. Second, we need to value and support them. What can the UN and other International Organizations bring to the relationship? We can bring convening power, provide coordination support, lend legitimacy, enable ideational factors, and release material & intellectual resources. Simply put, we can provide the entrepreneurial environment to foster such work and collaboration.
Who do we need to make orchestration happen? We know that there are willing partners in the DHNetwork and several information management NGOs. The DHNetwork has even created guidance on how to collaborate with them. But we need more “orchestrators” in the formal humanitarian organizations. These need to be risk takers, innovators, and have a desire to improve the system. Shadrock Roberts, when he was with USAID, was a perfect example. We need more Shadrocks. These are the kind of people who need to be organizationally recognized, be protected by central innovation units and be allowed to fail occasionally. And we need more creative partnerships so that at the time of crisis, when an organization’s culture becomes malleable, orchestration can happen.
When I reflect on my work in UN-OCHA and co-founding the DHNetwork (in my spare time), I have realized that orchestrators are going to be the ones who grow the symbiotic exchange between the bottom-up (DHNetwork) and top-down (UN) camps. As Patrick Meier mentioned, in his introduction to my keynote at the International Conference of Crisis Mappers 2013, I have become somewhat of a “translational leader”. We need more people who are willing to actively straddle the two sides to make things happen. We need strong leaders who advocate reform that can lead us beyond gridlock. Will you be one of them?
This problem is hard. A few people will not be enough.