The “Time-bottlenecks” in Research Integrity and Validity

Focus Group Discussion exercise

According to USAID research standards, quality data must exhibit five key attributes, V-TRIP; i.e. Validity, Timeliness, Reliability, Integrity and Precision.

Valid data is data that shows a true representation of the measure of interest (indicator), and its changes can credibly be associated with the interventions in question. It should be free of sampling and non-sampling errors. The validity of data is achieved by developing proper data collection tools and their subsequent effective use during the data collection exercise. Soon after the exercise, the integrity of the data in question must be protected, and this is often linked to the capabilities of the management system in place to reduce the possibility of introducing bias either by transcription error or deliberate manipulation during data entry and cleaning.

In reality though, isn’t this just idealism? During a data collection exercise, there are usually many factors in play that may hinder the collection of valid data. Therefore, what follows data collection is often putting in hard work on preserving the integrity of invalid data.

The challenge does not start at the field but from the planning of a research exercise, and most of it has to do with time. There are key areas that are associated with the issue of time that must be dealt with precisely to avoid these “time-bottlenecks”. I conducted a mini-research for purposes of getting opinions and experiences (basically qualitative in nature) from fellow researchers and here are some of the concerns raised:-

1.    Training period. Proper planning ultimately determines the level of success of any given project. Therefore, it is advisable to spend as much time as needed to prepare, so that on execution every possible angle of challenges and risks will have been mitigated or prepared for. Training of enumerators is part of planning. Spending only a day on training enumerators who are going to carry out a seven day survey only to end up getting 50% of the responses wrong doesn’t make sense. Wouldn’t it be wiser to spend two days on training and increase the precision and validity of collected data to 90%? Release enumerators to the field only when you are sure they will bring you, not good but excellent data.

2.    Data collection period. A questionnaire that takes an hour during a mock survey in the training venue will not take the same amount of time in the field. It will take probably a half an hour more. Therefore it is not logical to expect an enumerator to bring back eight questionnaires at the end of the day. The plan must consider sampling method used and time of travel to access target participants. All these are about TIME. If you give unreasonable targets the enumerator will use unethical means (compromised integrity) to reach the target and the result will most likely affect the validity of the data.

3.    Sample size versus daily target. Often, the aspect of bias and assumption among enumerators comes in when they have had to ask the same questions over and over and keep getting the same answers. By the time they are administering the one-hour-long, fifth questionnaire of the day they have basically switched to ‘auto-drive-mode’. What they do is to assume responses to some questions will be similar to what they got before and therefore they do not pose these questions to the respondent but fill in the assumed response. This is also reported to happen whenever respondents look unsettled, seem to be in a rush or when the enumerator is tired, feels like they are far from reaching set daily target or are running out of time as the day concludes.

Is it possible to deal with these “time bottlenecks” to beat the issue of validity and integrity at the level of data collection?

Several suggestions were put across but there was no single standing solution. The suggested approaches must be combined to move from 90% that can be achieved with proper planning, to 98%. First of all sufficient time must be allocated and used in the planning phase. Train, carry out mock-survey, re-train, pre-test with a sample of targeted respondents then re-train. Ask questions and engage trainees.  It helps in gauging their level of understanding of the tool, their confidence on the tool and their level of preparation to undertake the exercise. Do not depend on getting phone calls to clarify issues for enumerators after deployment to field. Network reception may be terrible or something else may render it impossible to communicate, then enumerators will make-do with guesswork.  Attain an excellent mark before deployment. 

Secondly, allocate sufficient time for the survey. Don’t give unreasonable targets because enumerators will hit the target but will deliver invalid data. It was also suggested that sound recording would be a great tool for confirming validity.

The Rio+20 Nairobi Declaration

These are the main concerns of the Kenyan youths as per the  Kenyan Youth Strategy Meeting for Rio+20 Summit held in nairobi on the 13th and 14th October, 2011 at the UN complex, Gigiri.

Preamble

We, the delegates to the Kenyan Youth Strategy Meeting for Rio +20 at the United Nations Complex at Gigiri, Nairobi:

Acknowledge the African indigenous knowledge of the sacred value of the environment to biodiversity well-being.

Commit to promote innovations that will develop a green economy and promote the eradication of poverty.

Take note of the past declarations towards environmental sustainability both at the African and Global level, there is an urgent need for structural and infrastructural interventions in policy formulation, implementation and evaluation.

Recognize the current global environmental challenges, particularly climate change, which impact our common future and wellbeing, we commit ourselves to support of the following mechanisms:

  1. Good governance and transformative leadership.
  2. Promote Education, information exchange, communication and awareness
  3. Achieve sustainable agricultural practices to reduce hunger, starvation and enhance food security.
  4. Advocate for the development and implementation of sustainable development policies towards a Green Economy.
  5. Invest in and promote eco-friendly entrepreneurship and job creation.
  6. Attain sustainable green cities and villages.
  7. Promote public engagement and participation through culture and volunteerism.
  8. Promote Youth Development and capacity building

Download the Full Version of the Nairobi Declaration here: Nairobi Declaration 2

TECHNOLOGY: Making the most of mobiles

LONDON, 7 September 2011 (IRIN) – It is not often a technology guru will say, “Forget the internet!” but Ken Banks, founder of Kiwanja.net, advocates going back to basics – using mobile phones rather than the internet, and pretty basic phones at that.

While mobile phones are ubiquitous in Africa, the internet has nothing like the same penetration and is almost non-existent in rural areas. Says Banks: “For example, in Zimbabwe, there’s 2-3 percent internet penetration. If your amazing, whizzy mobile tool needs the internet, and you are looking to deploy it in Zimbabwe, you have lost 97 percent of people before you start.”

making the most of mobiles
Even the most basic mobile phones are able to use innovative tools

Dillon Dhanecha’s company, The Change Studio, was trying to distribute management tools and training through the internet, and admits it fell into exactly the trap Banks was describing. “We were developing short YouTube clips and so on, but I was in Rwanda a few weeks ago and trying to access our site from my Smartphone, and it just wasn’t happening.”

But there are plenty of options with even a not-very-smart phone: one of the pioneers was M-Pesa, designed as a tool for repaying microfinance loans. But Kenyans found all kinds of other uses; for instance, people afraid to carry large sums of cash while travelling would send it to themselves for collection at their destination. It was also key to the recent Kenyans for Kenya drought aid funding drive.

Tracking livestock

Another phone-based tool playing an important role in the drought-affected areas of East Africa is EpiCollect, developed by Imperial College, London, which allows the geospatial collation of data collected by mobile phone. Kenyan vets are using it for disease surveillance, monitoring outbreaks, treatments, vaccinations and animal deaths.

Even where there is no mobile-phone signal, they can record data by phone and store it until it can be transferred to a computer, producing an interactive map pinpointing where each observation has been made, with additional information about locality, even photographs, available at the click of a mouse.

Nick Short, of the NGO VetAid, has been greatly impressed by the possibilities, and the fact that ministries of agriculture and the UN Food and Agriculture Organization (FAO) can now track what is happening in real time.

“When I worked in Botswana,” he says, “We had an outbreak in the northwest of a disease called CBPP. It took us about two-and-a-half months to hear the disease was in the country. By the time we got there about 20,000 cows had died; we ended up killing 300,000 cattle.”

Short is also hoping its use during the current drought will help leverage assistance, helping potential donors pinpoint exactly where their money will be going. “Just watching the BBC is not good enough,” he says. “This way people will actually see the animals they are benefiting.”

Banks has developed an SMS-based tool, Frontline SMS, which will work with even the simplest phones. By connecting a standard mobile phone to a laptop, data can be received or transmitted wherever a basic phone signal is available, without any need for 3G or an internet connection. It is freely available to any not-for-profit organization.

In Afghanistan it has been used to send out security alerts to field workers. It tracks drug availability in clinics across East Africa, and house demolitions in Zimbabwe. Civil society groups in Nigeria have used it to collate information from their election observers, and it is used by a company distributing agricultural pumps in Kenya and Tanzania to keep in touch with farmers. Specialized versions are being developed for health and educational sectors, for NGOs working in law and microfinance, and for community radio stations.

Nay-sayers

But while the developers may be entranced by their tools, some dissenting voices were raised at the 1 September meeting in London. A Ghanaian lawyer, who declined to be named, said: “I find this depressing. Just monitoring is not sufficient; monitoring is just collecting data while people die.”

Short disagreed: “Without these tools no one knows what is happening in remote areas, and if you don’t know what is happening, you can’t do anything about it… If there were an outbreak of disease, we wouldn’t know about it until it was too late, and the animals were already dead.”

Shewa Adeniji, director of a small NGO called Flourish International, which sponsors community clinics in Ghana, expressed wider concerns about Africa’s love-affair with the mobile phone. “There are glaring benefits, but it’s adding to poverty on the ground. You have people in Nigeria struggling to pay 1,000 naira for medical insurance, and yet they will buy 1,000 naira top-up for their phones. These are misplaced priorities and meanwhile the telecom companies are going to African countries to milk them of their money.”

Banks accepted there had been cases of people buying phone credit rather than food or sending their children to school but pointed out that building a transmission network, especially in rural areas, costs money. “If mobile phone [companies] didn’t make money, we wouldn’t have the network of coverage we have. And once the network is there, people can use it… The technology can be used to do both good and bad, and you can’t really control that. You can just as easily spread a hate message as a health message, but you just have to hope that people will use it in a positive way.”

eb/mw
Theme (s): Aid Policy, Early Warning, Food Security, Health & Nutrition,
[This report does not necessarily reflect the views of the United Nations]
Source: http://www.irinnews.org/report.aspx?reportid=93675

%d