6. Monitoring and Evaluation

Cash transfer programs should be measured against the same sectoral indicators and humanitarian benchmarks that CARE and implementing partners would normally use, as the objective of CTP will never be to simply deliver cash, but to improve outcomes relating to basic needs, livelihoods, food security, shelter, nutrition, WASH, etc. Therefore, the monitoring plans should focus on program objectives and outputs, rather than the delivery modality.

Some specific indicators may be added when implementing CTP, particularly those dealing with individual, household, community, or market-level outcomes. As for any indicators set by CARE, those used in cash programming should be gender-sensitive, and impact and process should be monitored from a gender-segregated perspective, to ensure the differing needs of women and men are covered adequately.

Outcomes and impact indicators:

  • Spending pattern within beneficiary households, through analysis of expenditure prioritization;
  • CTP effects on filling sectoral needs gaps (determining percentage of beneficiaries with access to 20L of safe drinking water per day, percentage of repaired/reconstructed houses that fulfill minimal safety and disaster risk reduction criteria, etc.) or effect on cross-cutting objectives when using multipurpose cash (percentage of households with confirmed reduction of negative coping mechanism index, percentage of households able to cover their Minimum Expenditure Basket [MEB]);
  • Impacts of CTP on community relations (e.g. ability to share, solidarity, power dynamics);
  • Impact of CTP on household relations, possibly determined by the number of complaints received by beneficiaries about CTP use within the household;
  • Impact on women, such as the percentage of women who own a mobile phone or who report having a say in how funds are spent;
  • Impact on power dynamics (e.g. decision-making on purchases, household investments)
  • Transformation in market systems, livelihood systems, or ownership and management of assets due to CTP, as seen by the number of traders, prices, volume, and quality of goods in local markets.

When CfW is being considered, monitoring scheme that examines the impact on the number of children dropping out of school to enable parents or caregivers to work is also recommended.

Process and output indicators:

  • Community satisfaction regarding targeting systems;
  • Community satisfaction regarding CfW project selection processes;
  • Beneficiary satisfaction regarding the chosen modality compared to in-kind aid;
  • Beneficiary satisfaction regarding the payment agent and distribution process;
  • Beneficiary perception of their safety throughout the process;
  • Whether cash or vouchers are received by the right person, safely, on time and in the right amount.

Tips on ensuring a gender focus in monitoring:

  • Disaggregate groups: whenever possible talk men and women separately and be sure to include ranges of ages, abilities, and ethnicities
  • Strive for representation in monitoring team: it matters not only who is asking the questions, but who is analyzing the data

Ask about access and change: see if CTP has changed who can access markets, or not, and what those changes may mean to different groups of men and women

Market monitoring should be part of the monitoring, evaluation, accountability, and learning (MEAL) framework when the project entails repeated payments, though is not implemented for one-off payments. In these cases, markets should be assessed at preparedness and design stages, to determine whether CTP is appropriate, with market impacts assessed as part of the final evaluation. Market monitoring is the co-responsibility of the logistics and programs team and should:

  • Ensure the grant amount will adequately cover the identified needs or MEB;
  • Ensure the quality and availability of goods is at least as good as it was at project onset;
  • Assess the continuous appropriateness of the chosen delivery modality;
  • Ensure the project is not creating inflation or doing any other harm to the market.

If a country program has done a pre-crisis market analysis, results should be shared with the wider team. Market monitoring should allow for assessment of multiplier effects of the project on the local economy. In this case, the multiplier effect is an indirect benefit of CTP, whereby money distributed to beneficiaries and later spent in local markets results in benefits to local traders. This positive effect cascades down to other market actors, beyond direct CTP beneficiaries. When assessing the multiplier effect, trading results from women and men should be separated, to make sure that indirect positive economic impacts benefit both. For more on how to calculate the multiplier effect, refer to the Social Accounting Matrix (SAM)

Market monitoring frequency should be determined by and will vary depending on the context. In volatile contexts, market monitoring can be implemented once a month. In more stable environments, it may be done every three months. MARKit[3] can be used to inform which market places and actors should be consulted during monitoring.

Market indicators for monitoring will depend on the modality chosen to deliver the project, with common indicators including:

For any type of modality:

  • Household access (physical, social, financial) to market to purchase key commodities;
  • Availability, price, and quality of key commodities in different types of markets (source markets, central markets, etc.);
  • Power dynamic in between traders (cartel, etc.).

When cash grants or value vouchers are used:

  • Prices of commodities used to determine grant/value voucher amount.

When commodity vouchers are used:

  • Availability and price of goods covered by vouchers for beneficiaries and non-beneficiaries;
  • Presence of traders not involved in the voucher scheme.

When the project uses CfW approach:

  • Beneficiaries dropping out of lower paying but more sustainable work for inclusion in the CfW/Food for Work;
  • Availability of workers for traditional daily labor employers;
  • Daily rate of unskilled labor.

 

Market monitoring should include strict minimum price monitoring, as sharing data collected for price monitoring may help support the entire response community. Other humanitarian agencies may be carrying out similar responses in nearby areas, and it may be important to understand how changes in nearby markets are conveyed to one another. Humanitarian clusters may also use this data for decision-making.

In some cases, it may be helpful for different actors providing CTP to monitor markets collaboratively. In addition to providing an opportunity for joint decision-making, added transparency regarding price and market monitoring results can build trust and understanding.

Guiding principles of price monitoring include:

  • Ensure consistency: When collecting prices, make sure they use the same units of measurement for items of similar characteristics, quality, variety, etc. Ideally, visit the same trader each time.
  • Check for irregularities: If one price is much higher or lower than the others, it is likely that there has been an error. In this case:
    • Double-check to see if the price was converted to the correct unit of measurement;
    • If the price was converted correctly, check to see if the reported price is correct, possibly by asking a key informant from the market;
    • If the price is correct, speak with traders to understand the reason for price difference.
  • Plot the average commodity prices across markets over time.
  • Be aware of seasonality: Prices change along seasonal patterns, so when monitoring prices, it is critical to use the seasonal calendar you have drawn. This tool allows comparison of price changes in a reference year and helps determine whether any observed price fluctuations are normal for that time of year.
  • Be aware of the difference between indicated and true prices: Negotiation and bargaining may lead to discrepancy between prices used for monitoring inflation/grant appropriateness and the actual price paid to a vendor. This can be moderated by cross-checking prices with buyers.
  • Combine price monitoring with volume monitoring as it is difficult to analyse market price trends without volume trends.

 

More information regarding price monitoring (although limited to food price monitoring) can be found in the MARKit: http://www.crs.org/our-work-overseas/research-publications/markit

[1] http://siteresources.worldbank.org/INTPSIA/Resources/490023-1121114603600/14017_chapter14.pdf

[2] P. Cretti (2010), The Impact of Cash Transfers on Local Markets, A case study of unstructed markets in Northern Uganda, CaLP.

[3] http://www.crs.org/our-work-overseas/research-publications/markit

.

 

As for any other projects, an important component of accountability towards beneficiaries is the setup of an internal complaints and a response mechanism.

Do:

  • Raise community awareness of their right to make reasonable feedback and complaints, and to receive a response within a certain time frame;
  • Ensure mechanisms are in place to deal with serious complaints like allegations of sexual abuse, fraud or other sensitive issues;
  • Use feedback and complaints information to improve project impact;
  • Make sure staff are well trained to handle complaints and know what to do when they receive feedback;
  • Help develop an internal learning culture, where feedback and complaints are welcomed and not feared by staff.

Don’t:

  • Establish feedback or complaints systems that are difficult to access by vulnerable groups or that can be manipulated by the elite;
  • Fail to investigate and act on feedback and complaints;
  • Forget to close the feedback loop – have you reported back to communities?

Beneficiaries should have at least two channels for giving feedback, depending on what communication options exist in a given location.  Options can include:

  • Information stations staffed by both women and men, with separate areas for women and men at the distribution site, if possible;
  • Feedback boxes at each distribution site;
  • A phone number (via WhatsApp/phone calls) for beneficiaries to contact;
  • IMO/Viber contact information..

 

Use of the feedback mechanisms should be clearly outlined through informational materials provided before and during distribution and through pre-distribution community meetings. Measures should be taken to ensure that all beneficiaries are aware of and understand feedback and complaint mechanisms, know how to submit a report, and can give feedback freely, without fear of retribution. All language, reading, seeing, and reasoning barriers should be accounted for and addressed.

All feedback should be collected by the partner M&E Team and channeled to the most appropriate department. Ensure that partner M&E team is staffed by women, to accommodate gender sensitivity needs. For extended cash distributions, information on feedback received and resolved will be provided to the CARE MEAL team through monthly reporting.

Serious complaints (such as sexual abuse, child abuse, fraud, and/or corruption) must be shared immediately with CARE program managers.

Each program manager must monitor:

  1. Type of complaint, feedback, and/or question received;
  2. When it was received;
  3. Was the complaint, feedback, or question acknowledged;
  4. Current status of the complaint (closed, in progress, etc.).

Complaints are tagged by organization, sector, project, location, etc., for efficient routing and processing, and are then categorized according to:

  1. Outside of scope;
  2. Ideas for future consideration;
  3. Request for information;
  4. Programmatic complaint- urgent;
  5. Women SEA / harassed complaint – urgent
  6. Breach of code of conduct-immediately pulled out of the system and referred to the Deputy Director who resolves according to policies.

Different job functions and their duties:

  • Complaint manager: oversees project management and operations;
  • Complaint registrar: manages complaint processing;
  • Technical focal point: manages complaint resolution;
  • Deputy director: manages operational leadership;

Country director: manages commitment and accountability.