Step 5 – Value the changes / benefits

In this step, you will: Establish the relative importance of the changes from the perspective of those affected.

This step helps to determine the relative value of the benefits or changes that you identified matter the most to people, using a method called swing weighting. 

For this step
Resources required	Interviews and workshops with directors, program managers and volunteers.Interactive, participatory evaluation tool using swing weighting to assess the relative value of the changes.
Expected outcomes	Generation of value weights
Tools	Stakeholders for value weighting process (Tool 5a) Value weight generation tool (Tool 5b)

Process

Understanding swing weighting 

This step helps assess the relative importance of the different benefits resulting from your program. Swing weighting is a well-established multi-criteria decision analysis (MCDA) method, which is especially helpful when comparing intangible or difficult-to-quantify benefits. It evaluates how much value (or “swing”) a benefit gains when it moves from its worst state (the baseline) to its best state (the level after the program’s implementation), from the perspective of those affected.

1. Determine who will be providing their value weights (Tool 5a)

Legitimacy, defensibility, and rigor in the value weighting process are crucial since the results are used to translate benefits into dollar equivalents. Therefore, there are several things to keep in mind when engaging with participants in the process. Of course, there are practical factors such as available time and resources, but it is essential to prioritize those directly affected by the program. Remember that values are unique to each individual, and require thoughtful judgments and a good understanding of the context. The process should be transparent, iterative, and collaborative, ensuring that the final weights for each participant are well-informed and that they are comfortable with them. 

It is important to think about a few criteria for a weighting process that is objective, legitimate, and defensible as possible:

• Diverse and affected parties:  Those impacted by the program offer essential perspectives, lending legitimacy to the process. Involving a diverse group ensures that a broad range of views is considered, preventing any one perspective from skewing results.
• Informed participants: Participants should have a good understanding of the context and how to do a swing weighting exercise. This can be achieved through briefings, training, or other educational methods prior to the exercise.
• Transparency: Clear communication about how the weights are elicited and used builds trust and allows for any biases or errors to be identified and addressed. 
• Independence and avoiding cognitive biases: While collaboration is encouraged, participants should be allowed to express their individual views without undue influence from dominant voices, or groupthink or other biases (e.g., confirmation bias). Proper facilitation and education can help mitigate these biases.
• Reflection and iteration: Participants should have the opportunity to reflect on their judgments and make adjustments, especially after group discussions or when new information is presented. 
• Ethical considerations: Participants should be treated fairly, and the process should be ethical. Coercion or manipulation would undermine the legitimacy of the resulting value weights.
• Validation: Post-process validation can be used to check the defensibility of the weights. This might involve comparing the results to other methods or checking for internal consistency.
• Documentation: Thorough documentation of the process, including the rationale behind certain weightings and any challenges faced, can aid in defending the legitimacy of the weights.

Some groups or individuals to consider including are:

• Volunteers and staff: These individuals often have a deep personal investment in the program, and can speak to motivations that might be purely intangible; like personal satisfaction, community building, or a sense of purpose. They also can bridge the gap between local community members and technical experts, providing a balanced perspective grounded in reality.
• Local community members and leadership: These are the individuals directly impacted by the quality of the water and are often the primary beneficiaries of a community water monitoring program. They can provide first-hand accounts of the intangible benefits, such as increased sense of safety, community cohesion, or cultural preservation.
• Local environmental groups: They often have a vested interest in the health of the local water source and can contribute knowledge on the broader environmental benefits and ecosystem services that the community might value but might not be immediately obvious.
• Water quality experts: These individuals can provide technical expertise and can quantify how monitoring efforts lead to improvements in water quality. They can help relate tangible water quality data to potential intangible benefits.
• Health professionals: The quality of water has direct and indirect implications on public health. Health professionals can speak to the potential benefits in terms of reduced disease prevalence, health cost savings, and overall community well-being.
• Economists/social scientists: These experts can help in translating intangible benefits into tangible economic or social metrics. Their expertise in valuation techniques can be invaluable in such exercises.
• Local government representatives: They can offer insights into the broader community benefits, potential policy implications, and how the program might fit into larger regional or national goals.
• Cultural or indigenous leaders: In many places, water sources have cultural, spiritual, or historical significance. Leaders from these groups can provide insights into these intangible values that might not be immediately obvious to outsiders.
• Youth representatives: The younger generation can offer a unique perspective on the value of water sources for the future. They might place different weight on intangible benefits and offer insights into long-term community aspirations.

2. Setting up a table of important end benefits and the associated changes from the program – baseline to current (Tool 5b)

Using Tools 4a and 4b, list the key end benefits and associated changes your program has influenced. It should already be organized into categories like water quality, community engagement, environmental awareness, regulatory compliance, ecosystem health, and public health improvement. For each category:

1. Baseline (Before): Describe the initial status before the program started, highlighting observed conditions or limitations. 
2. Change Attributed to Program: Outline the improvements compared to the baseline, showcasing the program’s tangible advancements. 

Make sure there is clarity and coherence when presenting the changes, and consider adding notes or specific data points to emphasize progress. This structured approach highlight the transformative journey of the water monitoring initiative, ensuring clear communication of its impact.

3. Assigning Swing Weights (Tool 5b)

Once the benefits and changes are outlined, work with participants to rank them based on their perceived importance. The most impactful change receives the highest rank (#1) – ranked as the foremost swing – and typically is allotted a weight of 100. The subsequent changes are then ranked accordingly, relative to the top-ranked transformation, reflecting their comparative significance. 

This process facilitates the identification of the most influential changes and creates a hierarchy of impacts, quantifying the relative importance of each benefit. Step 5 enables a clearer understanding of the varying degrees of impact, guiding your calculations for value and return on investment in Step 6.