Program Evaluation – Lecture 4 Stakeholder Analysis, Mapping, and Engagement

Recap lecture 3

Theory of change
Logic model
DAG: Directed Acyclic Graphs
Causal models
Jobs Plus in NYC case
Ask a good question game

Today’s agenda

Stakeholders
Identify key stakeholder
Engaging stakeholders
ARPR-E case
Review of logic models

The first step

Stakeholder Identification and Analysis Techniques

Conducting basic stakeholder identification and analysis
Choosing evaluation stakeholder analysis participants
Creating an engagement plan (approach, frequency, goals)

Questions to ask

Who the evaluation sponsor(s) is.
Who the day-to-day process champion(s) is, meaning the day-to-day evaluation process manager; this person may be the evaluator, but maybe not.
Who the stakeholders, key stakeholders, and primary intended users are.
What the purpose(s) or intended use(s) of the evaluation is.
Who the members of the evaluation coordinating committee or task force are, if such a group is to be formed.
How the different stakeholders will be involved at different stages in the evaluation process.
Who the evaluator(s) is, or at least what his or her qualifications should be, and who the members of any required evaluation team might be.

Types of stakeholders

Implementers
Decision makers
Participants
Partners

Useful thinking

Internal vs. external
Primary vs. secondary
Influential vs. non-influential
Supportive vs. adversarial

How to involve stakeholders

Invite stakeholders to a meeting
Involve stateholders in key activities
- Plan
- Implement
- Outcome
- Recommendation

Engagement level

No engagement
Engage as data source
Inform
Consult
Involve
Collaborate
Empower

Mapping stakeholders

List all potential stakeholders
Categorize them
Describe interest and influence
Note preferred communication channel
Develop strategies
Monitor and update
Share and review
Develop an action plan

Stateholders strategies and criteria

An auspicious day to witness first power from @SouthForkWind with many dedicated colleagues and friends. For the first time in history, families and businesses on Long Island are using offshore wind! Congrats to @OrstedUS @EversourceCorp @EHTownTrustees and @LongIslandPower! pic.twitter.com/ughxe03eHK
— Doreen Harris (@Doreen_M_Harris) December 7, 2023

Offshore wind example

Stakeholders	Interests and concerns
Local residents	Concerned about visual and noise pollution, potential impacts on property value, and benefits to the local community like job opportunities and community facilities.
Environmental & conservation groups	Interested in the project’s ecological impact, particularly bird migration and marine life, but also supportive of renewable energy initiatives.
Local and national governments	Interested in sustainable energy goals, economic development, job creation, and regulatory compliance; concerned about residents NIMBY.
Utility company	Interested in increasing share of renewables per regulation, concerned about grid reliability.
Potential energy consumers	Interested in reliable, cost-effective, and sustainable energy.
Company investors and shareholders	Looking for profitability, sustainable investments, and risk management.
Local fishing and maritime industries	Concerned about potential disruptions to fishing routes and marine ecosystems.
Nearby tourism businesses	Concerns about potential impacts on the aesthetics of the landscape and tourist attractions.
Renewable energy advocacy groups	Supportive of clean energy but keen on ensuring it’s done right without unnecessary environmental impacts.

Prioritize stakeholders

New York offshore wind delay evaluation example

Level 1 Stakeholders will include developers, State policy makers, regulators. You decide to in close contact with these groups on a monthly basis.
Level 2 Stakeholders will include people like local policy makers, utilities who you need to engage with for the same purpose, but not as often. Your team will engage with these people quarterly.
Level 3 Stakeholders will include people who aren’t directly involved but who need to be kept informed about your efforts, like heads of community organizations and advocacy groups, and committee leaders. You decide to update them half-yearly/annually.

Eisenhower Matrix

I have two kinds of problems: the urgent and the important. The urgent are not important, and the important are never urgent.

— Dwight D. Eisenhower

Missing opportunities

Not engage stakeholders
Engage stakeholders too late
Not inform stakeholders

UNEG Principles

Identify the stakeholders and the role these will play in the evaluation.
Promote an inclusive and diverse stakeholder engagement with a tailored approach by stakeholder
Engage stakeholders early on and throughout the evaluation process
Seek opportunities to engage with stakeholders in order promote an evaluation culture
Ensure communication with stakeholders is clear and tailored to their specific needs
Abide to the highest ethical standards in engaging with evaluation stakeholders, ensuring respect and sensitivity to stakeholder’s diversity and human rights
Seek and use stakeholders’ feedback on the evaluation process for effective learning and continuous improvement of evaluation practice

Story

John Muir and the National Park System

References

Newcomer, Kathryn E., Harry P. Hatry, and Joseph S. Wholey. 2015. Handbook of Practical Program Evaluation. 4th edition. San Francisco: Jossey-Bass.