Agenda

Day One: Wednesday, November 13, 2024

10:15 EST

45 min
Élyse McCall-Thomas

From Theory to Practice: The Development, Implementation, and Impact of Evaluation Policy

Élyse McCall-Thomas, Evaluation Manager, Natural Sciences and Engineering Research Council, PhD Candidate, University of Ottawa

  • The journey from theoretical frameworks to practical application in evaluating policy.
  • The key components of evaluation policies.
  • How individuals and organizations translate these theories into actionable practices.
  • Implementation challenges and strategies.
  • Real-world examples of integrating evaluation policies into organizational processes.
  • The broader impact of policies on decision-making, accountability, and policy improvement.
  • How evaluation practice can influence future policy directions.

11:00 EST

45 min
Jérome Mercier

Analytics and Policy Evaluation

Jérome Mercier, Director General, Evaluation, Employment and Social Development Canada

  • The role of evaluation in the policy development ecosystem with a focus on its systemic analytical approach bringing together various quantitative and/or qualitative lines of evidence.
  • An overview of commonly used lines of evidence along with high-level considerations about complementarity that are often considered when determining which ones to use.
  • The distinction between the concepts of contribution and attribution in policy evaluations.
  • The importance of neutrality, prudence, balanced reporting, and transparency when presenting evaluation evidence.
  • Thoughts on the future of analytics and policy evaluation.

11:45 EST

45 min
André Loranger
Keynote Address

Policy Evaluation in the Digital Age: New Methods and Approaches

André Loranger, Chief Statistician of Canada

  • Data-driven policy evaluation.
  • The role of big data and machine learning.
  • Identification of policy outcomes and unintended consequences.
  • Integration of administrative data such as health records, tax data, and employment information with traditional survey data for more comprehensive evaluations of public policies.
  • The use of Gender-Based Analysis Plus (GBA Plus) as a critical component in evaluating how policies affect various groups.
  • How digital tools help to disaggregate data along lines of gender, age, ethnicity, and other demographic factors, ensuring that policies are inclusive and equitable.
  • How new digital tools allow for real-time or near-real-time evaluation of policy initiatives.

12:30 EST

40 min

Break

13:10 EST

40 min
Denise Gareau

Using Robust Intersectional Gender-based Analysis Plus (GBA Plus) in Policy and Program Evaluation

Denise Gareau, She, Her, Elle, Director, Gender-based Analysis Plus (GBA Plus) Unit, Women and Gender Equality (WAGE) Canada

The Government of Canada is increasingly applying GBA Plus in the design and implementation of federal initiatives to help ensure more inclusive impacts. That’s because GBA Plus leads to more sophisticated understanding of diverse people and the barriers they face by guiding analysis of discriminatory norms, practices, systems, and structures that create/perpetuate inequalities. It helps provide a clearer view of how government actions affect – and are affected by – the context in which they are implemented. Robust and intersectional GBA Plus helps ensure that the evaluation of policies and programs are:

  • Inclusive.
  • Challenge assumptions about power and agency.
  • Sensitive to context and to the intersecting factors that create unique experiences of inequality.

By sustaining its commitment to GBA Plus and to its continuous improvement for close to 30 years, the Government of Canada has strengthened its capacity to generate meaningful evidence on the complexities of experience across diverse populations and ultimately have access to insights on what works, under what circumstances and for whom.

14:30 EST

15 min

Break

15:15 EST

45 min
Kim van der Woerd Sofia Vitalis

How Evaluation Practices Can Advance Truth and Reconciliation

Kim van der Woerd, Lead for Strategy and Relations, Reciprocal Consulting

Sofia Vitalis, Lead of Education and Mentorship and Partner , Reciprocal Consulting Inc

The importance of reconciliation cannot be overstated. There is an increasing focus on evaluating policies aimed at Indigenous economic development, health, education, and governance through a reconciliation lens.

  • How have past evaluation policies impacted policy?
  • Proceeding with principles of cultural responsiveness and cultural safety.
  • Examining your procedures and taking steps to practice active reconciliation.

16:00 EST

End of Day One

Day Two: Thursday, November 14, 2024

10:15 EST

45 min
Julie Gilbert

Understanding the Health Impacts of Policies & Actions

Julie Gilbert, Julie Gilbert Consulting

Evaluating health initiatives is crucial to determining their effectiveness, efficiency, and impact on public health. The evaluation process generally involves several key steps and considerations:

  • Define objectives and scope.
  • Develop Logic Model or Theory of Change.
  • Establish evaluation criteria and indicators.
  • Selecting evaluation methods.
  • Data collection.
  • Data analysis.
  • Interpretation of Results.
  • Reporting and Dissemination.
  • Ethical Considerations.
  • Why Canada needs to adopt a Health in All Policies (HiAP) framework.
  • Addressing the social determinants of health and reducing health inequities.
  • Engaging the public and community.

11:00 EST

45 min
Chitra Sridhar

Artificial Intelligence and Policy Evaluation for Better Policy Outcomes

Chitra Sridhar, Leader, GTA, KPMG’s Data & Analytics Centre of Excellence, KPMG Canada

Artificial Intelligence is braced to have a dramatic and profound impact on policy making—the ability to sense patterns of need, develop evidence-based programs, forecast outcomes, and analyze effectiveness. AI could advance the evaluation stage of the policy cycle by providing faster and more accurate data that can assess the impact of policies.

  • Leveraging digital tools to collect and analyze data for evaluating policies in real-time or near real-time.
  • AI and the policymaking cycle.
  • Using AI and technology in evaluation practices.
  • AI and evaluation.
  • AI and the Government of the Future.

11:45 EST

45 min
Pushpita Saha

The Rights Based Approach to Policy Development

Pushpita Saha, Program Monitoring Evaluation and Learning Officer, Oxfam

A rights-based approach ensures that policy evaluations not only measure effectiveness in terms of outcomes but also assess the policy’s adherence to human rights standards, promoting a more just and equitable society. A human rights-based approach causes policy makers to think about marginalized individuals up front, integrating human rights principles and standards into the evaluation process. Key elements and steps include:

  • The human rights framework.
  • Identification of rights holders and responsible entities.
  • Define objectives from a rights perspective.
  • Human rights goals and impacts on rights.
  • Participatory evaluation.
  • Select human rights indicators and relevant disaggregation of data.
  • Data collection and analysis.
  • Evaluate impact on human rights.

12:30 EST

60 min

Break

13:30 EST

45 min
Neil Bouwer

Horizontal Review of Innovation and Clean Tech Programs in the Government of Canada

Neil Bouwer, Visiting Professor of Practice, Max Bell School of Public Policy

Departments often share program objectives, but are seldom evaluated together. Such evaluations can be useful to assess the effectiveness of a suite of programs against Government priorities. This session will discuss the horizontal evaluation of innovation and clean tech programs in the federal government, including:

  • Strategic policy context.
  • Leadership approach.
  • Management dynamics.
  • Methodologies.
  • Findings – Best Practices in Federal Business Innovation Program Delivery.
  • Lessons Learned.

14:15 EST

15 min

Break

14:30 EST

45 min
Leslie A. Fierro, PhD, MPH

Strengthening Evaluation Capacity to Improve Policies, Programs, and Initiatives

Leslie A. Fierro, PhD, MPH, Sydney Duder Professor of Program Evaluation, Max Bell School of Public Policy, McGill University

Strengthening organizations’ capacity to commission, conduct, and use high-quality evaluations requires intentional action on the part of evaluators. In this session, attendees will be introduced to:

  • A process that can be used to create a strategy for strengthening existing evaluation capacity in organizations.
  • Key outcomes of evaluation capacity strategies.
  • Common approaches for measuring and enhancing evaluation capacity.
  • Attendees will also learn about potential ways to continue enhancing their own competencies to support evaluation capacity efforts.

15:15 EST

45 min
Dr. Nelson Jatel

Evaluating Policy by Network Metrics

Dr. Nelson Jatel

Learn how network metrics can provide innovative, systems-based approaches to understanding and improving policy-making processes. Topics will include:

  • Network metrics as tools to map relationships between stakeholders, organizations, and institutions involved in policy-making.
  • Common metrics revealing insights into influence, communication patterns and power dynamics within networks.
  • Case studies applying network metrics in policy context.
  • How network analysis can offer evidence-based insights that lead to more effective policy development
  • Challenges and opportunities
  • Future of network metrics in policy evaluation:

16:00 EST

End of Day Two