Day One: Wednesday, November 13, 2024
10:00 EST
15 minWelcome and Opening Remarks from the Chair
10:15 EST
45 minFrom Theory to Practice: The Development, Implementation, and Impact of Evaluation Policy
- The journey from theoretical frameworks to practical application in evaluating policy.
- The key components of evaluation policies.
- How individuals and organizations translate these theories into actionable practices.
- Implementation challenges and strategies.
- Real-world examples of integrating evaluation policies into organizational processes.
- The broader impact of policies on decision-making, accountability, and policy improvement.
- How evaluation practice can influence future policy directions.
11:00 EST
45 minAnalytics and Policy Evaluation
Jérome Mercier, Director General, Evaluation, Employment and Social Development Canada
- The role of evaluation in the policy development ecosystem with a focus on its systemic analytical approach bringing together various quantitative and/or qualitative lines of evidence.
- An overview of commonly used lines of evidence along with high-level considerations about complementarity that are often considered when determining which ones to use.
- The distinction between the concepts of contribution and attribution in policy evaluations.
- The importance of neutrality, prudence, balanced reporting, and transparency when presenting evaluation evidence.
- Thoughts on the future of analytics and policy evaluation.
11:45 EST
45 minPolicy Evaluation in the Digital Age: New Methods and Approaches
- Data-driven policy evaluation.
- The role of big data and machine learning.
- Identification of policy outcomes and unintended consequences.
- Integration of administrative data such as health records, tax data, and employment information with traditional survey data for more comprehensive evaluations of public policies.
- The use of Gender-Based Analysis Plus (GBA Plus) as a critical component in evaluating how policies affect various groups.
- How digital tools help to disaggregate data along lines of gender, age, ethnicity, and other demographic factors, ensuring that policies are inclusive and equitable.
- How new digital tools allow for real-time or near-real-time evaluation of policy initiatives.
12:30 EST
40 minBreak
13:10 EST
40 minUsing Robust Intersectional Gender-based Analysis Plus (GBA Plus) in Policy and Program Evaluation
The Government of Canada is increasingly applying GBA Plus in the design and implementation of federal initiatives to help ensure more inclusive impacts. That’s because GBA Plus leads to more sophisticated understanding of diverse people and the barriers they face by guiding analysis of discriminatory norms, practices, systems, and structures that create/perpetuate inequalities. It helps provide a clearer view of how government actions affect – and are affected by – the context in which they are implemented. Robust and intersectional GBA Plus helps ensure that the evaluation of policies and programs are:
- Inclusive.
- Challenge assumptions about power and agency.
- Sensitive to context and to the intersecting factors that create unique experiences of inequality.
By sustaining its commitment to GBA Plus and to its continuous improvement for close to 30 years, the Government of Canada has strengthened its capacity to generate meaningful evidence on the complexities of experience across diverse populations and ultimately have access to insights on what works, under what circumstances and for whom.
13:50 EST
40 minEmbedding a Gender and Diversity Lens into the Policy Design, Delivery and Evaluation Processes
- How EDI has advanced for specific policies and programs.
- Recruitment.
- Retention.
- Call to action.
- Response to BLM OAG Report.
- EDI Audits – internal and external facing programs looking at outcomes.
- Immigrant and refugee issues.
14:30 EST
15 minBreak
14:45 EST
45 minEvaluating Policy Engagement
15:15 EST
45 minHow Evaluation Practices Can Advance Truth and Reconciliation
Kim van der Woerd, Lead for Strategy and Relations, Reciprocal Consulting
Sofia Vitalis, Lead of Education and Mentorship and Partner , Reciprocal Consulting Inc
The importance of reconciliation cannot be overstated. There is an increasing focus on evaluating policies aimed at Indigenous economic development, health, education, and governance through a reconciliation lens.
- How have past evaluation policies impacted policy?
- Proceeding with principles of cultural responsiveness and cultural safety.
- Examining your procedures and taking steps to practice active reconciliation.
16:00 EST
End of Day One
Day Two: Thursday, November 14, 2024
10:00 EST
15 minWelcome and Opening Remarks from the Chair
10:15 EST
45 minUnderstanding the Health Impacts of Policies & Actions
Evaluating health initiatives is crucial to determining their effectiveness, efficiency, and impact on public health. The evaluation process generally involves several key steps and considerations:
- Define objectives and scope.
- Develop Logic Model or Theory of Change.
- Establish evaluation criteria and indicators.
- Selecting evaluation methods.
- Data collection.
- Data analysis.
- Interpretation of Results.
- Reporting and Dissemination.
- Ethical Considerations.
- Why Canada needs to adopt a Health in All Policies (HiAP) framework.
- Addressing the social determinants of health and reducing health inequities.
- Engaging the public and community.
11:00 EST
45 minArtificial Intelligence and Policy Evaluation for Better Policy Outcomes
Chitra Sridhar, Leader, GTA, KPMG’s Data & Analytics Centre of Excellence, KPMG Canada
Artificial Intelligence is braced to have a dramatic and profound impact on policy making—the ability to sense patterns of need, develop evidence-based programs, forecast outcomes, and analyze effectiveness. AI could advance the evaluation stage of the policy cycle by providing faster and more accurate data that can assess the impact of policies.
- Leveraging digital tools to collect and analyze data for evaluating policies in real-time or near real-time.
- AI and the policymaking cycle.
- Using AI and technology in evaluation practices.
- AI and evaluation.
- AI and the Government of the Future.
11:45 EST
45 minThe Rights Based Approach to Policy Development
Pushpita Saha, Program Monitoring Evaluation and Learning Officer, Oxfam
A rights-based approach ensures that policy evaluations not only measure effectiveness in terms of outcomes but also assess the policy’s adherence to human rights standards, promoting a more just and equitable society. A human rights-based approach causes policy makers to think about marginalized individuals up front, integrating human rights principles and standards into the evaluation process. Key elements and steps include:
- The human rights framework.
- Identification of rights holders and responsible entities.
- Define objectives from a rights perspective.
- Human rights goals and impacts on rights.
- Participatory evaluation.
- Select human rights indicators and relevant disaggregation of data.
- Data collection and analysis.
- Evaluate impact on human rights.
12:30 EST
60 minBreak
13:30 EST
45 minHorizontal Review of Innovation and Clean Tech Programs in the Government of Canada
Neil Bouwer, Visiting Professor of Practice, Max Bell School of Public Policy
Departments often share program objectives, but are seldom evaluated together. Such evaluations can be useful to assess the effectiveness of a suite of programs against Government priorities. This session will discuss the horizontal evaluation of innovation and clean tech programs in the federal government, including:
- Strategic policy context.
- Leadership approach.
- Management dynamics.
- Methodologies.
- Findings – Best Practices in Federal Business Innovation Program Delivery.
- Lessons Learned.
14:15 EST
15 minBreak
14:30 EST
45 minStrengthening Evaluation Capacity to Improve Policies, Programs, and Initiatives
Strengthening organizations’ capacity to commission, conduct, and use high-quality evaluations requires intentional action on the part of evaluators. In this session, attendees will be introduced to:
- A process that can be used to create a strategy for strengthening existing evaluation capacity in organizations.
- Key outcomes of evaluation capacity strategies.
- Common approaches for measuring and enhancing evaluation capacity.
- Attendees will also learn about potential ways to continue enhancing their own competencies to support evaluation capacity efforts.
15:15 EST
45 minEvaluating Policy by Network Metrics
Learn how network metrics can provide innovative, systems-based approaches to understanding and improving policy-making processes. Topics will include:
- Network metrics as tools to map relationships between stakeholders, organizations, and institutions involved in policy-making.
- Common metrics revealing insights into influence, communication patterns and power dynamics within networks.
- Case studies applying network metrics in policy context.
- How network analysis can offer evidence-based insights that lead to more effective policy development
- Challenges and opportunities
- Future of network metrics in policy evaluation:
16:00 EST
End of Day Two