Measurement Demystified

2023 Webinar Series

Measurement and Reporting for L&D—An In-Depth Examination of the Most Important Topics

Join us for this expanded, nine-part online series to learn the latest thinking and techniques for measurement and reporting from one of the profession’s leading authorities. Each session will focus on one or two key topics and build on the previous sessions. Choose individual sessions to suit your interest or join us for all eight to provide a comprehensive overview of the topic.

The nine-part series began in January 2023 and concludes in September 2023. In October, the nine-part series will start again, running through June 2024. You can start at any point in the series. Scroll down to find the session that is up next.

Each online session will be 60 minutes, including questions. The sessions will also be recorded so you can listen at your convenience. Each session will begin with a brief overview of the framework and end with steps you can take to implement the key concepts. Every effort will be made to answer questions from the participants in real time. A certificate of completion will be awarded to those who attend all eight sessions.

Dr. David Vance, Executive Director of the Center for Talent Reporting and former Chief Learning Officer for Caterpillar, will conduct the series. Dave and his team at Caterpillar won numerous awards for measurement, and Caterpillar University was ranked #1 in the 2005 ATD BEST Awards.  Dave was selected as 2006 CLO of the Year by Chief Learning Officer magazine. He recently published the second edition of his textbook, The Business of Learning, and he is the co-author with Peggy Parskey of Introduction to Measurement, Analytics and Reporting to be published by ATD in October. He teaches human capital management in two Ph.D. programs as well as the executive education program at George Mason University. Dave speaks at ATD ICE each year and is a frequent speaker at other events as well. The Center for Talent Reporting is a nonprofit organization dedicated to improving the measurement, reporting and management of human capital through publications, speaking, workshops and an annual conference.

All webinars begin at 11 am Central Time ( Noon Eastern, 10 am Mountain, 9 am Pacific)

All webinar recordings and PowerPoints will be available to CTR members within 48 hours at the Members Resource Center on the website. Recordings and PowerPoints will be e-mailed within 48 hours to everyone who registered for the webinar.

2023 and 2024 Webinars:

Introduction to Measurement, Reporting and the Talent Development Reporting Framework: Session 1

October 11, 2023 | 11 am to Noon CT

In the first session, we will begin by addressing the four overarching reasons for measuring which will provide the foundation for all seven sessions. These four reasons (inform, monitor, evaluate, and manage) will be captured in a measurement hierarchy. We will also define measurement and analytics and explore the differences between the two. Next, we will share the Talent Development Reporting Principles (TDRp) framework for organizing the 170+ measures we have in learning. This framework provides a common language and classification scheme not only for the learning profession but for all HR processes. We will then introduce you to the four general types of reports used in the profession and the three specific TDRp management reports. Last, we will briefly explore the new SEC requirements and the ISO standards for human capital reporting.

Focus on Efficiency Measures: Session 2

November 15, 2023 | 11 am to Noon CT

In the second session, we will explore efficiency measures in detail. Efficiency measures represent the largest category of measures in learning and include measures such as the number of participants, costs, completion rates and dates, utilization rates, and reach. If a measure’s name begins with “number of” it is usually an efficiency measure. More than 100 efficiency measures will be shared, including those for informal as well as formal learning. We will also share which efficiency measures are recommended by ISO and which are benchmarked by ATD.

Focus on Effectiveness and Outcome Measures: Session 3

December 6, 2023 | 11 am to Noon CT

In the third session we will explore the other two categories of measures in detail (effectiveness and outcome). In learning these measures are known as the four levels of Kirkpatrick and the five levels of Phillips. We will explore the difference between the two approaches for level 4 in detail, giving special attention to Phillips’ level 4 impact which will double as an outcome measure. Special attention will be paid to the issue of isolating the impact of learning from other factors, which continues to be one of the most contentious in the field. We will also share which effectiveness measures are commonly benchmarked.

Selecting Your Measures: Session 4

April 12, 2023 | 11 am to Noon CT

In the fourth session, we will provide detailed guidance on how to use the TDRp framework to select the measures that are right for you, which is typically the most difficult part of creating a measurement strategy. We will begin by examining measures for individual programs that are aligned to goals and needs as well as courses that are not aligned. Next, we will explore initiatives designed to improve processes and systems, including informal learning. As well as initiatives to improve efficiency or effectiveness measures across all programs. Last, we will explore measures to aggregate across programs and initiatives.

Feb 14, 2023  | 11 am to Noon CT

In the fifth session, we will describe the different types of reports in detail and explore how measures are used in scorecards, dashboards, and reports. We will revisit the purpose of the measures discussion from the first session and share detailed recommendations about which reports are appropriate depending on the purpose of the measures.) We will also explore the concept of running like a business and its implications for reporting. Examples will also be provided in all cases.

June 14, 2023 | 11 am to Noon CT

In the sixth session, we will explore the three types of TDRp reports (program, operations, and summary) in detail. These three reports are recommended to manage learning and are an essential part of running learning like a business. The program report is used to manage an individual program to meet a specific company goal or need. The operations report is used to manage improvement in efficiency and effectiveness measures across all programs. It is also used to manage department initiatives to improve processes and systems. The summary report is intended for senior leaders like the CEO, CFO, and governing board. It is a high-level summary demonstrating alignment, impact, and improvements in efficiency and effectiveness.

July 19, 2023 | 11 am to Noon CT

In the seventh session, we will explore where the numbers come from that are used
in the TDRp management reports. Our focus in this session will be on creating
plan numbers, which form the basis for running learning like a business and are
the most important numbers in any management report. This session will be more
technical than the others but will provide excellent insight into creation and
use of the reports.

August 9, 2023 | 11 am to Noon CT

In the eighth session, we continue to explore where the numbers come from that are used in the TDRp management reports. Our focus this month will be on reporting and using year-to-date results, including how to estimate them when necessary. Our second focus will be on creating forecasts, which are another distinguishing feature of management reports. Like last month’s session, this session will be more technical than the others but will provide excellent insight into creation and use of the reports. Statistical significance for effectiveness measures will be addressed as will the use of seasonally adjusted data in forecasting.

September 13, 2023 | 11 am to Noon CT

The ninth and final session is the culmination of your journey. We will provide guidance on how to use all that you have learned in the first eight sessions to create a robust measurement and reporting strategy. We will share the recommended steps and elements that you should consider, including the creation of measures library. We will conclude with thoughts on roles and responsibilities as well as change management considerations.

September 6, 2023 | 11 am to Noon CT

The US Securities and Exchange Commission adopted new rules in August 2020 mandating public disclosure of material key human capital metrics and initiatives for all publicly traded companies in the US. These rules became effective November 9 with no grace period so companies must begin to disclose material human capital information with their next quarterly or annual release.

Given the tight labor market, other organizations will eventually be forced to follow. The recently released standards for human capital reporting by ISO will be used by many as a starting point in deciding what to disclose and as a defense against a shareholder lawsuit for failure to disclose material human capital information.

Join us for this webinar to learn more about both the SEC’s new rules and ISO’s recommendations. Particular attention will be given to the eight ISO recommended measures for learning and development, including the proposed definitions. The webinar will conclude with a recommended strategy to address
the new requirements and a list of starter measures for your organization’s consideration.

May 10, 2023 | 11 am to Noon CT

The US Securities and Exchange Commission adopted new rules in August 2020 mandating public disclosure of material key human capital metrics and initiatives for all publicly traded companies in the US. These rules became effective November 9 with no grace period so companies must begin to disclose material human capital information with their next quarterly or annual release.

Given the tight labor market, other organizations will eventually be forced to follow. The recently released standards for human capital reporting by ISO will be used by many as a starting point in deciding what to disclose and as a defense against a shareholder lawsuit for failure to disclose material human capital information.

Join us for this webinar to learn more about both the SEC’s new rules and ISO’s recommendations. Particular attention will be given to the eight ISO recommended measures for learning and development, including the proposed definitions. The webinar will conclude with a recommended strategy to address
the new requirements and a list of starter measures for your organization’s consideration.