Pearson is the world’s leading company and it wants to be the heart of the debate about the transformation of education systems to improving outcomes, about preparing more people better for the 21st century economy.
The fundamental purpose of Pearson
is to help people make progress in their lives through learning. Therefore, a measure or a way of tracking the progress which is actually made about the central purpose of whatever products/services people use, is what is most required. Pearson is now shifting from inputs to the educational process to how it can help deliver results. It wants to measure gains in learning, and not just the financial returns. Keeping this in mind, Pearson has created an Efficacy Framework’, which is a set of tools for assessing if the conditions are in place for an education program to deliver its promised learning outcomes, which it plans to share publically.
The Efficacy Framework has been in the making since at least 2 years. It has already been used by Pearson internally to review almost 100 Pearson programs and since the beginning of 2013 is requiring the efficacy review for any product of more than $3 million or any proposed acquisition. Pearson aims to publish the details of its Efficacy Framework, so that the Efficacy Framework is owned and shared widely and can also be shaped by other practitioners. The essence of the framework is to analyze the data sets produced by the activity, have interactions with stakeholders and see whether there are clearly defined outcomes and a path to achieve those outcomes. This is not just a research-based approach, but develops as a systematic approach. The use of Efficacy Framework, is not to cancel projects, but change the development of products which are part of projects and thereby increase the data analytics in products.
The Efficacy review process is a consultative, collaborative way of working with teams to understand what elements of efficacy they are delivering today to learners and what they can do to improve that to really deliver the outcomes that people are coming to Pearson to achieve. This process takes place in two stages; first stage is to examine the current state of the product or service, by examining different documents relevant to it. Second stage of the review process is getting the project team to work on that product or service to establish where the current state of the product or service stands relevant to the Efficacy Framework and rating it on each part of the framework.
The efficacy framework has a basic design. It has four sections, in which you will need to consider three areas that will contribute to a product’s or service’s efficacy, and then answer questions about different factors. The questions have to be answered by assigning a color rating. The different color ratings will be combined for each section, and across the whole review, to give you an overall rating and some suggestions to support you to deliver learning outcomes in the future. To start with the review process, you need to have access to information about your product including; data on outcomes, education evidence used to create the product, feedback from users, business or implementation plans and about 40 minutes of your time. Efficacy framework asks four key questions:
What outcomes are we trying to achieve?
In order to have a measurable impact on learning outcomes, you need to be clear about what you are aiming to achieve. Your outcome should describe the exact skills, behavior and mindset your product or service aims to improve. It should also describe who exactly will benefit and how ambitious it is. This step requires you to reflect about; the intended outcomes- identification, measure, deadlines and documentation of the outcomes. The overall design- effective product design, feedback of others, usability for others. Value for money- benefits of your product or service to your target group, cost competitiveness.
What evidence do we have to achieve these outcomes?
A well designed solution requires evidence, or proof that you can achieve the intended outcomes. Your evidence should be high quality, comprehensive and consistently applied to improve your solution design. You need to reflect about; the comprehensiveness of evidence- methods employed to collect it, if it was collected for all stages of your product design, evidence of all users. Quality of evidence- if evidence is unbiased, applicable to your product or service, recent and a measure of success. Application of evidence- stored and accessible to people and analyzed.
What plans and governance systems do we have to achieve these outcomes?
Achieving your intended outcomes takes dedicated work and careful planning. This is why the framework prompts you to think about what plans, governance and systems are in place to deliver them. Things to be considered in this step are; Action plan- long and short-term priorities, risks and updation. Governance- who is responsible for decision-making and consultants. Monitoring and reporting- communication with stakeholders, real-time user feedback, identifying issues and tools to monitor progress.
What capacities do we have to deliver these outcomes?
In order to achieve your intended outcomes, every person involved in the project will need to have the capabilities, relationships and desire to drive results. This section prompts you to think about whether you have the resources (time/people/budget etc.) and abilities you need to deliver your intended outcomes. Things to be considered are; Internal capacity and culture- if you have the right number of people with right skillsets, focused culture, budget, leaders for support. User capacity and culture- measures to build user skills.
When you complete the online tool you will receive a report including recommendations on how you might increase your impact on learner outcomes, which you can download and share with others via email. You may also revisit the tool in future to check your scores again and keep track of your progress.
Efficacy framework takes the broad aspiration of making sure every product or service has an impact on people’s lives and thinks about making it practical and actionable so that any team, person or individual can really apply their thinking on how they work on a day-to-day basis. Be a part of this wonderful global initiative to improve learning outcomes. Share your views with us. Make your comments in the Comment Box.
