Learner Responsive success rate methodology 2008/09 - FAQs

Please note: this information is only relevant to learning providers receiving LSC Learner Responsive Funding

1. Why is the calculation method of Learner Responsive success rates being changed for 2008/09?
2. What are the benefits of changing the calculation methodology?
3. How will it affect me as a provider?
4. How will performance be monitored over time?
5. How are Qualification Success Rates (QSRs) calculated?
6. Why can't we run the 2007/08 methodology and the new 2008/09 methodology side-by-side for the first year of change so we can have comparible data?
7. What are the main principles and changes to the 2008/09 Learner Responsive success rate methodology?
8. Where can I get access to the published definitions of success rate methodologies once they become available?
9. How does the LSC know what the consequences of the introduction of the new methodology will be?
10. Why are these changes being communicated now?
11. Will the current methodology apply for 2009/10?

1. Why is the calculation method of Learner Responsive success rates being changed for 2008/09?

One of the key drivers for the change in the calculation of success rates arises from the introduction of demand-led funding in 2008/09. A key element of the changes, which has an impact on success rates, is the definition of a start date for learners for funding purposes.

For 2008/09, this means a change from the traditional approach based on specified census dates, to one based on the amount of time a learner must remain on programme (six weeks for programmes of at least 24 weeks, and lower periods for shorter courses).
Back to top

2. What are the benefits of changing the calculation methodology?
The proposed changes to align success rates with the new start definitions will ensure consistency within and between funding models (for Employer Responsive and Learner Responsive), and are consistent with the original policy of New Measures for Success and demand-led funding. This will simplify the process, ensuring consistency between the definition of learner ‘start’ dates used for funding, and those used for assessing quality of provision.
Back to top

3. How will it affect me as a provider?

Success rates: The expectation is that the success rate results in 2-08/09 will be broadly comparable with previous years, taking all factors into account. The Data Harmonisation Group expects the quality of provision to continue to increase as per the trend over previous years, and it also expects some change from the introduction of demand-led funding. However, the full impact of any changes nationally and locally, at provider and subject area level, won’t exactly be known until the 2008/09 data has been analysed following submission of full-year data (F05).
National level success rate figures: These are published to the sector through the Post-16 Education Statistical First Release (SFR). This publication follows Office of National Statistics protocols, which advises that historical trend information be provided where this is possible. Where a methodology changes mid-time series, the SFR will note the change both in the SFR and in the accompanying definitions document. This process will be followed when the success rate figures are published.
QSR Performance Indicator in the Framework for Excellence: The method of calculating the QSR Performance Indicator will not change. However, changes to the 2008/09 success rates, i.e. the source data, may lead to possible changes to the grid and assessment criteria.

Funding Allocations and Minimum Levels of Performance (MLP):

Back to top

4. How will performance be monitored over time?

Careful consideration is currently being given to producing detailed guidance for Ofsted inspectors and others, so that they take the change in methodology into account when assessing performance. This will assist with the process of monitoring performance over time and ensure that informed judgements are made on a ‘like-for-like’ basis.
Back to top

5. How are Qualification Success Rates (QSRs) calculated?
Success rates are based upon learning aim cohorts – the cohort is derived from the expected end year. Success rates are calculated at the end of 2008/09, and are based on the year in which learning aims are expected to be achieved.

Download our diagram showing how Learner Responsive QSRs will be calculated at the end of 2008/09.

Back to top

6. Why can’t we run the 2007/08 methodology and the new 2008/09 methodology side-by-side for the first year of change so we can have comparable data?
It is recommended that success rates for 2008/09 below national level, for example provider level, are calculated and published using only one methodology - the new one. This is recommended because to generate and publish success rates in both ways would lead to confusion for learners, providers and agencies when reporting trends and performance.
Back to top

7. What are the main principles and changes to the 2008/09 Learner Responsive success rate methodology?

Changes to ensure that the 2008/09 methodology reflects demand-led funding:

For the 2008/09 academic year only within the 2008/09 success rate methodology, remove the use of the Census reference date (1st November) to determine the inclusion/ exclusion of learners and replace with the appropriate length of time that a learner needs to be enrolled on a learning aim to receive funding (6 weeks/ 2 weeks, depending on course length).

Report on the Learner Responsive provision as it is defined i.e. without the inclusion of FE NVQs, which have transferred to Employer Responsive provision.

Exclusion of Entry to Employment (E2E) provision (which becomes part of Learner Responsive from 2008/09) as E2E is not a programme aimed at delivering whole qualifications.

Changes to ensure that success rates are more focussed on LSC-funded provision (simplification of the process) and that there is more comparability between Learner Responsive and Employer Responsive success rates:

Include LSC-funded learning aims only – removing the inclusion of non-funded learning aims where there is at least one LSC-funded learning aim in a cohort. (Note: The Learning Aims Database (LAD) provides a complete and detailed record of all qualifications and learning aims. )

Exclude all learners below the age of 16 from the success rate calculation as they are not funded by the LSC.

Continuation of current principles of the methodology:

Exclusion of provision flagged as Train to Gain from the 2006/07 ILR data to avoid double counting.

Exclusion of provision classified as Qualification Credit Framework units as the methodology only measures ‘whole’ qualifications.

Exclusion of provision relating to the National Vocational Training Pilot (NVTP) and other pilots leading to job outcomes.

Exclusion of Programme-Led Pathway provision at overall programme aim level, but include component qualifications where they are part of the learners’ programmes (this will be reviewed for 2009/10).

Individual qualifications that are done as components of Diplomas will be calculated within the QSR, but Diplomas will not be calculated at programme aim level (this will be reviewed for 2009/10).

Back to top

8. Where can I get access to the published definitions of success rate methodologies once they become available?
Published definitions, once available, are published on the Data Service website, in the Data Dictionary.
Back to top

9. How does the LSC know what the consequences of the introduction of the new methodology will be?
We are trying to give the sector advance notice of the changes, so that preparation is made for the changes that may occur. The alignment of these data definitions for success and quality is an indication of an overall ‘direction of travel’ for all the organisations involved - including the LSC, BIS, DCSF and Ofsted - towards greater simplicity and transparency.

Modelling of the likely impact has been done based on 2007/08 data but, as previously stated, the full impact of the change locally, at provider and subject area level, won’t be known until the 2008/09 data has been analysed following submission of full-year data.
Back to top

10. Why are these changes being communicated now?
The changes are being communicated now to advise the further education and training system of the recently approved methodology. The move to demand-led funding has been on the way for two years and the methodology for the calculation of success rates has now been approved to reflect demand-led funding.

There has been ongoing work over several years to align the definition of a start in the calculations of funding and success rates, and between different funding streams. This was a key element in the work begun through New Measures of Success in 2003/04, and more recently in the development of the demand-led funding system.

The proposed approach was set out in the second consultation on the funding proposals in June 2006. That document gave the proposed new definition of a start date and confirmed that the intention was that this would be used both for funding and success rates. The new definition was introduced for funding and data collection purposes in 2008/09 and the calculation of success rates is following along the same lines.

The proposed changes to align success rates with the new start definition are therefore essential to produce consistency in the system, both within and between funding models, and are consistent with the original policy intent of New Measures for Success and demand-led funding.
Back to top

11. Will the current methodology apply for 2009/10?
The changes that have been made for 2008/09 should continue into 2009/10, and we will endeavour to minimise further changes. However, the 2008/09 methodology will need to be reviewed to ensure that it is still appropriate, accurate and reflective of the 2009/10 Learner Responsive provision.
Back to top