Assessment of the train performance trajectories in Network Rail’s Route Strategic Plans for PR18 Mandate L4AR004: Phase 1 report 18 June 2018 [Issue for publication] Prepared jointly with Winder Phillips Associates
Contents Section 1. 3 Purpose of the mandate and Arup approach Section 2. Summary of Phase 1 findings 4 Section 3. 8 Phase 2 Section 4. Historical Performance 9 Appendix A 14 Assessment of Route Strategic Plans Appendix B Documents received 25 Train Performance Trajectories for PR18 18 June 2018 2 [PHASE 1 REPORT]
1. Purpose of the Mandate and Arup Approach This report Purpose of Mandate This report describes our findings from Phase 1 of the study, To advise on namely to gain early familiarisation of Route Strategic Plans • level of NR assurance to performance trajectories (RSPs) and assess NR internal assurance. Please refer to • whether performance targets are appropriate the Phase 2 report (Assessment of train performance trajectories in Network Rail’s Route Strategic Plans for PR18) for our main assessment, conclusions and Scope recommendations. What? Detail Level Pax/freight Comparable Route Measure Network Rail’s proposed Route Passenger Approach to Phase 1 (Network Rail caused delay minutes performance trajectory and plus TOC on TOC delay minutes)* assessment of the floor We have reviewed the December versions of the RSPs alongside the assurance reports produced by the National Freight targets (FDM, FDM-R) Network Rail’s proposed Route, FNPO Freight Performance Team. We have also held the following performance trajectory and assessment of the floor meetings. Note that we have not had time to assimilate NR/Customer agreed ‘top level’ Network Rail’s proposed Route, FNPO Passenger & findings from 31 Jan meetings in this report. measures (various) – in particular performance trajectories freight where these are not agreed between Network Rail and its TOC Date Purpose customers Cancellations Do scorecards or Network Rail’s Route, FNPO Passenger & 15 Jan To understand ORR views of performance plans plans provide sufficient freight protections against excessive cancellations 25 Jan To understand NR assurance by Business Review Team & Scotland Does Network Rail’s proposal Network Rail Passenger & National Performance Team meet the HLOS targets Route freight Network Rail’s assurance process How has Network Rail? Is the Network Rail Passenger and 26 Jan To understand planning & assurance in Wales output from this process freight robust? 31 Jan To understand planning & assurance in LNW 31 Jan To understand planning & assurance in LSE Train Performance Trajectories for PR18 18 June 2018 3 [PHASE 1 REPORT]
2. Summary of Phase 1 - Overview of Measures in Long-term scorecards FNPO Anglia LNE & Midlands LNW Scotland South East Western Wessex Wales Freight Delivery FDM - National Freight Delivery Freight Delivery FDM-R FDM FDM FDM-R FDM FDM-R Metric Metric (FDM) Metric (FDM-R) Right time Right Time departures Departures (freight) Caledonian Sleeper Right Time On Time at all Caledonian Sleeper Caledonian Right Time MAA Punctuality at all Right-time arrivals at GWR Right Time Right Time Metrics - Right time Arrival recorded stations Right Time Arrivals Sleeper Right (final destination recorded stops Reading [Cross Departures leaving Time Arrivals only) [GWR] Country] Wales Route at Severn Tunnel Junction On-Time Moving Right-time at Annual Average destination [HEX] Right-time departure at Bristol Parkway [Cross Country] Passenger Average Passenger Average passenger Average Passenger Lateness Lateness lateness Lateness PPM (Cross Public PPM MAA PPM Moving Annual PPM PPM [GWR] PPM PPM Country) Performance Average Public Measure (PPM) Performance Charter Trains - GWR - Amalgamated Measure (PPM) PPM PPM on North Downs and Portsmouth Cardiff Route CaSL (Cross Cancelled and Level of Cancellations Cancellations NR contribution to Level of CaSL Cancellations Country) Significantly CaSL MAA cancellations Late (CaSL) FOC on TOC delay NR caused Delay Infrastructure Delay DPI Reduction Delay minutes NR caused delay (Delay Minutes by the route (Track & Non-Track affecting TOC (NR minutes caused, TOC on TOC Minutes/100 train Assets) & FOC on TOC not Delay Metrics km) including TOC on self) Network Network Network Network performance Network performance - performance: performance: - passenger Performance passenger Passenger Passenger T3 Moving Annual %age improvmt Average (Euston- in average minute Other Watford Service per mile travelled Group) [Abellio ScotRail] Train Performance Trajectories for PR18 18 June 2018 4 [PHASE 1 REPORT]
2. Summary of Phase 1 Findings (1) Target Setting Historic Performance • CP6 targets are generally based on the CP5 outturn • 5 ‘common’ performance metrics reviewed (FDM, forecast Cancellations, PPM, On Time, APL) as shown in the following slides. • Trajectories are generally flat (or show only very • In general, CP6 trajectory start points (and end CP5 modest change), which we propose to investigate in more detail in Phase 2 forecasts) look sensible compared to recent performance • Major mismatch with DfT and TfL Franchise/Concession target setting process • The “Consistent Route Measure – Performance” (CRM-P) is excluded from this phase • Plan narrative often does not align with new scorecard • As agreed with NR and ORR, we have also not targets – they usually use PPM only. focused on other metrics which are included in • Evident from Route teleconferences that modelling specific individual Route plans has been done, but this has not been shared with us to date as RSP format does not require it • Also evident that very different modelling approaches Overview of Plans have been undertaken by the Routes. • The draft plans currently contain little analysis or data • It is unclear how some targets have been set: • The linkages between elements such as asset plans - Definition of central target varies (e.g. changed from and performance outputs are limited – an overview of P50 to P80 in LSE) delays caused by asset type would help - Definition and treatment of Above and Below target • There is little historical performance context to most threshold appear to differ between Routes of the plans • Risk is not quantified in most plans • Use of waterfall charts would have assisted (we understand that this will be included in the next issue) Train Performance Trajectories for PR18 18 June 2018 5 [PHASE 1 REPORT]
2. Summary of Phase 1 Findings (2) Risk Assurance • Risk is discussed in all plans, but given greater • The BRT checks have focused much wider than emphasis in some (e.g. Western) performance, looking at the plans in the round • Particular focus on risks from projects and new rolling • NPT checks described as qualitative rather than quantitative – no detailed challenge of numbers stock • Not clear that the benefits of these schemes are • Route checks are varied with some being more about included in all RSPs (Business Case may not be the process rather than independent scrutiny or reflected) challenge by independent party • Of the 3 Routes reviewed, only LSE appears to have Operator Agreement carried out structured assurance • Only Scotrail has a signed agreement to targets • Should there be internal independent scrutiny of • State of discussions unclear forecasts? • Little linkage with TOC plans seen to date • Impact on non lead TOCs not clear e.g. GTR impact Situational Analysis on LNE/EM in the SE plan • Only FNPO, Wessex & LNW contain any historical Cancellations analysis • Very little mention in plans • Good plans should contain or reference to a quality • Don’t appear to take benefits from TOC fleet plans review of historical delivery • Cannot verify that they will be balanced against other • Seeing how the plans link to this would assist ORR in measures understanding if the RSP will underpin the targets • It will also support applications for additional funding by demonstrating gaps (Appendix D schemes) Train Performance Trajectories for PR18 18 June 2018 6 [PHASE 1 REPORT]
Colour code for confidence rating 2. Summary of Phase 1 Findings (3) Reasonably high confidence Some confidence Low confidence Reasonably high confidence gap Insufficient information provided Preliminary views of RSPs FDM Based on reading the plans only • FNPO plan does highlight historical causes of poor performance • Target is flat through CP6 at similar levels to now • Linkage to geographic Route plans is not clear • Build up of actions to deliver target is not shown Lessons mentioned by the Routes • Clearer remit – metrics, report templates, tools • Better management from the centre of TOC expectations of performance targets • Additional analyst to make the modelling more granular (service groups / line of route) Train Performance Trajectories for PR18 18 June 2018 7 [PHASE 1 REPORT]
Recommend
More recommend
