Overstay2 colour

The Overstay2 colour is a colour assigned to a patient based on data collected as part of Project Overstay2, which is run through the Overstay2 scoring models to generate a score and the Overstay2 colour. The colour modifies the Overstay2 processes on the units to reduce discharge delay. See Overstay2 Overview for more context.

The colour is based on a probability threshold that is set to balance the number of patients followed under the more elaborate Overstay2 processes on the units to reduce discharge delay#red process wile minimizing the number of patients who would overstay by more than 10 days that are not captured. It was set separately for each of the Overstay2 scoring models, so see those pages for the actual thresholds.

Method of setting threshold

Optimal threshold

• A 2 x 2 classification table for incrementing probability levels from 0.001 to 0.200 helps visualize the performance of the predicted results with actual outcomes.
• Sensitivity (true positive rate) is the proportion of actual positives correctly identified by the predicted results. High sensitivity means few false negatives.
• Specificity (true negative rate) is the proportion of actual negatives correctly identified by the predicted results. High specificity means few false positives.
• Positive Predicted Value (PPV) is the likelihood that a positive result is a true positive.
• Negative Predicted Value (NPV) is the likelihood that a negative result is a true negative.
• False Positive (FP) Rate is the proportion of actual negatives but predicted incorrectly as positives.
• False Negative (FN) Rate is the proportion of actual positives but predicted incorrectly as negatives.
• Optimal threshold is the probability level which shows equal or close to equal sensitivity or specificity in the testing data set. This threshold (also called cut-off) is being used to classify patients.
• The optimal threshold by site is below:

Site	Data Set	Optimal Cut-off	Sensitivity	Specificity	PPV	NPV	FP Rate	FN Rate
GGH	Training	0.076	74.2	74.3	16.1	97.7	25.7	25.8
GGH	Validation	0.079	74.4	74.5	16.4	97.7	25.5	25.6
HSC	Training
HSC	Validation
STB	Training
STB	Validation

There was a concern that this would have overwhelmed the Overstay2 processes on the units to reduce discharge delay so we considered a #pragmatic threshold.

Pragmatic threshold

We considered whether we should change the threshold from the #Optimal threshold to reduce the number of patients who are assigned as red, in addition to modifying the Overstay2 processes on the units to reduce discharge delay .

Site	Data Set	Pragmatic Cut-off	Sensitivity	Specificity	PPV	NPV	FP Rate	FN Rate
GGH	Training	0.094	67.2	79.0	17.5	97.3	21.0	32.8
HSC	Training
STB	Training

I think this can go, just leaving it until confirmed

We used the following process to set a red/yellow threshold that our Overstay2 processes on the units to reduce discharge delay can function with. Goals/considerations The thresholds were set to stratify patients based on the following considerations: goal: reduce LOS 10-15% no more than 15-17% of patients tagged Use Positive Predictive Value (PPV) which denotes out of the patients tagged, how many are true positives. capture 60-75% of all delayed discharges Use Sensitivity or True Positive Rate which denotes out of delayed discharges, how many are correctly identify as delayed. {

Log

• 2025-06-13 - continuous model no better, confirmed using threshold of 0.077 for now
• 2025-06-11 - meeting with DR and JM to discuss, considered continuous rather than binary model, Julie will generate
• 2025-06-08 - email to determine how to set colour in that new context
• 2025-05-27 - Julie finalized an updated Overstay2 scoring model for Grace Hospital
• 2025-02-24 - we realized we needed separate Overstay2 scoring models for each site
• 2025-02-21 - 2025-02-21 JM DR Overstay model decision and cut-off - We will use Model 8 at cut-off of 0.069
• 2025-02-20 - decided that we want
  • goal: reduce LOS 10-15%
  • no more than 15-17% of patients tagged
  • capture 60-75% of all overstayers