Overstay2 colour

The Overstay2 colour is a colour assigned to a patient based on data collected as part of Project Overstay2, which is run through the Overstay2 scoring models to generate a score and the Overstay2 colour. The colour modifies the Overstay2 processes on the units to reduce discharge delay. See Overstay2 Overview for more context.

The colour is based on a probability threshold that is set to balance the number of patients followed under the more elaborate Overstay2 processes on the units to reduce discharge delay#red process wile minimizing the number of patients who would overstay by more than 10 days that are not captured. It was set separately for each of the Overstay2 scoring models, so see those pages for the actual thresholds.

Method of setting threshold

Optimal threshold

• A 2 x 2 classification table for incrementing probability levels from 0.001 to 0.200 helps visualize the performance of the predicted results with actual outcomes.
• Sensitivity (true positive rate) is the proportion of actual positives correctly identified by the predicted results. High sensitivity means few false negatives.
• Specificity (true negative rate) is the proportion of actual negatives correctly identified by the predicted results. High specificity means few false positives.
• Optimal threshold is the probability level which shows equal or close to equal sensitivity or specificity in the testing data set. This threshold (also called cut-off) is being used to classify patients.
• The optimal threshold by site is below:

Site	Data Set	Optimal Cut-off	Sensitivity	Specificity	PPV	NPV
GGH	Training	0.077	73.60	74.50	16.1	97.7
GGH	Validation	0.080	74.66	74.69	16.8	97.7
HSC	Training
HSC	Validation
STB	Training
STB	Validation

This would have resulted in a assigning xxx% of patients a "red". This would have overwhelmed the Overstay2 processes on the units to reduce discharge delay so we needed to determine a #pragmatic threshold.

Pragmatic threshold

We used the following process to set a red/yellow threshold that our Overstay2 processes on the units to reduce discharge delay can function with.

Goals/considerations

The thresholds were set to stratify patients based on the following considerations:

• goal: reduce LOS 10-15%
• no more than 15-17% of patients tagged
  • Use Positive Predictive Value (PPV) which denotes out of the patients tagged, how many are true positives.
• capture 60-75% of all delayed discharges
  • Use Sensitivity or True Positive Rate which denotes out of delayed discharges, how many are correctly identify as delayed.

Site	Data Set	Treshold Cut-off	Sensitivity	Specificity	PPV	NPV
GGH	Training	0.114	60.3	83.5	19.5	96.9
GGH	Training	0.076	75.3	74.0	16.2	97.8
GGH	Training	0.061	82.2	69.1	15.0	98.3
GGH	Training	0.090	69.4	77.4	17.0	97.2
GGH	Validation	0.110	60.5	82.3	18.9	96.7
GGH	Validation	0.078	75.2	74.3	16.7	97.8
GGH	Validation	0.060	82.3	68.5	15.2	98.3
GGH	Validation	0.084	73.0	75.6	17.0	97.6
HSC	Training
HSC	Validation
STB	Training
STB	Validation

How did those get set, though? If they are arbitrary then any rigor in balancing them is just dazzle. Ttenbergen 19:48, 8 June 2025 (CDT)

• SMW

• Cargo

• Categories

Setting threshold based on these goals

Emailed 2025-06-08: we need to understand the mechanism we used to set the threshold. To be updated when clear. Ttenbergen 17:57, 8 June 2025 (CDT)

• SMW

• Cargo

• Categories

Log

• 2025-06-08 - email to determine how to set colour in that new context
• 2025-05-27 - Julie finalized an updated Overstay2 scoring model for Grace Hospital
• 2025-02-?? - we realized we needed separate Overstay2 scoring models for each site
• 2025-02-21 - 2025-02-21 JM DR Overstay model decision and cut-off - We will use Model 8 at cut-off of 0.069
• 2025-02-20 - decided that we want
  • goal: reduce LOS 10-15%
  • no more than 15-17% of patients tagged
  • capture 60-75% of all overstayers