Overstay2 colour: Difference between revisions
Ttenbergen (talk | contribs) No edit summary |
|||
| Line 5: | Line 5: | ||
== Method of setting threshold == | == Method of setting threshold == | ||
=== Optimal threshold === | === Optimal threshold === | ||
* A 2 x 2 classification table for varying probability levels helps visualize the performance of the predicted results with actual outcomes. | |||
* | * Sensitivity (true positive rate) is the proportion of actual positives correctly identified by the predicted results. High sensitivity means few false negatives. | ||
* Specificity (true negative rate) is the proportion of actual positives correctly identified by the predicted results. High specificity means few false positives. | |||
* Optimal threshold is the probability level which shows equal or close to equal sensitivity or specificity in the testing data set. This threshold (also called cut-off) is being used to classify patients. | |||
This would have resulted in a assigning | This would have resulted in a assigning | ||
'''''xxx% ''''' | '''''xxx% ''''' | ||
of patients a "red". This would have overwhelmed the [[Overstay2 processes on the units to reduce discharge delay]] so we needed to determine a [[#pragmatic threshold]]. | of patients a "red". This would have overwhelmed the [[Overstay2 processes on the units to reduce discharge delay]] so we needed to determine a [[#pragmatic threshold]]. | ||
=== Pragmatic threshold === | === Pragmatic threshold === | ||
Revision as of 17:56, 10 June 2025
The Overstay2 colour is a colour assigned to a patient based on data collected as part of Project Overstay2, which is run through the Overstay2 scoring models to generate a score and the Overstay2 colour. The colour modifies the Overstay2 processes on the units to reduce discharge delay. See Overstay2 Overview for more context.
The colour is based on a probability threshold that is set to balance the number of patients followed under the more elaborate Overstay2 processes on the units to reduce discharge delay#red process wile minimizing the number of patients who would overstay by more than 10 days that are not captured. It was set separately for each of the Overstay2 scoring models, so see those pages for the actual thresholds.
Method of setting threshold
Optimal threshold
- A 2 x 2 classification table for varying probability levels helps visualize the performance of the predicted results with actual outcomes.
- Sensitivity (true positive rate) is the proportion of actual positives correctly identified by the predicted results. High sensitivity means few false negatives.
- Specificity (true negative rate) is the proportion of actual positives correctly identified by the predicted results. High specificity means few false positives.
- Optimal threshold is the probability level which shows equal or close to equal sensitivity or specificity in the testing data set. This threshold (also called cut-off) is being used to classify patients.
This would have resulted in a assigning xxx% of patients a "red". This would have overwhelmed the Overstay2 processes on the units to reduce discharge delay so we needed to determine a #pragmatic threshold.
Pragmatic threshold
We used the following process to set a red/yellow threshold that our Overstay2 processes on the units to reduce discharge delay can function with.
Goals/considerations
The thresholds were set to stratify patients based on the following considerations:
- goal: reduce LOS 10-15%
- no more than 15-17% of patients tagged
- capture 60-75% of all delayed discharges
|
How did those get set, though? If they are arbitrary then any rigor in balancing them is just dazzle. Ttenbergen 19:48, 8 June 2025 (CDT) |
Setting threshold based on these goals
|
|
Emailed 2025-06-08: we need to understand the mechanism we used to set the threshold. To be updated when clear. Ttenbergen 17:57, 8 June 2025 (CDT) |
Log
- 2025-06-08 - email to determine how to set colour in that new context
- 2025-05-27 - Julie finalized an updated Overstay2 scoring model for Grace Hospital
- 2025-02-?? - we realized we needed separate Overstay2 scoring models for each site
- 2025-02-21 - 2025-02-21 JM DR Overstay model decision and cut-off - We will use Model 8 at cut-off of 0.069
- 2025-02-20 - decided that we want
- goal: reduce LOS 10-15%
- no more than 15-17% of patients tagged
- capture 60-75% of all overstayers