Re-analysis and generation of Overstay2 model: Difference between revisions
Ttenbergen (talk | contribs) |
Ttenbergen (talk | contribs) |
||
| Line 29: | Line 29: | ||
* This resulted in a dataset with the following: | * This resulted in a dataset with the following: | ||
** Total hospitalizations: {{DJ}} | |||
** Overstay >= 10 days =1: 4.1% | |||
** Overstay < 10 days = 0: 95.9% | |||
{{Discuss| add a table of admission numbers by year and site }} | {{Discuss| add a table of admission numbers by year and site }} | ||
The SAS code defining this dataset can be found | |||
{{DJ|where?}} | |||
The CFE code defining this dataset | |||
{{DT | still needs to be set up by Tina... }} | |||
{{Collapsable | {{Collapsable | ||
| Line 72: | Line 81: | ||
}} | }} | ||
=== Model development Inclusion/Exclusion of "Green" admissions === | |||
If we plan to generate overstay colours like the last time, then the one group who would not have the model applied to them would be the “greens”, since the decision tree turns them green before the model would be applied. If we were able to determine who these greens would have been, would we want to exclude them from the model? | |||
There is no way to exclude the greens from the model, so we won’t try. | |||
== Analysis and model generation == | == Analysis and model generation == | ||
Revision as of 17:06, 23 February 2025
This page is about the development of the model for generating scores/colours for Project Overstay2. Since our data collection and the healthcare system changed since the first iteration, we did a re-analysis and generation of Overstay2 model, resulting in the Overstay2 scoring model that generates the colour. Also see the Overstay2 Overview.
Defining the contributing factors data
The model depends on a regression analysis of a number of possible factors in our regularly collected data. Our data structure had changed since the original project, so we cleaned up our definitions, resulting in the Data definition for contributing factors for the Overstay2 project.
|
Still needs:
|
Model dataset and date range
- Dataset: We used the file 2025-2-3_13.56.31_Centralized_data.accdb as a basis for the project. A copy for future reference is at
- \\ad.wrha.mb.ca\WRHA\HSC\shared\MED\MED_CCMED\Julie\MedProjects\Overstay_Project_2025
- Reference Admit DtTm: We based the date range on the first medicine admit date during a Data definition for contributing factors for the Overstay2 project#Hospitalization, based on the earliest Boarding Loc dttm.
- Dataset inclusion criteria: (all/and) of the following
- Reference Admit DtTm >=2020-11-01 and <2025-01-01
- RecordStatus = Vetted
- final dispo of the Data definition for contributing factors for the Overstay2 project#Hospitalization is to a destination outside of the hospital of the admission (can be to other hospital)
- HOBS: include the record only if:
- the first medicine admission during a hospitalization is on a HOBS unit, and
- there is a Transfer_Ready_Dttm associated with that unit, and
- the patient is discharged from that unit to a non-hospital location
|
|
- This resulted in a dataset with the following:
- Total hospitalizations:
|
|
{{{1}}} |
- Overstay >= 10 days =1: 4.1%
- Overstay < 10 days = 0: 95.9%
|
|
add a table of admission numbers by year and site |
The SAS code defining this dataset can be found
|
|
where? |
The CFE code defining this dataset
|
still needs to be set up by Tina... |
| Specific decisions were discussed and made. |
|
JM had found Vetted n=226 cases with Last discharge DtTm (in ICU or Med) after 2024 until Feb 3,2025. Only 13 did not leave own site, 19 expired, 194 left the site. From the 213, some are long stayed patients admitted Aug –1, Sept-3, Oct-8, Nov-18, Dec=196. (DR agreed in the meeting with JM Feb10).
|
Model development Inclusion/Exclusion of "Green" admissions
If we plan to generate overstay colours like the last time, then the one group who would not have the model applied to them would be the “greens”, since the decision tree turns them green before the model would be applied. If we were able to determine who these greens would have been, would we want to exclude them from the model?
There is no way to exclude the greens from the model, so we won’t try.
Analysis and model generation
Dataset split into training and validation data
We separated the population into two datasets based on the odd/even status of the last digit of the Chart number:
- Even: Training set
- Odd: validation set
Model generation and testing
|
|
|
Decision on a model
|
|
|
This resulted in Overstay2 scoring model.
Decision on a probability threshold
The overstay score generated by Overstay2 scoring model is used to assign an Overstay2 colour based on a threshold value, which affects the patient care team activities of the Overstay2 processes on the units to reduce overstay. This section explains how we decided on that threshold value.
Optimal threshold
|
|
|
Pragmatic threshold
The drives a process that requires additional work form the patient care team. There are limits to those resources. The #optimal threshold would have resulted in a assigning xxx% of patients an Overstay2 colour of "red". This would have overwhelmed the Overstay2 processes on the units to reduce overstay.
|
|
|
It was determined that a xxx-yyy% of "red" would be the maximum we could assign, at least during the initial phase of the project. To achieve this, we chose a threshold of 0.069
For the selected Overstay2 scoring model this led to the following predicted values
- Overstay2 colour = "red": xxxx%
|
|
Do you have numbers for something like false positives/ false negatives/ positive predictive value/ etc? Will rely on you to make this something that would satisfy someone questioning this from a statistical angle. Ttenbergen 15:19, 23 February 2025 (CST) |
|
|
|