Re-analysis and generation of Overstay2 model

This page is about the development of the model for generating scores/colours for Project Overstay2. Since our data collection and the healthcare system changed since the first iteration, we did a re-analysis and generation of Overstay2 model, resulting in the Overstay2 scoring model that generates the colour. Also see the Overstay2 Overview.

Defining the contributing factors data

The model depends on a regression analysis of a number of possible factors in our regularly collected data. Our data structure had changed since the original project, so we cleaned up our definitions, resulting in the Data definition for contributing factors for the Overstay2 project.

Still needs: considerations values we considered and rejected minimize duplication of Data definition for contributing factors for the Overstay2 project

• SMW

• Cargo

• Categories

Model dataset and date range

• Dataset: We used the file 2025-2-3_13.56.31_Centralized_data.accdb as a basis for the project. A copy for future reference is at
  • \\ad.wrha.mb.ca\WRHA\HSC\shared\MED\MED_CCMED\Julie\MedProjects\Overstay_Project_2025

• Reference Admit DtTm: We based the date range on the first medicine admit date during a Data definition for contributing factors for the Overstay2 project#Hospitalization, based on the earliest Boarding Loc dttm.

• Dataset inclusion criteria: (all/and) of the following
  • Reference Admit DtTm >=2020-11-01 and <2025-01-01
  • RecordStatus = Vetted
  • final dispo of the Data definition for contributing factors for the Overstay2 project#Hospitalization is to a destination outside of the hospital of the admission (can be to other hospital)
  • HOBS: include the record only if:
    • the first medicine admission during a hospitalization is on a HOBS unit, and
    • there is a Transfer_Ready_Dttm associated with that unit, and
    • the patient is discharged from that unit to a non-hospital location

is this really simply "to a non-hospital location" or is it the same as above: "to a destination outside of the hospital of the admission (can be to other hospital)"?

• SMW

• Cargo

• Categories

• This resulted in a dataset with the following:
  • Total hospitalizations:

{{{1}}}

• SMW

• Cargo

• Categories

• • Overstay >= 10 days =1: 4.1%
  • Overstay < 10 days = 0: 95.9%

add a table of admission numbers by year and site

• SMW

• Cargo

• Categories

The SAS code defining this dataset can be found

where?

• SMW

• Cargo

• Categories

The CFE code defining this dataset

still needs to be set up by Tina...

• SMW

• Cargo

• Categories

ExpandSpecific decisions were discussed and made.

JM had found Vetted n=226 cases with Last discharge DtTm (in ICU or Med) after 2024 until Feb 3,2025. Only 13 did not leave own site, 19 expired, 194 left the site. From the 213, some are long stayed patients admitted Aug –1, Sept-3, Oct-8, Nov-18, Dec=196. (DR agreed in the meeting with JM Feb10). First Med Admits who were RecordStatus = incomplete but with Dispo DtTm present are excluded. First Med Admits who were still in the unit are excluded (ie no Dispo DtTm) First Med Admits who were RecordStatus = vetted are included. Deceased should be included: I think there was talk about excluding these; I don’t think that is valid. We don’t know when they arrive that they will die, and if they die after becoming transfer ready that is still an overstay we could have avoided. Discharge to or Previous Location = Hospice should be included – for the same reason we would include PCH. Palliative patients should be included because our definition “Palliative care” (ICD10 Z51.5) doesn’t imply death is imminent. Palliative patients were excluded before, but our definition has changed, and how this appears to be handled now has as well. Also, they may be waiting in hospital for a hospice, so again, that’s overstay. Discharged to STB Palliative Care - -included (DR agreed in the meeting with JM feb10) AMA – include these. Initial thought was that AMA implies they were not discharge ready, but it could also include those who were sick of waiting for a PCH and walked out. They might just be someone who waited for 2 weeks while dispo ready and eventually ran away because they did not want to wait for home care or etc any longer. But can someone be transfer ready and still leave AMA? Yes, e.g. when they were transfer ready but the discharge took so long that they no longer are and now can leave AMA again. JM found 3061 dispo AMA (2810 wo TR_dt, 251 w TR_Dt) Dispo TCU/TCE – include, and treat as discharge from this hospital Dispo HSC Lennox Bell/Institution NOS – treat as we would back-to-PCH/home Dispo another ward within WPG (LAU at CON, OAKS, VIC)? – include, and treat as discharge from this hospital Unknown disposition at discharge on the last admission – those transferred to another service (ICU/ OR/ etc within the same hospital - already excluded with RecordStatus = ”incomplete” and by only including if (1c) Dispo Transfers to different hospital ICU within Winnipeg – include Transfers outside WPG – include and treat as if discharged Overstay 5 to 9 days - included as normal (Rodrigo excluded these from model building) A null Tr_DtTm will be allowed This defines “hospitalization” as per-site, so if the patient is moved to subsequent medicine wards at a different hospital there will be a new record EMIP / TR_DtTm during ED portion of visit: treat this as you would on the ward. The First TR DTtm at ER will be taken regardless whether there is a second TR dttm when patient moved to a Med Med ward (DR agreed in the meeting with JM Feb10)

Model development Inclusion/Exclusion of "Green" admissions

If we plan to generate overstay colours like the last time, then the one group who would not have the model applied to them would be the “greens”, since the decision tree turns them green before the model would be applied. If we were able to determine who these greens would have been, would we want to exclude them from the model?

There is no way to exclude the greens from the model, so we won’t try.

Analysis and model generation

Parameter candidates

See Data definition for contributing factors for the Overstay2 project for the parameters currently in use.

1. Age
2. Pre-living situation – from non-PCH/Chronic
3. ADL components and
   • (ADLSCore-12) *NH among those who came from PCH/CHF
   • (ADLSCore-12) *Age - interaction with Age
4. GCS components
5. Postal Code
   • WRHA – Winnipeg
   • Northern
   • Northwestern Ontario
   • P0X (Kenora Region)
   • P0Y (Whiteshell Park Region)
   • Urban P9N (Kenora)
   • Urban P8T (Sioux Lookout)
   • Urban P8N (Dryden)
   • P0W (Rainy River Region)
   • P9A(Fort Frances)
   • Rest MB
   • Rest
     • Analysis notes: JM found postal code N/A =2759, JM used the R_Province, Pre_inpt_Location, Previous Location instead to define the 5 categories above. Also encountered no match in the Postal_Code_Master List but was able to categorized based on the first 3 characters (N=273) - list given to Pagasa to add. (DR agreed in the meeting with JM Feb10)
6. Charlson Comorbids (Categories and Total Score)
   • MI, CHF, PVD, CVA , Pulmonary, Connective, Ulcer, Renal
   • Charlson Score * NH among those who came from PCH/CHF
7. Other Diagnoses (admit and comorb) :
8. Diagnoses that might prevent/delay meeting PCH/Home Care criteria
   • having a trach: "Tracheostomy, has one" ICD10 Z93.0
   • having a PEG (Percutaneous Feeding Tube): (has gastrostomy code, ICD10 Z93.1)
   • possibly "Iatrogenic, mechanical complication/dysfunction, internal prosthetic device or implant or graft NOS" - it implies that an internal device is there, PCHs would disallow some of these. ICD10 T85.6
   • possibly: CCI "Implantation of Internal Device" - PEGs are included in this, and some others might also disqualify; if the device stays the pt might not be accepted by PCH but we would not necessarily code removal CCI component2 53
   • "Suprapubic catheter, indwelling, has one" ICD10 Z93.5
   • "Artificial opening NOS, has one" - these include Ileal conduit (urostomy), PD catheter, Nephrostomy tube, Mitrofanoff procedure ICD10 Z93.8
   • "Ileostomy or colostomy, has one", "Gastrostomy, has one” ICD10 Z93.4, ICD10 Z93.1
   • addiction opioids and stimulants
     • stimulants:
     • "Stimulants incl methamphetamine, * "
     • "Cocaine, *" ICD10 F15.0,F15.2,F15.3, T40.5, F14.0, F14.2, F14.3
     • opioids: "Opioid/narcotic, *" ICD10 T40.6
   • Dementia ICD10 F01.1, F03, G30
9. Pre-admit inpatient location: homeless

Dataset split into training and validation data

We separated the population into two datasets based on the odd/even status of the last digit of the Chart number:

• Even: Training set
• Odd: validation set

Model generation and testing

we should add some basic info details can remain in other files such as SAS, but this should include file links

• SMW

• Cargo

• Categories

Decision on a model

the statistical tests that were done to evaluate the model the factors leading to our decision on "Model 8" links to files

• SMW

• Cargo

• Categories

This resulted in Overstay2 scoring model.

Decision on a probability threshold

The overstay score generated by Overstay2 scoring model is used to assign an Overstay2 colour based on a threshold value, which affects the patient care team activities of the Overstay2 processes on the units to reduce overstay. This section explains how we decided on that threshold value.

Optimal threshold

What was the consideration for the initial choice of, I think, 0.051?

• SMW

• Cargo

• Categories

Pragmatic threshold

The drives a process that requires additional work form the patient care team. There are limits to those resources. The #optimal threshold would have resulted in a assigning xxx% of patients an Overstay2 colour of "red". This would have overwhelmed the Overstay2 processes on the units to reduce overstay.

initial thoughts were "15-17% being red, with an aim to get 60-75% of overstay patients"

• SMW

• Cargo

• Categories

It was determined that a xxx-yyy% of "red" would be the maximum we could assign, at least during the initial phase of the project. To achieve this, we chose a threshold of 0.069

For the selected Overstay2 scoring model this led to the following predicted values

• Overstay2 colour = "red": xxxx%

Do you have numbers for something like false positives/ false negatives/ positive predictive value/ etc? Will rely on you to make this something that would satisfy someone questioning this from a statistical angle. Ttenbergen 15:19, 23 February 2025 (CST)

• SMW

• Cargo

• Categories

Does this page miss anything that is not addressed elsewhere as per pages either linked from here or from Overstay2 Overview? If not feel free to delete this question. Ttenbergen 15:19, 23 February 2025 (CST)

• SMW

• Cargo

• Categories