Peer Audit

The CCMDB Peer Audit is a real-time audit to quantify the variability in our data collection.

Preamble

What the Peer Audit is and is not

The Peer Audit is not meant identify "wrong" data or to single out a specific collector who is doing something bad. We are trying to quantify the precision rather than the accuracy of our individual data elements. A lot of us have hunches about where there are problems, this audit is to give us objective indicators.

Following that, please do not compare notes on the patients you are auditing as this would prevent us from getting an accurate idea how consistent our data is.

Goals and follow-ups to the peer audit

Julie will generate accuracy scores from the audit data, and we will provide these back to you in this article.

• You have stated that the audit is to check precision as opposed to accuracy, but the next paragraph talks about Julie tabulating & quantifying accuracy. Isn't this contradictory?--Jpeterson 14:42, 27 November 2009 (CST)

Once we know which areas have the least accuracy we will look for reasons for the inaccuracies and try to eliminate them. This will largely happen ad-hoc using the wiki. We may also come back to you personally to find out why there are discrepancies, but this is to find the reasons and fix the underlying problem, not to criticize individuals.

We will also use our findings to correct data, but this is for a very small subset of our database and just coincidental.

Start Date

• programming by Tina - Done
• testing by collection office - in progress
• pilot by collectors - START TEST: Nov 17.09.TOstryzniuk 13:00, 17 November 2009 (CST)
  • sites to start Nov 17.09:
    • HSC SICU & MICU (Joyce and Lois)
    • HSC Med all wards (Gail, Con, Pat, Marie)
    • VIC Med (Tara and Shirley start Nov 18)
    • GRA Med all wards (Steph and Sheila)
    • STB CICU - Laura K
  • Start Week of Nov 23.09 on Thursday Nov 26.09
    • STB -all ICUs- MICU CICU & CCU (Kym and Darlene & Laura)
    • STB Med all wards (Deb, Elaine, Galye)
    • VIC Med - all wards (Wendy, Tara, Shirley)

Processes and Procedures

Data Collection

What is collected

All data elements for patients

• Includes: Medicine TMP: Moves data

• exclude TISS for ICU
• exclude all TMP file special projects:
  • • Current: Catheter09, QASeptic, QAInf, Kym-Euroscore,
    • Not being done but listed in TMP: Transfusion, SmartPump Audit.

How it is collected

Every data collector (except community ICU) has an audit ward assigned in Peer Audit Partners.

Starting on Thursday morning, the first patient admitted or transferred to the audit ward whom you have no prior information and whose chart you have seen the first time will be an audit patient and will be followed as if he or she were a patient admitted to the regular ward of that collector.

The serial numbers to be used for audit patients will be 111 to 140 (if you need higher, you will not be able to send your audit data, contact Trish or Tina). If a patient is not discharged by next Thursday, use the next number, e.g. 112. Re-use earlier numbers once they become available, i.e. once patient 111 is sent and deleted, use the number for the next audit patient.

On the next send day a separate batch is sent for peer audit patients discharged during the previous week. To do this, make sure you either first delete your regular sent patients, or that you uncheck their "FinalCheck" checkbox. The records are sent as a separate batch with the following parameters

• batch: "a"
• initials: dd-mmm-yy- and use your own initials, e.g. 03-jun-09_TT
• EXAMPLE: a 03-Jun-09_BO

This will differentiate this data from our other collection data and facilitate further processing by our statistician.

Data Sending

If the batch is "a" (i.e. for audit patients) then CCMDB.mdb will send and temp information to the following alternative audit locations:

• Tasks: Regional Server\output\tasks_audit.mdb
• Temp: Regional Server\output\TmpV2_audit.mdb
• Don't need to process pending for this.

Central Office Data Processing

• Pagasa will run a batch file Peer Audit move csv to X bat each week.
  • BATCH FILE:
    • Moves all files labels with C_A* or M_* to SAN drive (X):\med_CCMED\CCMDB\Peer Audit
    • Also copies tmpV2_1_audit.mdb and TASKS_1_audit.mdb to same folder.

• Waiting for update to Data Appending ProcessTtenbergen 16:05, 10 September 2009 (CDT)

Central Office- Data Analysis

• Statistician will retrieve file from X:\\Med_CCMED\CCMDB|Peer Audit location
• match every field one on one, give count of good vs bad and degree of difference
• do a pair-analysis for dxs, Admit 1 specific, and others regardless of diagnosis number

Follow-Up

• Post accuracy scores to this article
• further investigate causes for differences

QUESTIONS from Collectors

Discussion

Coverage

• I regularly cover for both Kym and Darlene for sick, vacation and we help each other out when the others are busy. How do I audit an area if I am covering for vacation in the same area??--LKolesar 11:07, 18 November 2009 (CST)
  • Good question. If you are covering for vacation in the same area, don't do an audit on that week. The main idea for the peer audit is to repeat the data collections totally by 2 different data collectors. -- JMojica 14:14, 18 November 2009 (CST)
  • Don't do audits during week you are covering vacation.TOstryzniuk 15:33, 18 November 2009 (CST)
    • Scenario: Audit patient started by one collector Jane in week 1. Week 2 collector Jane becomes unavailable. Collector Betty takes over for her. Betty is collector whom Jane usually audits. Option a: Does Betty discard the audit patient started by Jane? Option b: Does Betty leave the already started audit patient for Jane to finish when Jane gets back, but simply not start a new audit patient that week? I think the option be would be better, otherwise we could loose a LOT of patients to weird exceptions. Ttenbergen 17:20, 23 November 2009 (CST)

Start Date at StB

• For SBGH med, our start date is the week of Nov.23/09. Do we audit the first patient admitted to the ward for Monday Nov. 23/09, or do we audit the first patient admitted to the ward for Thursday, Nov. 26/09? (under "How it is collected-it says to audit "the first patient admitted to the audit ward on Thursday morning") We will also be doing some vacation relief starting the month of December, and actually continuing on for most of January and February 2010. So, do we do the same as Laura, above, and not audit if you are covering for vacation?DPageNewton 14:36, 18 November 2009 (CST)
  • STB will start "next week" and audit the first patient that arrived on audit ward on Thursday Nov 25.09.TOstryzniuk 15:33, 18 November 2009 (CST)

First admitted vs first moved?

• For Vic med- We collect the admit date/time pt admitted to medicine service and the actual date/time the patient is "moved" to the ward.
  • Do you want the first patient on the Thursday to be the first patient to arrive to the ward that day or the first patient admitted to the medicine service that day?
    • The first patient that arrives from another unit "OR" is admitted from ER to the ward that you are auditing on, which ever one is "first" to arrive on that ward.

What if no patients are admitted on a Thursday?

• What do you wish us to do if there are no patients admitted that day?
  • Choose the first patient that arrived on the ward on the next day which would be a Friday.
    • This is actually addressed more generally in the original definition: "Starting on Thursday morning, the first patient" means that it doesn't matter when the patient actually arrives, pick the first one after Thursday morning. Ttenbergen 17:20, 23 November 2009 (CST)

Exceptions to 1pt per week

• Do you want at least 1 patient per week for the audit?
  • Yes, you must audit one patient per week, every Thursday from now on.
    • Except if a ward is being covered. Ttenbergen 17:20, 23 November 2009 (CST)

What if a collector does not work on a Thursday?

• Also, Shirley works the Mon, Tues, and Wed while Wendy and I work the Tues, Wed, and Thurs. How do you want us to work this so we are consistent?
  • On a Monday, Shirley will still have to audit (enter on net book) the first patient who arrived on her assigned auditing ward on the Thursday of the previous week.

Long Stay Patients

• Also, what happens if the patients we are auditing have prolonged lengths of stay (ie 6 mos to a year)?
  • You will continue to follow patients until discharged or moved from your audit ward. Julie can decide what she wants to do. Lets see how many end up being in this group. She will be able to see by the pending reports and will advise further.

EMIP/OVER pts

• Also, Wendy and I now do S3 as well and all the EMIP/OVER patients and split the workload between our laptops. Do you want us to follow any of the S3 patients as most of these patients have been transferred from other wards and have been medically stable and are usually waiting placement?
  • Shirley will audit N5 & S3. See: Peer Audit Partners for details. We will not audit EMIP or Over.---TOstryzniuk 13:29, 19 November 2009 (CST)

Audit Transfers as well?

• Do you want us to follow patients that are transferred between wards or just new admits?
  • Both. If the first patient that arrives on your audit ward on Thursday is a patient that was transferred over from your own collection ward do not audit that patient. Select the "next" patient that arrived on your audit ward on Thursday. The main idea of a peer audit is to repeat data collection for a patient by two totally different data collectors.--TOstryzniuk 13:29, 19 November 2009 (CST)

• Sorry lots of question but we would like to be consistent with other locations.TAngell 15:04, 18 November 2009 (CST)

STB RE: Lab and Pharmacy items

• • We at STB want to know who would need validation (via the audit) on lab and pharmacy counts??? To my knowledge, no one has ever used this data. We are fine with auditing the diagnostics, comorbids and apache scores but auditing labs and pharmacies seems unnecessary, is tedious and time consuming. We would appreciate this being looked at before the audits begin. Thank you. KYM, LAURA and Darlene. --LKolesar 14:59, 24 November 2009 (CST)

STB Med Location on Registry page on PDA

• The audit starts for SBGH tomorrow. I will be auditing Gayle's ward. For the location on the registry page, do I put E6, Gayle's ward, or do I put the location as 5B, my ward? DPageNewton 18:18, 25 November 2009 (CST)
  • Location: STB_E6. --TOstryzniuk 18:58, 27 November 2009 (CST)