DSM Lab Extract: Difference between revisions

From CCMDB Wiki
Jump to navigation Jump to search
Line 36: Line 36:
== Processing data received from DSM ==
== Processing data received from DSM ==
See [[Instructions for importing a batch of DSM Data]]
See [[Instructions for importing a batch of DSM Data]]
== Processing data on receipt ==
Open and run Module DSM_Processor Sub DSM_Data_prepper.
If there is a problem, clear out all tables (empty 0_TrialExport and run clear_made_tables), run compact and repair, try again.
The last column of the data can have problematic content. Last time it contained something that line-broke but wasn't a carriage return or line break. Finding short lines allowed detection and correction.


== File Share ==
== File Share ==

Revision as of 17:44, 2015 October 15

We are working on a way to get DSM to extract Labs for import to our DB.

Outstanding Issues

additional labs

  • we want at least NA, K, Lactate but Allan Garland is consulting with colleagues to find out if we need additional
    • Lactacte, NA and K are included with current output.

Cultures

When DSM sent the original batch of data they were not sure yet about how to extract cultures. We discussed this on the phone on our meeting on Aug 14 but I have not followed this up with an email and will not right now because the data looks so bad I don't know if we will want to pursue this.

  • Blood
    • ??
  • Sputum
    • Site = Sputum? There are a lot
  • Urine
    • Site = Urine? There are a handful 18

ABGs

The ABGs will not be complete, are only entered at STB and used to be at HSC but flakey lately. More info to come. Ttenbergen 13:17, 2014 August 14 (CDT) There was also discussion whether we want to include venous blood gasses.

duplicate records

There were 1,865,570 records in the export. There were 1,863,373 unique records. Why the duplicates? Need to confirm if this is still a problem. Ttenbergen 17:09, 2015 October 15 (CDT)

Requesting data

A file is exported to #File Share, and we ask #DSM Contacts to provide a new extract (how often? Ttenbergen 17:21, 2015 October 15 (CDT) Template:Discussion).

see Instructions for requesting a batch of data from DSM

Actions by DSM

DSM will take our file and generate an output of all labs for those patients for the admission. They will return it to us via the file share.

Processing data received from DSM

See Instructions for importing a batch of DSM Data

File Share

\\ad.wrha.mb.ca\wrha\REGION\Shared\CCMDB_Lab_Exchange

DSM Contacts

  • main contact: Alun Carter
  • Loretta Eng 9261406 Leng @ dsmanitoba.CA

Log

  • 2015-10-15 found bug in import, reimported, looks much better now, asked Pagasa, Gail and Trish if we could review a few more discrepancies Monday.
  • 2015-10-08 Pagasa and Gail and Trish reviewed data and returned excel sheet; need to review differences and find out if I can filter the extras
  • 2015-10-05 pushed Trish on date
  • 2015-09-30 emailed Trish to find out when she wants to review and test
  • 2015-09-28
  • directory issue fixed
  • can’t see directory that Alun made;
    • asked him to copy content to a diff directory I made.
    • Launched ticket with eHealth to fix underlying issue.
  • 2015-09-24 Alun emailed that new data is ready
  • 2015-09-21 replied busy but will get me a timeline.
  • 2015-09-21 emailed Alun f/u
  • 2015-08-13 finished first round of http://ccmdb.kuality.ca/index.php?title=DSM_Labs_Importer.mdb waiting for data for further tests
  • 2015-08-06 Alun is on vacation until Aug 24; no new import from him
  • 2015-07-22 requested a new output from Alun because missing time for "arrived" means comparisons will always be off
  • 2015-06-29 working...
  • 2015-06-25 re-starting; found erroneous line because it causes a short line.
  • 2014-12-29 importing new export into test tool; there were problems with the data file, then other things became higher priority
  • 2014-12-24 received new export from Alun
  • 2014-12-15 fu email to Alun; he figures won’t get to it till after Christmas… * 2014-12-05 provided new extract, updated wiki
  • 2014-11-20 Loretta emailed me to find out details… emailed her some. We are set to meet Dec 1, will need to review before then.
  • 2014-11-06 Allan will book meeting with Lab folk, hopefully over next week or so, snooze for 10days
  • 2014/Oct/20 no news from, whatever…
  • 2014/Oct/16 emailed Allan for update
  • Sept 2014 received extract from Alun carter, some good some bad
  • week of 2014-12-05 met with Loretta, Alun and Allan; agreed on next round of tests; Send new test data to Alun today. Ttenbergen 11:28, 2014 December 5 (CST)
  • 2014/09/25 met with Allan Garland, discussed. he was going to talk to DSM. Ttenbergen 12:56, 2014 September 29 (CDT)
  • email to put this on the Task group's agenda for this Thursday. Ttenbergen 10:39, 2014 September 22 (CDT)
  • did an initial analysis which doesn't look good. emailed to Allan and Julie for comment Ttenbergen 17:00, 2014 September 8 (CDT)
  • discussed labs list with Allan, now need to do some more validationTtenbergen 17:09, 2014 September 4 (CDT)
  • 2014/aug/25 started to review data, stuck on matching labs now, will need to talk with Trish when back. Also emailed Allan with some weirdness in the data. Ttenbergen 19:38, 2014 August 25 (CDT)
  • 2014/Aug/21 we received data last week, but have not formally reviewed it. Ttenbergen 10:39, 2014 August 21 (CDT) Sent list of labs to Allan, he offered to review... Ttenbergen 20:59, 2014 August 21 (CDT)
  • 2014/Aug/14 phone call with Loretta to confirm what is needed. Requested early sample data so we can start our processing... Ttenbergen 13:17, 2014 August 14 (CDT)
  • 2014/Aug/07 submitted test data at Loretta’s request
  • 2014/Jul/31 emailed Loretta again, since no news from Alun; ooomsg says back Tue Aug 5
  • 2014/Jul/21 We have a file share. Alun will hopefully have test data by end of week, or early next week
  • 2014/Jul/17 Email from Chad Kitching re. share folder description
  • 2014-07-14 emailed Alun and Loretta re progress
  • 2014-06-16 talked with Loretta on Friday. She said Alun Carter is the data guy we will work with. She suggested a different format and requested a list of labs we want, which I gave her. Also emailed Alun to get this moving. However, he is on vacation for 3 weeks, will f/u when he is back
  • 2014-06-12 emailed Loretta a test extract;
  • 2014-06-12 emailed Loretta a sample request file. Ttenbergen 17:47, 2014 June 12 (CDT)