Instructions for importing a batch of DSM Data

Preliminary information, this is not active yet

see Instructions for requesting a batch of data from DSM for counterpart

We receive data from DSM that needs to be imported into our main repository.

Instructions

Something is still not right with the code for reconnecting, Tina needs to look into. Ttenbergen 17:04, 2018 May 17 (CDT)

• SMW

• Cargo

• Categories

Prep

1. copy files to local, they are large and won't work well on a share.
   • make a new local directory
   • Put the following into the directory
     • Centralized_data.mdb
     • DSM_Labs_Consistency_check.accdb
     • DSM Labs data.accdb
     • PHI not needed - PHI.mdb is not needed during import because it matches by D_ID
2. copy the newly received batch of data from DSM_Lab_Extract#File_Share to a different local directory that contains only those files
3. open DSM_Labs_Consistency_check.accdb and click the Re-link button

Check for new labs

1. Open DSM_Labs_Consistency_check.accdb
2. Click the "Check DSM .csv for new labs " button
3. pick the directory with the new data files
4. wait for the check to complete, about half an hour
5. if the program tells you there are new labs, review them and ask Trish or Allan if any of them should be considered as part of our counts
   • open the following tables, and check labs added today
     • s_group, sort by added_dt
     • s_mapping_lab, sort by date_listed newest first
   • If Trish or Allan says any of them need to be added talk to Tina, we can document that process when it next happens. It involves editing table s_mapping_labs or s_group to accommodate the new data.

actual data import

If the above listed no new labs, or the new labs were resolved, click the "Import DSM .csv" button and follow essentially the same steps as above to import the actual data.

Cleanup

1. Move files back to their master directories
   • DSM_Labs_Consistency_check.accdb - only need to move if new labs were found
   • DSM Labs data.accdb
   • move the folder you processed from the directory we share with DSM to the archive directory in the same folder
2. enter import date in log below

Integrity checks after import

Please note here if any are needed.

D_IDs in CFE for which no DSM labs exist

This could happen if

• we have a bad MRN / PHIN
• we change/fix a D_ID after we request data

This could also be true where no labs were sent for, eg a patient who dies shortly after arrival. In the past we would have entered a "no labs" for these. Do we want to do something similar? It would have to be Pagasa that does it. Might be a lot of extra work. Need to review. (ex. wrong D_ID when exported but found it error and so fixed it before the data for import comes back). For now we do not have an entry like that. And it might not be worth it - what would Pagasa do to check that the no-labs are legit?

• SMW

• Cargo

• Categories

D_IDs in DSM data for which no CFE record exists

Could happen if we change/fix a D_ID after requesting. Problem is, we can find if one doesn’t exist, but not if one was changed, but the one that was in the export is actually in present now. Not sure how we would catch that.

Duplicate RequestNo

Should never happen due to how import works, but can check anyway.

Known issues

None at this time

How does it work

• Lab identification in the DSM data

Log

eg copy/paste and fill in the following: 
* ~~~~~ - imported batch

• 14:01, 2019 February 14 (CST) - tried to import 2018-06-19_request but there was no data there
• 11:34, 2018 May 18 (CDT) - imported batch 2018-04-05
• 17:47, 2018 April 9 (CDT) - imported batch 2017-10-26 and 2018-01-25