Instructions for importing a batch of DSM Data

Preliminary information, this is not active yet

see Instructions for requesting a batch of data from DSM for counterpart

We receive data from DSM that needs to be imported into our main repository.

Instructions

1 I have to make/fix a connector module for this.

• SMW

• Cargo

• Categories

Prep

1. copy files to local, they are large and won't work well on a share.
   • make a new local directory
   • Put the following into the directory
     • Centralized_data.mdb
     • DSM_Labs_Consistency_check.accdb
     • DSM Labs data.accdb
     • PHI not needed - PHI.mdb is not needed during import because it matches by D_ID
2. copy the newly received batch of data from DSM_Lab_Extract#File_Share to a different local directory that contains only those files
3. open DSM_Labs_Consistency_check.accdb and follow the instructions in Instructions for requesting a batch of data from DSM#Instruction/steps for requesting data to reconnect the tables

Check for new labs

1. Open DSM_Labs_Consistency_check.accdb
2. Click the "Check DSM .csv for new labs " button
3. pick the directory with the new data files
4. wait for the check to complete, about half an hour
5. if the program tells you there are new labs, review them and ask Trish or Allan if any of them should be considered as part of our counts; click "No" for this
   • open the following queries (they are based on labs imported within the last hour)
     • s_group_ListNewlyAdded
     • s_mapping_lab_ListNewlyAdded
   • If any of the new ones are identified by Trish or Allan as needing to be counted then you have to ask Tina or Julie to add them to the list; the step to add them is to put the name of the lab they will be counted as into the "Our_Lab_Name" field, but we need to confirm first that this would not result in a double-count.
   • If no additional lab just close it.

There is a newly found and new added query for each; why are there two and can I delete one set?

• SMW

• Cargo

• Categories

actual data import

If the above listed no new labs, or the new labs were resolved, click the "Import DSM .csv" button and follow essentially the same steps as above to import the actual data. Close the file

Cleanup

Move files to their permanent homes. Make sure you do all of these!

1. Make a new directory in \\Hsc1msfp0001\med_ccmed\CCMDB\DSM Labs Imports\archive_processed and call it e.g. DSM_2019-03-27
   1. If labs were found, then copy (not move) DSM_Labs_Consistency_check.accdb to that new directory
   2. copy (not move) DSM Labs data.accdb to that new directory
2. move (not copy) DSM_Labs_Consistency_check.accdb and DSM Labs data.accdb from your computer to \\Hsc1msfp0001\med_ccmed\CCMDB\DSM Labs Imports\ (not the archive) - this is the master version.
3. move (not copy) the folder of data you processed from the directory it arrived where we share with DSM to the archive_raw directory in the same folder (these are big so they really should be moved, not copied and left in two places)
4. enter import date in log below

Data Integrity Checks (automatic list)

	App	Status
Query Import request matcher	DSM Labs Consistency check.accdb	needs review

queries for which we have no page yet

Duplicate RequestNo

Should never happen due to how import works. I am not even sure what that would check for.

Known issues

None at this time

How does it work

• Lab identification in the DSM data

Log

eg copy/paste and fill in the following: 
* ~~~~~ - imported batch

• 16:20, 2019 April 10 (CDT) - imported batch
• 2018-06-19 - It was sent twice. (First sent it was a mix up due to wrong file date instead of June 19, 2018 sent file date was July 11/2018) Imported Apr.10/19.
• 2019-03-27 - Imported Apr. 10/19
• 2019 March 09 - imported data for 2018-07-11, 2018-10-31, 2019-01-23
• 14:01, 2019 February 14 (CST) - tried to import 2018-06-19_request but there was no data there
  • imported batches
• 11:34, 2018 May 18 (CDT) - imported batch 2018-04-05
• 17:47, 2018 April 9 (CDT) - imported batch 2017-10-26 and 2018-01-25