PHI copy automation: Difference between revisions

From CCMDB Wiki
Jump to navigation Jump to search
← Older edit
No edit summary
No edit summary
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{LegacyContent
We store personal health information separately from our other data. This process is par of managing that data.  
| explanation=This process would lock up occasionally preventing collectors from sending. So, we stopped the scheduled task that ran this every 5 minutes. We were initially going to review this to make it work again, but with COVID a desktop scheduled task became entirely impractical anyway. As of 2022-08 we are hoping for Shared Health to provide either interim functionality on the Automation PC or maybe an actual solution to our storage requirements that doesn't need this. See also [[Re-platforming]].
We can't have [[PHI.mdb]] on the Regional server because too many people have access and we can't log the access.  


| content=
==Process==
* During [[sending]], [[CCMDB.accdb]] exports PHI files into {{Regionalserver|Output\_PHI}} .
* Running [[Pull]] or manually running it will run [[Copy PHI.bat]]
* [[Copy PHI.bat]] copies the files from {{Regionalserver|Output\_PHI}} to {{S:\MED\MED_CCMED}}CCMDB\PHI. Then it calls the script [[Load Data.vbs]] to integrate them into [[PHI.mdb]].
* [[Load Data.vbs]] accesses [[PHI Loader.accdb]] and runs the sub ''loadAllCSVFiles'', which calls the code that imports all csv files found within the directory.  The csv loading code also moves the files again into an archive folder {{S:\MED\MED_CCMED}}CCMDB\PHI\PHI Archive\ to prevent duplicate processing.


=== Running this manually ===
* navigate to {{S:\MED\MED_CCMED}}CCMDB\PHI'''
* double-click [[copy_PHI.bat]] and let it finish. May take a few (<10) minutes, depending on the build-up of files
* check/refresh {{S:\MED\MED_CCMED}}CCMDB\PHI''' to make sure no *.csv files remain in that directory
** if any are left, troubleshoot; files that remain in there have likely not been integrated into [[PHI.mdb]]
{{DT| there is a data processing instruction for this on the wiki, no time to find it right now.  }}


==Where does it live?==
== Future of this process ==
[[Data Processor PC]]
Depending how PHI is stored in the future, [[Re-platforming]] might make this obsolete.


==What does this scheduled task do?==
{{LegacyContent
Every 60 minutes it runs the batch file [[Copy PHI.bat]].
| explanation=This used to be a scheduled, automated process, but we stopped that because we spent much time troubleshooting. It is now run as part of [push pull]] only.  


It copies the PHI files exported by [[CCMDB.accdb]] during [[sending]] to {{Regionalserver|Output\_PHI}} to  {{S:\MED\MED_CCMED}}CCMDB\PHI}}. Then it calls the script [[Load Data.vbs]].
| content=


The script [[Load Data.vbs]] accesses [[PHI Loader.accdb]] and runs the sub ''loadAllCSVFiles'', which calls the code that imports all csv files found within the directory.  The csv loading code also moves the files again into an archive folder {{S:\MED\MED_CCMED}}CCMDB\PHI\PHI Archive\}} to prevent duplicate processing. See [[Automatic updating of MS Access Databases using scheduled tasks]] for more info on how this works.
This used to be set up as a scheduled task on the [[Data Processor PC]] before COVID. It ran the batch file [[Copy PHI.bat]] every 60 minutes. See [[Automatic updating of MS Access Databases using scheduled tasks]] for more info on how this works.


'''If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.''' - This needs to be reviewed as the last line of the batch file has changed the the return value may be altered
This process would lock up occasionally preventing collectors from sending. So, we stopped the scheduled task that ran this every 5 minutes. We were initially going to review this to make it work again, but with COVID a desktop scheduled task became entirely impractical anyway.
 
If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.


== If the automation were not working... ==
== If the automation were not working... ==
Line 24: Line 36:
* computer is rebooted and network connection to X is checked to see if it has been re established, otherwise this bat files is not able to do its job.   
* computer is rebooted and network connection to X is checked to see if it has been re established, otherwise this bat files is not able to do its job.   
**occurrence: Monday June 15.14 at 1700 hrs and all day June 16.14: issue: Catastrophic hardware failure on this network drive had occurred.  
**occurrence: Monday June 15.14 at 1700 hrs and all day June 16.14: issue: Catastrophic hardware failure on this network drive had occurred.  
}}


== Related Articles ==
== Related Articles ==
{{Related Articles}}
{{Related Articles}}


 
[[Category:2013 data upgrades]]
[[Category: 2013 data upgrades]]
[[Category:PHI]]
[[Category: PHI]]
 
}}

Latest revision as of 10:37, 10 November 2025

We store personal health information separately from our other data. This process is par of managing that data. We can't have PHI.mdb on the Regional server because too many people have access and we can't log the access.

Process

Running this manually

  • navigate to S:\MED\Med_CCMED\​S:\MED\Med_CCMED\CCMDB\PHI
  • double-click copy_PHI.bat and let it finish. May take a few (<10) minutes, depending on the build-up of files
  • check/refresh S:\MED\Med_CCMED\​S:\MED\Med_CCMED\CCMDB\PHI to make sure no *.csv files remain in that directory
    • if any are left, troubleshoot; files that remain in there have likely not been integrated into PHI.mdb

there is a data processing instruction for this on the wiki, no time to find it right now.

  • SMW


  • Cargo


  • Categories

Future of this process

Depending how PHI is stored in the future, Re-platforming might make this obsolete.

Legacy Content

This page contains Legacy Content.

  • Explanation: This used to be a scheduled, automated process, but we stopped that because we spent much time troubleshooting. It is now run as part of [push pull]] only.
  • Successor: No successor was entered

Click Expand to show legacy content.

This used to be set up as a scheduled task on the Data Processor PC before COVID. It ran the batch file Copy PHI.bat every 60 minutes. See Automatic updating of MS Access Databases using scheduled tasks for more info on how this works.

This process would lock up occasionally preventing collectors from sending. So, we stopped the scheduled task that ran this every 5 minutes. We were initially going to review this to make it work again, but with COVID a desktop scheduled task became entirely impractical anyway.

If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.

If the automation were not working...

If the Data Processor PC that this is scheduled on isn't running (shutdown) then data might be left on the server for some time. Probably OK for now since it would still only be temporary.

If X drive goes down

If main office notices that X drives is down, Data processor must not remove the csv files from the Regional server until:

  • computer is rebooted and network connection to X is checked to see if it has been re established, otherwise this bat files is not able to do its job.
    • occurrence: Monday June 15.14 at 1700 hrs and all day June 16.14: issue: Catastrophic hardware failure on this network drive had occurred.


Related Articles

Related articles:
Retrieved from "https://ccmdb.kuality.ca/index.php?title=PHI_copy_automation&oldid=167955"