PHI copy automation: Difference between revisions

From CCMDB Wiki
Jump to navigation Jump to search
m cleanup and links
No edit summary
 
(17 intermediate revisions by 2 users not shown)
Line 1: Line 1:
==Where does it live?==
We store personal health information separately from our other data in [[PHI.mdb]].
[[Data Processor PC]]
We can't have [[PHI.mdb]] on the Regional server because too many people have access and we can't log the access.
But, our data collectors need to be able to [[send]] this data to a location they can access.
This is the process how we handle that.


==What does this scheduled task do?==
==Process==
Every 60 minutes it runs the batch file [[Copy PHI.bat]].
* During [[sending]], [[CCMDB.accdb]] exports PHI files into {{Regionalserver|Output\_PHI}}.
* The [[Data processor]] does [["Pulling" Centralized data.accdb]] as part of [[Centralized data Vetting Process]]
** Under the hood this runs [[Copy PHI.bat]]
* [[Copy PHI.bat]] copies the files from {{Regionalserver|Output\_PHI}} to {{S:\MED\MED_CCMED}}CCMDB\PHI. Then it calls the script [[Load Data.vbs]] to integrate them into [[PHI.mdb]].
* [[Load Data.vbs]] accesses [[PHI Loader.accdb]] and runs the sub ''loadAllCSVFiles'', which calls the code that imports all csv files found within the directory.  The csv loading code also moves the files again into an archive folder {{S:\MED\MED_CCMED}}CCMDB\PHI\PHI Archive\ to prevent duplicate processing.  


It copies the PHI files exported by [[CCMDB.mdb]] during [[sending]] to {{Regionalserver|Output\_PHI}} to  X:\CCMDB\PHI. Then it calls the script [[Load Data.vbs]].
=== Running this manually ===
* navigate to {{S:\MED\MED_CCMED}}CCMDB\PHI'''
* double-click [[copy_PHI.bat]] and let it finish. May take a few (<10) minutes, depending on the build-up of files
** to check if this is still running, click ctrl-shift-esc to open task manager and search for "windows scripting host" - if persent, the process is still running.
*** If still present after 15 minutes, or if the number of .csv files doesn't go down every minute or so, then there likely is some error that isn't trapped. To troublehsoot, manually run ''loadAllCSVFiles'' in [[PHI Loader.accdb]] and follow up on problems
* check/refresh {{S:\MED\MED_CCMED}}CCMDB\PHI''' to make sure no *.csv files remain in that directory
** if any are left, troubleshoot; files that remain in there have likely not been integrated into [[PHI.mdb]]


The script [[Load Data.vbs]] accesses [[PHI Loader.accdb]] and runs the sub ''loadAllCSVFiles'', which calls the code that imports all csv files found within the directory.  The csv loading code also moves the files again into an archive folder X:\CCMDB\PHI\PHI Archive\ to prevent duplicate processing.
== Future of this process ==
Depending how PHI is stored in the future, [[Re-platforming]] might make this obsolete.


'''If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.''' - This needs to be reviewed as the last line of the batch file has changed the the return value may be altered [[User:Bcloutier|Bcloutier]] 19:47, 2013 November 20 (CST)
{{LegacyContent
| explanation=This used to be a scheduled, automated process, but we stopped that because we spent much time troubleshooting. It is now run as part of [push pull]] only.
 
| content=
 
This used to be set up as a scheduled task on the [[Data Processor PC]] before COVID. It ran the batch file [[Copy PHI.bat]] every 60 minutes. See [[Automatic updating of MS Access Databases using scheduled tasks]] for more info on how this works.
 
This process would lock up occasionally preventing collectors from sending. So, we stopped the scheduled task that ran this every 5 minutes. We were initially going to review this to make it work again, but with COVID a desktop scheduled task became entirely impractical anyway.
 
If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.


== If the automation were not working... ==
== If the automation were not working... ==
Line 17: Line 39:
If main office notices that X drives is down, Data processor must not remove the csv files from the Regional server until:
If main office notices that X drives is down, Data processor must not remove the csv files from the Regional server until:
* computer is rebooted and network connection to X is checked to see if it has been re established, otherwise this bat files is not able to do its job.   
* computer is rebooted and network connection to X is checked to see if it has been re established, otherwise this bat files is not able to do its job.   
**occurrence: Monday June 15.14 at 1700 hrs and all day June 16.14: issue: Castrophic hardware failure on this network drive had occurred.  
**occurrence: Monday June 15.14 at 1700 hrs and all day June 16.14: issue: Catastrophic hardware failure on this network drive had occurred.  
}}


== Related Articles ==
{{Related Articles}}


[[Category: 2013 data upgrades]]
[[Category:2013 data upgrades]]
[[Category: Scheduled Tasks]]
[[Category:PHI]]
[[Category: PHI]]

Latest revision as of 13:23, 6 January 2026

We store personal health information separately from our other data in PHI.mdb. We can't have PHI.mdb on the Regional server because too many people have access and we can't log the access. But, our data collectors need to be able to send this data to a location they can access. This is the process how we handle that.

Process

Running this manually

  • navigate to S:\MED\Med_CCMED\S:\MED\Med_CCMED\CCMDB\PHI
  • double-click copy_PHI.bat and let it finish. May take a few (<10) minutes, depending on the build-up of files
    • to check if this is still running, click ctrl-shift-esc to open task manager and search for "windows scripting host" - if persent, the process is still running.
      • If still present after 15 minutes, or if the number of .csv files doesn't go down every minute or so, then there likely is some error that isn't trapped. To troublehsoot, manually run loadAllCSVFiles in PHI Loader.accdb and follow up on problems
  • check/refresh S:\MED\Med_CCMED\S:\MED\Med_CCMED\CCMDB\PHI to make sure no *.csv files remain in that directory
    • if any are left, troubleshoot; files that remain in there have likely not been integrated into PHI.mdb

Future of this process

Depending how PHI is stored in the future, Re-platforming might make this obsolete.

Legacy Content

This page contains Legacy Content.

  • Explanation: This used to be a scheduled, automated process, but we stopped that because we spent much time troubleshooting. It is now run as part of [push pull]] only.
  • Successor: No successor was entered

Click Expand to show legacy content.

This used to be set up as a scheduled task on the Data Processor PC before COVID. It ran the batch file Copy PHI.bat every 60 minutes. See Automatic updating of MS Access Databases using scheduled tasks for more info on how this works.

This process would lock up occasionally preventing collectors from sending. So, we stopped the scheduled task that ran this every 5 minutes. We were initially going to review this to make it work again, but with COVID a desktop scheduled task became entirely impractical anyway.

If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.

If the automation were not working...

If the Data Processor PC that this is scheduled on isn't running (shutdown) then data might be left on the server for some time. Probably OK for now since it would still only be temporary.

If X drive goes down

If main office notices that X drives is down, Data processor must not remove the csv files from the Regional server until:

  • computer is rebooted and network connection to X is checked to see if it has been re established, otherwise this bat files is not able to do its job.
    • occurrence: Monday June 15.14 at 1700 hrs and all day June 16.14: issue: Catastrophic hardware failure on this network drive had occurred.


Related Articles

Related articles: