PHI copy automation: Difference between revisions

(55 intermediate revisions by 4 users not shown)

Line 1:

~~As part~~ of ~~the [[2013~~ data ~~upgrades]],~~ [[~~CCMDB~~.mdb]] ~~will now send [[some data]] to~~ the [[Regional ~~Server]]\Output\_PHI ("_" to sort separate from hospitals) which will be picked up by a scheduled task on Trish~~'~~s computer calling '''X:\CCMDB\PHI\[[copy_PHI.bat]]''' every '''10 minutes''' to copy~~ the ~~data to '''X:\CCMDB\PHI'''~~.

We store personal health information separately from our other data. This process is par of managing that data.

We can't have [[PHI.mdb]] on the Regional server because too many people have access and we can't log the access.

If the ~~task~~ runs and there ~~are~~ no ~~problems~~ the task ~~will list "0x0" in~~ the ~~last results column~~. ~~Any other code would require review~~.

==Process==

* During [[sending]], [[CCMDB.accdb]] exports PHI files into {{Regionalserver|Output\_PHI}} .

* Running [[Pull]] or manually running it will run [[Copy PHI.bat]]

* [[Copy PHI.bat]] copies the files from {{Regionalserver|Output\_PHI}} to {{S:\MED\MED_CCMED}}CCMDB\PHI. Then it calls the script [[Load Data.vbs]] to integrate them into [[PHI.mdb]].

* [[Load Data.vbs]] accesses [[PHI Loader.accdb]] and runs the sub ''loadAllCSVFiles'', which calls the code that imports all csv files found within the directory. The csv loading code also moves the files again into an archive folder {{S:\MED\MED_CCMED}}CCMDB\PHI\PHI Archive\ to prevent duplicate processing.

=== Running this manually ===

* navigate to {{S:\MED\MED_CCMED}}CCMDB\PHI'''

* double-click [[copy_PHI.bat]] and let it finish. May take a few (<10) minutes, depending on the build-up of files

* check/refresh {{S:\MED\MED_CCMED}}CCMDB\PHI''' to make sure no *.csv files remain in that directory

** if any are left, troubleshoot; files that remain in there have likely not been integrated into [[PHI.mdb]]

{{DT| there is a data processing instruction for this on the wiki, no time to find it right now. }}

== Future of this process ==

Depending how PHI is stored in the future, [[Re-platforming]] might make this obsolete.

{{LegacyContent

| explanation=This used to be a scheduled, automated process, but we stopped that because we spent much time troubleshooting. It is now run as part of [push pull]] only.

| content=

This used to be set up as a scheduled task on the [[Data Processor PC]] before COVID. It ran the batch file [[Copy PHI.bat]] every 60 minutes. See [[Automatic updating of MS Access Databases using scheduled tasks]] for more info on how this works.

~~Hmmm? [[Centralized data~~.~~mdb]] vs [[Aggregator~~.~~mdb]]~~

This process would lock up occasionally preventing collectors from sending. So, we stopped the scheduled task that ran this every 5 minutes. We were initially going to review this to make it work again, but with COVID a desktop scheduled task became entirely impractical anyway.

~~Which drive? There was talk about using M:\?~~

If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.

== ~~Problem~~ ==

== If the automation were not working... ==

~~Tested~~ this ~~today and~~ data ~~is not being moved from regional~~ server. ~~Trish, were you logged out? You were not here. Need to discuss action plan to make sure data never piles up in there~~. ~~Ttenbergen 16:30, 2013 July 2 (CDT)~~

If the [[Data Processor PC]] that this is scheduled on isn't running (shutdown) then data might be left on the server for some time. Probably OK for now since it would still only be temporary.

~~Working now~~... ~~Still wondering if~~ this ~~was down because Trish was logged out~~. ~~Trish, could you confirm that you leave your PC running and locked when you are away? Ttenbergen 12:21, 2013 July 25 (CDT)~~

==If X drive goes down==

If main office notices that X drives is down, Data processor must not remove the csv files from the Regional server until:

* computer is rebooted and network connection to X is checked to see if it has been re established, otherwise this bat files is not able to do its job.

**occurrence: Monday June 15.14 at 1700 hrs and all day June 16.14: issue: Catastrophic hardware failure on this network drive had occurred.

}}

== Related Articles ==

[[Category: 2013 data upgrades]]

[[Category:2013 data upgrades]]

[[Category:PHI]]

@@ Line 1: / Line 1: @@
-As part of the [[2013 data upgrades]], [[CCMDB.mdb]] will now send [[some data]] to the [[Regional Server]]\Output\_PHI ("_" to sort separate from hospitals) which will be picked up by a scheduled task on Trish's computer calling '''X:\CCMDB\PHI\[[copy_PHI.bat]]''' every '''10 minutes''' to copy the data to '''X:\CCMDB\PHI'''.
+We store personal health information separately from our other data. This process is par of managing that data.
+We can't have [[PHI.mdb]] on the Regional server because too many people have access and we can't log the access.
-If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.
+==Process==
+* During [[sending]], [[CCMDB.accdb]] exports PHI files into {{Regionalserver|Output\_PHI}} .
+* Running [[Pull]] or manually running it will run [[Copy PHI.bat]]
+* [[Copy PHI.bat]] copies the files from {{Regionalserver|Output\_PHI}} to {{S:\MED\MED_CCMED}}CCMDB\PHI. Then it calls the script [[Load Data.vbs]] to integrate them into [[PHI.mdb]].
+* [[Load Data.vbs]] accesses [[PHI Loader.accdb]] and runs the sub ''loadAllCSVFiles'', which calls the code that imports all csv files found within the directory.  The csv loading code also moves the files again into an archive folder {{S:\MED\MED_CCMED}}CCMDB\PHI\PHI Archive\ to prevent duplicate processing.
+=== Running this manually ===
+* navigate to {{S:\MED\MED_CCMED}}CCMDB\PHI'''
+* double-click [[copy_PHI.bat]] and let it finish. May take a few (<10) minutes, depending on the build-up of files
+* check/refresh {{S:\MED\MED_CCMED}}CCMDB\PHI''' to make sure no *.csv files remain in that directory
+** if any are left, troubleshoot; files that remain in there have likely not been integrated into [[PHI.mdb]]
+{{DT| there is a data processing instruction for this on the wiki, no time to find it right now.  }}
+== Future of this process ==
+Depending how PHI is stored in the future, [[Re-platforming]] might make this obsolete.
+{{LegacyContent
+| explanation=This used to be a scheduled, automated process, but we stopped that because we spent much time troubleshooting. It is now run as part of [push pull]] only.
+| content=
+This used to be set up as a scheduled task on the [[Data Processor PC]] before COVID. It ran the batch file [[Copy PHI.bat]] every 60 minutes. See [[Automatic updating of MS Access Databases using scheduled tasks]] for more info on how this works.
-Hmmm? [[Centralized data.mdb]] vs [[Aggregator.mdb]]
+This process would lock up occasionally preventing collectors from sending. So, we stopped the scheduled task that ran this every 5 minutes. We were initially going to review this to make it work again, but with COVID a desktop scheduled task became entirely impractical anyway.
-Which drive? There was talk about using M:\?
+If the task runs and there are no problems the task will list "0x0" in the last results column. Any other code would require review.
-== Problem ==
+== If the automation were not working... ==
-Tested this today and data is not being moved from regional server. Trish, were you logged out? You were not here. Need to discuss action plan to make sure data never piles up in there. Ttenbergen 16:30, 2013 July 2 (CDT)
+If the [[Data Processor PC]] that this is scheduled on isn't running (shutdown) then data might be left on the server for some time. Probably OK for now since it would still only be temporary.
-Working now... Still wondering if this was down because Trish was logged out. Trish, could you confirm that you leave your PC running and locked when you are away? Ttenbergen 12:21, 2013 July 25 (CDT)
+==If X drive goes down==
+If main office notices that X drives is down, Data processor must not remove the csv files from the Regional server until:
+* computer is rebooted and network connection to X is checked to see if it has been re established, otherwise this bat files is not able to do its job.
+**occurrence: Monday June 15.14 at 1700 hrs and all day June 16.14: issue: Catastrophic hardware failure on this network drive had occurred.
+}}
+== Related Articles ==
+{{Related Articles}}
-[[Category: 2013 data upgrades]]
+[[Category:2013 data upgrades]]
+[[Category:PHI]]