Centralized data.mdb: Difference between revisions

From CCMDB Wiki
Jump to navigation Jump to search
JMojica (talk | contribs)
No edit summary
Line 13: Line 13:
* "Send_Centralized_append" (append new records from the collector's ccmdb_data.mdb)
* "Send_Centralized_append" (append new records from the collector's ccmdb_data.mdb)
**  why  do we need 2 separate queries? - I think one is enough which will do both.  If the DC sends their profiles from their ccmdb_data.mdb to the Centralized_data.mdb, the updating and/or appending should both occur depending on whether the profiles are previously sent or not - see below for more details. One important data I need is the field that will differentiate the completed profiles from incomplete ones (i.e. outstanding/ patients who are still in unit/ward). If that is already present, I need to know the label name.  [[User:JMojica|JMojica]] 13:03, 2013 April 9 (EDT)
**  why  do we need 2 separate queries? - I think one is enough which will do both.  If the DC sends their profiles from their ccmdb_data.mdb to the Centralized_data.mdb, the updating and/or appending should both occur depending on whether the profiles are previously sent or not - see below for more details. One important data I need is the field that will differentiate the completed profiles from incomplete ones (i.e. outstanding/ patients who are still in unit/ward). If that is already present, I need to know the label name.  [[User:JMojica|JMojica]] 13:03, 2013 April 9 (EDT)
{{Discussion}}  
*** I use two queries instead of a field to state if final. First query updates fields if a matching record exists. Second query adds the record if a matching record does not exist. Same idea, just done differently. The dataset does not keep track of whether the patient is final-checked. Ttenbergen 14:06, 2013 April 9 (EDT){{Discussion}}  
 
* If this is working as intended then any updates Pagasa makes to fix bad data will be re-overwritten with the bad data on the collection laptop every time the collector sends. Possible remedies are to make collectors delete sooner after sending or getting Pagasa to email collectors to fix errors at the root and waiting for that to update the dataset. That is assuming it works as intended. From your experience with the entries in L_Log and the sending/data retention habits of the collectors involved, does it sound like this is how it is working? Ttenbergen 18:05, 2013 April 8 (EDT)
* If this is working as intended then any updates Pagasa makes to fix bad data will be re-overwritten with the bad data on the collection laptop every time the collector sends. Possible remedies are to make collectors delete sooner after sending or getting Pagasa to email collectors to fix errors at the root and waiting for that to update the dataset. That is assuming it works as intended. From your experience with the entries in L_Log and the sending/data retention habits of the collectors involved, does it sound like this is how it is working? Ttenbergen 18:05, 2013 April 8 (EDT)


*[[User:JMojica|JMojica]] 11:24, 2013 April 9 (EDT) Here is my suggestion -  Populate the Centralized_data.mdb from DC's laptop at sending time (once a week), not when the DCs back-up their data at laptop .  The data will include both those checked completed and those incomplete.   
*[[User:JMojica|JMojica]] 11:24, 2013 April 9 (EDT) Here is my suggestion -  Populate the Centralized_data.mdb from DC's laptop at sending time (once a week), not when the DCs back-up their data at laptop .  The data will include both those checked completed and those incomplete.   
**On next's time sending (normally the following week or whatever frequency we decide), DC must make sure that the previously sent 'completed' are already removed from their laptop. This is required to avoid overwriting the data already in the Centralized_data.mdb which may have been changed by Pagasa due to some errors.
** It's already done at send time only. Ttenbergen 14:06, 2013 April 9 (EDT)
***There will be again a batch of completed profiles, some are new and some are already included in the last week's sent; and another batch of incomplete profiles, some are new and some are already included in the last week's sent.  Upon sending, these will populate the Centralized_data.mdb.  Those previously incomplete profiles but are now complete will be overwritten (e.g. all data will be updated).  Those previously incomplete profiles but still incomplete in this sending will also be overwritten (e.g. some data will be updated). Those with no previous profiles (complete or incomplete) will just be appended.
 
**Finding Errors -  The integrity checks of data will be applied to both complete and incomplete profiles.  
*On next's time sending (normally the following week or whatever frequency we decide), DC must make sure that the previously sent 'completed' are already removed from their laptop. This is required to avoid overwriting the data already in the Centralized_data.mdb which may have been changed by Pagasa due to some errors.
**Fixing Errors - Pagasa will do the fixing of errors in the Centralized_data.mdb but only for those with complete profiles.  If Pagasa finds errors in the data of the incomplete profiles, she will tell the error(s) to the corresponding DC, ask her the correction and the DC  will make any correction(s) in the laptop herself.  The corrections for the incomplete profiles will be  shown later in the next sending. Pagasa will do nothing in the incomplete profiles of Centralized_data.mdb .  
** I am open to that if there are no collection concerns from Pagasa and Trish, and if they are willing to push that through with collectors. Ttenbergen 14:06, 2013 April 9 (EDT)
***if I need to use the data from the incomplete profiles of the  Centralized_data.mdb and if there are some errors, I will substitute the erroneous value with the corrected value (provided by DC or Pagasa) in the SAS program in order to generate a correct report.  Once the correction has been sent to the Centralized_data.mdb, I will just change the SAS program.
 
*There will be again a batch of completed profiles, some are new and some are already included in the last week's sent; and another batch of incomplete profiles, some are new and some are already included in the last week's sent.  Upon sending, these will populate the Centralized_data.mdb.  Those previously incomplete profiles but are now complete will be overwritten (e.g. all data will be updated).   
* Those previously incomplete profiles but still incomplete in this sending will also be overwritten (e.g. some data will be updated).
** Both of those are what the update query does. It does not distinguish between complete/incomplete, doesn't need to. Ttenbergen 14:06, 2013 April 9 (EDT)
 
* Those with no previous profiles (complete or incomplete) will just be appended.
** That's what the append query does.
 
* Finding Errors -  The integrity checks of data will be applied to both complete and incomplete profiles.  
** We can do that. Ttenbergen 14:06, 2013 April 9 (EDT)
 
* Fixing Errors - Pagasa will do the fixing of errors in the Centralized_data.mdb but only for those with complete profiles.  If Pagasa finds errors in the data of the incomplete profiles, she will tell the error(s) to the corresponding DC, ask her the correction and the DC  will make any correction(s) in the laptop herself.  The corrections for the incomplete profiles will be  shown later in the next sending. Pagasa will do nothing in the incomplete profiles of Centralized_data.mdb.  
** This would mean we need to include the "FinalCheck" field (what tells CCMDB that it is complete and ready for sending) in the Centralized_data.mdb. I am starting to think I will need to include it, but I was hoping to change it from a boolean into a "status" field where we can designate the record as "incomplete", "sent", "finalized", and possibly "rejected" if we ever get to a point of pushing sent data back to collectors for fixing. Ttenbergen 14:06, 2013 April 9 (EDT)
 
*if I need to use the data from the incomplete profiles of the  Centralized_data.mdb and if there are some errors, I will substitute the erroneous value with the corrected value (provided by DC or Pagasa) in the SAS program in order to generate a correct report.  Once the correction has been sent to the Centralized_data.mdb, I will just change the SAS program.
** sounds good. Ttenbergen 14:06, 2013 April 9 (EDT)


== How to populate this ==
{{Discussion}}
We talked today at the ICU Task Meeting about whether or not to populate this table by push at every send. Julie was concerned that we would still not have as up-to-date data that way as we might be able to. One suggestion was to pull it from the backups instead. This could be done, but would need to happen very regularly to not miss patients who, e.g. arrive and get discharged on Wednesday morning, get entered by the collector, sent and deleted. Might need a multi-prong approach, where this table is push-populaetd when collectors send, '''and''' pull populated whenever Julie needs data. That way no patients would get missed and fresh data would be available when needed. I am planning to populate this table with a pair of queries: first one updating records that are already present, second one adding records that are new. Thoughts? Ttenbergen 17:09, 2012 December 21 (EST)
*I realized we just need to simplify our process and I think the suggestion above  will be sufficient. [[User:JMojica|JMojica]] 13:16, 2013 April 9 (EDT)


== Dependencies ==
== Dependencies ==

Revision as of 12:06, 2013 April 9

Data Repository containing:

Sending to ...

version 2012-12-21 implemented sending of data to Centralized_data.mdb. This is accomplished by queries

  • "Send_Centralized_update" (update existing records to data currently in the collector's ccmdb_data.mdb)
  • "Send_Centralized_append" (append new records from the collector's ccmdb_data.mdb)
    • why do we need 2 separate queries? - I think one is enough which will do both. If the DC sends their profiles from their ccmdb_data.mdb to the Centralized_data.mdb, the updating and/or appending should both occur depending on whether the profiles are previously sent or not - see below for more details. One important data I need is the field that will differentiate the completed profiles from incomplete ones (i.e. outstanding/ patients who are still in unit/ward). If that is already present, I need to know the label name. JMojica 13:03, 2013 April 9 (EDT)
      • I use two queries instead of a field to state if final. First query updates fields if a matching record exists. Second query adds the record if a matching record does not exist. Same idea, just done differently. The dataset does not keep track of whether the patient is final-checked. Ttenbergen 14:06, 2013 April 9 (EDT)Template:Discussion
  • If this is working as intended then any updates Pagasa makes to fix bad data will be re-overwritten with the bad data on the collection laptop every time the collector sends. Possible remedies are to make collectors delete sooner after sending or getting Pagasa to email collectors to fix errors at the root and waiting for that to update the dataset. That is assuming it works as intended. From your experience with the entries in L_Log and the sending/data retention habits of the collectors involved, does it sound like this is how it is working? Ttenbergen 18:05, 2013 April 8 (EDT)
  • JMojica 11:24, 2013 April 9 (EDT) Here is my suggestion - Populate the Centralized_data.mdb from DC's laptop at sending time (once a week), not when the DCs back-up their data at laptop . The data will include both those checked completed and those incomplete.
    • It's already done at send time only. Ttenbergen 14:06, 2013 April 9 (EDT)
  • On next's time sending (normally the following week or whatever frequency we decide), DC must make sure that the previously sent 'completed' are already removed from their laptop. This is required to avoid overwriting the data already in the Centralized_data.mdb which may have been changed by Pagasa due to some errors.
    • I am open to that if there are no collection concerns from Pagasa and Trish, and if they are willing to push that through with collectors. Ttenbergen 14:06, 2013 April 9 (EDT)
  • There will be again a batch of completed profiles, some are new and some are already included in the last week's sent; and another batch of incomplete profiles, some are new and some are already included in the last week's sent. Upon sending, these will populate the Centralized_data.mdb. Those previously incomplete profiles but are now complete will be overwritten (e.g. all data will be updated).
  • Those previously incomplete profiles but still incomplete in this sending will also be overwritten (e.g. some data will be updated).
    • Both of those are what the update query does. It does not distinguish between complete/incomplete, doesn't need to. Ttenbergen 14:06, 2013 April 9 (EDT)
  • Those with no previous profiles (complete or incomplete) will just be appended.
    • That's what the append query does.
  • Finding Errors - The integrity checks of data will be applied to both complete and incomplete profiles.
    • We can do that. Ttenbergen 14:06, 2013 April 9 (EDT)
  • Fixing Errors - Pagasa will do the fixing of errors in the Centralized_data.mdb but only for those with complete profiles. If Pagasa finds errors in the data of the incomplete profiles, she will tell the error(s) to the corresponding DC, ask her the correction and the DC will make any correction(s) in the laptop herself. The corrections for the incomplete profiles will be shown later in the next sending. Pagasa will do nothing in the incomplete profiles of Centralized_data.mdb.
    • This would mean we need to include the "FinalCheck" field (what tells CCMDB that it is complete and ready for sending) in the Centralized_data.mdb. I am starting to think I will need to include it, but I was hoping to change it from a boolean into a "status" field where we can designate the record as "incomplete", "sent", "finalized", and possibly "rejected" if we ever get to a point of pushing sent data back to collectors for fixing. Ttenbergen 14:06, 2013 April 9 (EDT)
  • if I need to use the data from the incomplete profiles of the Centralized_data.mdb and if there are some errors, I will substitute the erroneous value with the corrected value (provided by DC or Pagasa) in the SAS program in order to generate a correct report. Once the correction has been sent to the Centralized_data.mdb, I will just change the SAS program.
    • sounds good. Ttenbergen 14:06, 2013 April 9 (EDT)


Dependencies

TISS28 Data.mdb pulls data from Centralized data.mdb.