Import Log: Difference between revisions

From CCMDB Wiki
Jump to navigation Jump to search
← Older edit
TOstryzniuk (talk | contribs)
CCMDB_legit, approved, Bureaucrats, Administrators
27,522 edits
m Text replacement - "ccmdb.mdb" to "ccmdb.accdb"
 
(40 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This is a log files that is created automatically by the [[PDA Uploader]].exe. When a csv file from a data collection site is uploaded to the master database (for Medicine [[MedTMS]] or Critical Care[[TMSX]]), by the data processor, the PDA uploader creates a log file which records the following information:
{{LegacyContent
|explanation=No longer importing anything into TMSX or even using it.
|successor=
|content=
 
Also referred to as the [[AppendLog]]. 
 
This is a log files that is created automatically by the [[PDA Reader]]/[[PDA Uploader]]exe. every time the [[Data Processor]] uploads (appends) most of the sent csv files to either the Master Critical Care([[TMSX]]) database or the Master Medicine ([[MedTMS]])  [[:Category: Data Repository | Data Repository]]. 
 
Other parts of the data that is sent are automatically appended other [[:Category: Data Repository | Data Repositories]]. 
that reside on the [[Regional Server]].
==Information on Import Log==
*append batch number
*append batch number
*name of CSV file that was sent from site: e.g. 50_HSC_MICU_GH.txt
*name of CSV file that was sent from site to [[Regional Server]]: e.g. 50_HSC_MICU_GH.txt
*import date
*import date
*number of files in the batch sent from a site
*number of files in the batch sent from a site
*start or first study number in main database that the first file in batch appended to.
*start or first database [[Study number]] in main database that the first file in the batch appends to.
*end or last study number in main database that last file in batch was appended to.
*end of last database [[Study number]] in main database that last file in the batch appends to.
 
==Weekly printout==
The data processor prints out the Import Log report each week when she appends data to master database.
 
==Yearly changes==
At the beginning of each New Year, the import log file is renamed by the data processor and the record is kept in backup folder for that year.  A new log file will automatically be created with the append batch number starting back at number one.  This is done to avoid large numeric batch numbers.
 
==Location of Import log file==
===Paper Import log file===
A paper print out is done every week after appending data to master database and the paper is stored in an binder in the office.
*Blue label binder for ICU -paper printout
*Red label binder for MED -paper printout


===Electronic Import log file===
The automatically created log file is located on the local C drive of main data processor's computer:


At the beginning of the new year, this log file is renamed and and new log file starts with the append batch number starting back at number one.  This is done because the append number would become to huge.
Critical Care [[TMSX]] database:
*C:\CritPDAdnld\[[Import Log]]\'''CriticalLog.log'''


==Location of log file==
Medicine database [[MedTMS]]:
Is located on the local C drive of main data processor's computer.
*C:\MEDPDAdnld\[[Import Log]]\'''MedicalLog.log'''


Critical Care:
*C:\CritPDAdnld\ImportLog\CriticalLog.log


Medicine:
The import log is not readable in current format.  September 6.12 - The Database Programmer is working on changing [[PDA Uploader]].exe for both [[TMSX]] & [[MedTMS]] so that the import log is saved as a CSV (more user friendly file).    Currently the import log has some machine language written into it which makes rather unreadable. The only way this log is legible is if [[Data Processor]] executes a printout from the [[PDA Reader]]/[[PDA Uploader]].exe.  We will stop printing and storing these paper logs in binders above once update is completed.  Will have this in place before end of 2012. [[User:TOstryzniuk|Trish Ostryzniuk]] 18:57, 2012 October 25 (EDT)
*C:\MedPDAdnld\ImportLog\MedLog.log


==Paper Record==
*There will be a maximum of 40 characters allowed in the label name for sent CSV data file.  If we want electronic log, current PDA reader has a problem with sent files names longer than 16 characters.   It truncates the label name when it reads in the log files names. 40 character max will give us some flexibilty and we will than not have to be so strict with these files names, example - "re-sent data" where we typed in added info into send file so Pagasa is clear it is a resend or test also some staff do not use only two letters for their intials when sending files.- [[User:TOstryzniuk|Trish Ostryzniuk]] 20:57, 2012 October 25 (EDT)
A print out is done every week and stored in an binder in the office. Once for Critical Care and another one for Medicine.
*note: September 6.12 - The Database Programmer is working on changing [[PDA Uploader]].exe so that the import log is saved as a CSV.  Currently log has some machine language written into it which makes it unreadable.  We cannot view, print or export it. Only the PDA Uploader can execute it to output in readable format.
When programmer makes the changes we will no long print & store these logs in binders.[[User:TOstryzniuk|Trish Ostryzniuk]] 19:33, 2012 September 6 (CDT)


==Use of Import log==
==Use of Import log==
The import log (append log) provides the main office with quick and easy way for to locate and "original sent cvs file" from the backup files on M. 
When errors are found in master database by the [[Statistician]], we alway try and trace where the error may originating from in order to decide if error is avoidable or not.  We investigate if error was due to:
#collection error
#ccmdb.accdb problem (entry field limits or sending)
#appending error
#data Editing error
We must look at the original sent csv file, to see where error may have originated from before we pursue trying resolve problem.    We will then review collection guidelines on Wiki and update and also discuss with collector.  For ccmdb we notify [[User: Ttenbergen | Tina]] if there are changes that can be to be made to avoid error.  If due to appending we notify programmer for [[TMSX]] or [[MedTMS]].  If error is due to in office editing by [[Data Processor]] we review process.


[[Category:Legacy Data Processing]]
[[Category:Master Database Programs]]
[[Category:Master Database Programs]]
}}

Latest revision as of 09:55, 2020 June 10

Legacy Content

This page contains Legacy Content.
  • Explanation: No longer importing anything into TMSX or even using it.
  • Successor:

Click Expand to show legacy content.

Also referred to as the AppendLog.

This is a log files that is created automatically by the PDA Reader/PDA Uploaderexe. every time the Data Processor uploads (appends) most of the sent csv files to either the Master Critical Care(TMSX) database or the Master Medicine (MedTMS) Data Repository.

Other parts of the data that is sent are automatically appended other Data Repositories. that reside on the Regional Server.

Information on Import Log

  • append batch number
  • name of CSV file that was sent from site to Regional Server: e.g. 50_HSC_MICU_GH.txt
  • import date
  • number of files in the batch sent from a site
  • start or first database Study number in main database that the first file in the batch appends to.
  • end of last database Study number in main database that last file in the batch appends to.

Weekly printout

The data processor prints out the Import Log report each week when she appends data to master database.

Yearly changes

At the beginning of each New Year, the import log file is renamed by the data processor and the record is kept in backup folder for that year. A new log file will automatically be created with the append batch number starting back at number one. This is done to avoid large numeric batch numbers.

Location of Import log file

Paper Import log file

A paper print out is done every week after appending data to master database and the paper is stored in an binder in the office.

  • Blue label binder for ICU -paper printout
  • Red label binder for MED -paper printout

Electronic Import log file

The automatically created log file is located on the local C drive of main data processor's computer:

Critical Care TMSX database:

Medicine database MedTMS:


The import log is not readable in current format. September 6.12 - The Database Programmer is working on changing PDA Uploader.exe for both TMSX & MedTMS so that the import log is saved as a CSV (more user friendly file). Currently the import log has some machine language written into it which makes rather unreadable. The only way this log is legible is if Data Processor executes a printout from the PDA Reader/PDA Uploader.exe. We will stop printing and storing these paper logs in binders above once update is completed. Will have this in place before end of 2012. Trish Ostryzniuk 18:57, 2012 October 25 (EDT)

  • There will be a maximum of 40 characters allowed in the label name for sent CSV data file. If we want electronic log, current PDA reader has a problem with sent files names longer than 16 characters. It truncates the label name when it reads in the log files names. 40 character max will give us some flexibilty and we will than not have to be so strict with these files names, example - "re-sent data" where we typed in added info into send file so Pagasa is clear it is a resend or test also some staff do not use only two letters for their intials when sending files.- Trish Ostryzniuk 20:57, 2012 October 25 (EDT)

Use of Import log

The import log (append log) provides the main office with quick and easy way for to locate and "original sent cvs file" from the backup files on M.

When errors are found in master database by the Statistician, we alway try and trace where the error may originating from in order to decide if error is avoidable or not. We investigate if error was due to:

  1. collection error
  2. ccmdb.accdb problem (entry field limits or sending)
  3. appending error
  4. data Editing error

We must look at the original sent csv file, to see where error may have originated from before we pursue trying resolve problem. We will then review collection guidelines on Wiki and update and also discuss with collector. For ccmdb we notify Tina if there are changes that can be to be made to avoid error. If due to appending we notify programmer for TMSX or MedTMS. If error is due to in office editing by Data Processor we review process.

Retrieved from "https://ccmdb.kuality.ca/index.php?title=Import_Log&oldid=144692"