Changes to make Centralized data smaller: Difference between revisions

From CCMDB Wiki
Jump to navigation Jump to search
← Older edit
mNo edit summary
mNo edit summary
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
As of 2022-08-10 [[Centralized data.mdb]] was 1.348GB . This is large and slowly approaching MS Access' 2GB limit. The size also makes it slow to copy down, and even slower to copy up. This page is about attempts to make the file smaller.  
As of 2022-08-10 [[Centralized data.mdb]] was 1.348GB . This is large and slowly approaching MS Access' 2GB limit. The size also made it slow to copy down, and even slower to copy up. We made some changes to reduce the size by [[#Removing the L_Dxs table and L_Como table]] and [[#Reducing fields and field sizes in the Postal_Code_Master table]]. This initiative is now complete.  


=== [[L_Dxs table]] and [[L_Como table]] ===
=== Removing the [[L_Dxs table]] and [[L_Como table]] ===
We can reduce the file size to 0.810GB by moving the legacy tables [[L_Dxs table]] and [[L_Como table]] to [[Legacy TMS Data.mdb]]. These tables are “stale” and no longer change, so we don’t really need to have them in [[Centralized data.mdb]].  
We can reduce the file size to 0.810GB by moving the legacy tables [[L_Dxs table]] and [[L_Como table]] to [[Legacy TMS Data.mdb]]. These tables are “stale” and no longer change, so we don’t really need to have them in [[Centralized data.mdb]].  
==== Next Steps ====
{{Todo
| who = Tina
| todo_added = 2022-08-10
| todo_action = 2022-08-10
| question = 
* update CFE to stop pulling these tables from Centralized (need to find out if CFE really still needs these at all)
}}


==== Log ====
==== Log ====
* 2022-08-31: Teams with Pagasa where we removed the [[L_Dxs table]] and [[L_Como table]] from [[Centralized data.mdb]]
* 2022-08-10:  
* 2022-08-10:  
** emailed Pagasa to copy Legacy to local
** updated CFE re-link to these in [[Legacy_TMS_Data.mdb]] rather than in  [[Centralized data.mdb]]
** emailed Julie and confirmed we should do this
** emailed Julie and confirmed we should do this
** imported [[L Dxs table]] and [[L Como table]] into [[Legacy TMS Data.mdb]] as version 2022-08-10  
** imported [[L Dxs table]] and [[L Como table]] into [[Legacy TMS Data.mdb]] as version 2022-08-10  
{{DJ |
** Julie copied the  [[Legacy_TMS_Data.mdb]] to local where SAS will pull from (08Aug2022 )
** asked Julie to change her SAS to pull them from there and update here when done
}}


=== Postal_Code_Master table ===
=== Reducing fields and field sizes in the [[Postal_Code_Master table]] ===
Further, we have a [[Postal_Code_Master table]] in there. It’s in Centralized Data only because Pagasa maintains it, so if it was in CFE it would get lost if I make an update to that file. But there is no reason it couldn’t live in a different file, that would just have to be added to the push pull machinery for Pagasa. Removing this table further shrinks the file to 0.721GB
There is a [[Postal_Code_Master table]] in [[Centralized data.mdb]]. The table is likely in the correct Access file because Pagasa maintains it and Julie uses it.  
 
==== Next Steps ====
{{Todo
| who = Tina
| todo_added = 2022-08-10
| todo_action = 2022-08-10
| question = 
* Tina to figure out
}}


==== Log ====
==== Log ====
 
* removed "code" fields that duplicate "name" field data, and reduced field sizes as much as possible, which reduced table size from ~89MB to ~70MB.


== Related articles ==  
== Related articles ==  

Latest revision as of 14:19, 2022 September 1

As of 2022-08-10 Centralized data.mdb was 1.348GB . This is large and slowly approaching MS Access' 2GB limit. The size also made it slow to copy down, and even slower to copy up. We made some changes to reduce the size by #Removing the L_Dxs table and L_Como table and #Reducing fields and field sizes in the Postal_Code_Master table. This initiative is now complete.

Removing the L_Dxs table and L_Como table

We can reduce the file size to 0.810GB by moving the legacy tables L_Dxs table and L_Como table to Legacy TMS Data.mdb. These tables are “stale” and no longer change, so we don’t really need to have them in Centralized data.mdb.

Log

Reducing fields and field sizes in the Postal_Code_Master table

There is a Postal_Code_Master table in Centralized data.mdb. The table is likely in the correct Access file because Pagasa maintains it and Julie uses it.

Log

  • removed "code" fields that duplicate "name" field data, and reduced field sizes as much as possible, which reduced table size from ~89MB to ~70MB.

Related articles

Related articles:
Retrieved from "https://ccmdb.kuality.ca/index.php?title=Changes_to_make_Centralized_data_smaller&oldid=157358"