In the latest release 2.4 of QV Document Analyzer I’ve started the process of adding “Recommendations”, highlighting specific areas where potential improvement may be made to your document.
The recommendation included in this release is “Numeric field sizes”. Numeric fields that occupy more than 8 bytes of storage per value will be flagged. They will be highlighted on the Fields sheet and additional details will be provided on the Recommendation sheet. The potential savings from “fixing” the field is also shown.
So what is this 8 byte thing? Ideally, most numeric fields (which includes timestamps) can be represented by 8 bytes of storage. In the Number format property or the script num() function you assign a formatting mask to be used for string representation. QV stores the 8 byte number and applies the mask at display time.
Occasionally, particularly when doing an un-optimized load from QVD, the data can wind up being saved as “Mixed” which means both the numeric and string representations are stored for each value. So you may see Symbol widths of 17, 20, 24 or even greater for this field.
In larger apps with many unique numeric values, I’ve found significant memory savings by restoring these Mixed fields to a numeric format.
The usual tuning and optimizing caveat applies here. Don’t spend effort recovering a few megabytes. You have better things to do. But if you are wrestling with the megabeast app, fixing the biggest of those fields may be worth the effort.
-Rob
I wasn’t aware of the mixed data type issue. Your recommendation to explicity format these fields as numeric has already paid off for me in one of my “megabeast” apps. Thanks!
Thanks a lot!!!!!!
Hi Rob, should we not use arithmetic operator (field +0 or *1) instead of the num function which is dual ?https://dataonthe.rocks/reducing-the-size-of-qvds-and-standardizing-content/
If creating QVDs I would agree that you should use the arithmetic operator as we may not want to store formatted values in the QVD. If creating a QVW/QVW, fields created with Num are stored as “implied duals” (https://qlikviewcookbook.com/2017/03/dual-storage-vs-dual-behavior/) and should not take any more space than numeric fields — except storing the format mask once.