Category Archives: Qlik Sense

Productivity, Qlik Sense, QlikView, Scripting, Tools

Storing a Data Model in a Single QVD

18 Comments

Have you ever thought it might be interesting to store a  Qlik data model into a single QVD?  This can be useful in a number of cases such as:

  • Archiving (and retrieving) data models.
  • Overcoming the “single binary load” restriction.

QlikView Components (QVC) Version 11 introduced two new routines to do just that:

Qvc.ExportModel — Exports all tables of the current model into a single QVD.

Qvc.ImportModel — Import a data model created by Qvc.ExportModel.

Even if you don’t have QVC V11 installed, you can try Qvc.ExportModel right now using  http include.  Add these lines to any QlikView script (instructions for Qlik Sense further on down in this post).

$(Must_Include=https://github.com/RobWunderlich/Qlikview-Components/releases/download/v11.1/Qvc.qvs);
 CALL Qvc.ExportModel
 CALL Qvc.Cleanup

Mind the wrap. The Must_Include should be on one line. Using QVC requires the Qvc.qvs library be included (usually at the beginning of script), CALLing Qvc routines, and CALLing a Cleanup routine at the end of your script.

Assuming this script is included in “Sales Dash.qvw”,  the default exported model QVD will be named “Sales Dash.qvd” in the same directory.

 

Now, to import this QVD model into another qvw, replace the CALL to ExportModel in the above sample with:

CALL Qvc.ImportModel('Sales Dash.qvd')

The original model will be reconstructed as individual tables.

Qvc.ExportModel has three optional parameters:

CALL Qvc.ExportModel(['qvddir'],['qvdname'],['addTimestamp']);
Parameter Number Parameter Description
1 String. Optional. Relative or absolute directory where the model QVD will be stored. If relative, it follows the same rules as the STORE script statement for relative directory.
2 String, Optional. Name for the model QVD. If omitted, the name of the QVW will be used. For example, if QVW is “Sales.qvw”, then QVD will be “Sales.qvd”.
3 String, Optional. 1/0 True/False. If True, a timestamp of the form _YYYYMMDDhhmmss will be appended to the QVD name. Default if omitted is False.

 

Qlik Sense has no default path  so parameter #1, a lib:// for the QVD should be specified.  Alternatively, if a lib has been established with a DIRECTORY statement, parameter 1 can be omitted.

Qlik Sense will require a web file Connection for the http Must_Include.

webfile-connection

After defining the web connection and having an appropriate folder connection to store the QVD in,  Qlik Sense script would look like this:

$(Must_Include=lib://QvcWeb);
 Call Qvc.ExportModel('lib://QVData')
 CALL Qvc.Cleanup;

 

That’s all there is to it!  If you are already using QVC, I hope you’ll find these routines a welcome addition to the library.  If you are new to QC, explore more at QlikviewComponents.org.

-Rob

Thanks to Jörgen Peerik for raising the single-QVD export idea during a QVC class. 

 

Share
Productivity, Qlik Sense, QlikView, Scripting

Touchless Formatting

25 Comments

Summary: I show a scripting technique to assign display formats to loaded data without touching existing load statements. 

I coded in  SAS for many years and always appreciated the FORMAT statement which allows  assigning a display format to a field, independent of loading the field.

FORMAT OrderDate MM/DD/YYYY;

In QlikView and Qlik Sense script, there is an  equivalent that is useful to be aware of.   It’s not a statement, but a little known trick (so little known I’ve never seen anyone but me do it, although I’m sure others have thought if it).

// Load some dummy fields just to assign formats
TempFormatTable:
LOAD
 Date(0, 'MM/DD/YYYY') as OrderDate,
 Date(0, 'MM/DD/YYYY') as ShipDate,
 Num(0, '00000') as PostalCode,
 Num(0, '#,##0.00') as OrderTotal
AutoGenerate 1;

Facts: // Load the QVD
LOAD * FROM data1.qvd (qvd);

DROP TABLE TempFormatTable;  // Drop temp table

The formats assigned in the TempFormatTable will be inherited by any like-named fields in the QVD Load.   I sometimes find this easier than adding formatting function to the QVD Load statement because:

  • It maintains the optimized QVD load.
  • I can include a master list in the TempFormatTable. There is no error if a field doesn’t exist in the QVD.
  • Syntactically simpler.
  • I don’t touch the existing Load statement.

I don’t always format this way, but there are a number of scenarios where the technique is useful. A common application is to change formats from one locale to another. For example, loading a QVD created in Europe (with European formats) and assigning US Date and Number formats.

The technique works for any input source;  SQL, QVD, xls, etc. It works for both QlikView and Qlik Sense.

You may not ever need this tip, but if you do, I hope it saves you some time and makes your coding easier.

-Rob

Want more Tips & Tricks? Join me at an upcoming Masters Summit for Qlik event in Johannesburg (6-8 Sept) or Austin (10-12 Oct).  In addition to our two days of core sessions, Bill Lay’s “Tips & Tricks” on Day 3 always teaches me something new.  

Share
Qlik Sense, QlikView, Scripting, Visualization

QV12 Timestamp Parsing

8 Comments

Have you noticed something new in QlikView12 and Qlik Sense timestamp parsing? UTC timestamps are automatically understood.

(Note: the output displayed below utilizes the US Date format set in the script as:  SET TimestampFormat=’M/D/YYYY h:mm:ss[.fff] TT’;)

For example, the expression:

=Timestamp('20160504T142523.487-0500')

returns:

5/4/2016 7:25:23 PM

That is, the UTC offset of “-0500” is detected and the returned value is the UTC time, not the local time of 2:25:23 PM.

I can’t find anything in the help beyond an example  for Timestamp# that demonstrates this but provides no detail.

This parsing functionality is particularly useful now that the QlikView Server logfiles use the UTC format for times.

I’m not sure yet if I like the automatic conversion to UTC time.  For example, apps like the QlikView Governance Dashboard now report Session Start or Event times in UTC time, not local time.

It’s nice that the “T” character is understood. If you want local time, it’s easy enough to drop the offset (“-0500”) as

=Timestamp(left('20160504T142523.487-0500', 19))

which returns

5/4/2016 2:25:23 PM

-Rob

 

Share
Qlik Sense, QlikView, Visualization

Scoping Selections with Aggr()

10 Comments

Summary: Selections can be made in Calculated Dimensions, although the result may not always be what is expected or desired.   The Aggr() function can be used to control what field(s) get selected.

The technique discussed in this post applies to both QlikView and Qlik Sense.  The screenshots shown are from QlikView.  Some of the visuals are a bit different in Qlik Sense, but the idea and expressions demonstrated are the same.

A downloadable example to accompany this post is available here.

Consider a listbox created with an <Expression> value of:

=Customer & ' -- ' & Country

A listbox constructed this way is useful for providing additional context or an  additional search.

Selections made in that listbox will make underlying selections in both Customer and Country.

The user is probably  expecting a selection in Customer only.  To limit the selection to Customer, add an Aggr() function to the expression:

=aggr(
 Customer & ' -- ' & Country
 ,Customer)

Only the Customer field is listed in the Aggr() dimension, so selections will be made only in Customer.

A side effect of adding Aggr() is that gray (unassociated) rows no longer display.  We can fix that with a bit of Set Analysis.

=aggr(
 only({1<Customer={"*"}>} Customer & ' -- ' & Country)
 ,Customer)

Now the listbox looks and behaves as expected.

 

Another place you may need Aggr() to control selection intent is chart Calculated Dimensions.

Hovering over a Salesrep value in the chart below gives a contextual popup that identifies Manager and Hire Date associated with the Rep.

The column was created as a Calculated Dimension:

=SalesPerson
& chr(10) & 'Reports to ' & [Sales Manager]
& chr(10) & 'Hire Date ' & date(HireDate,'YYYY-MMM-DD')

Clicking Michelle in the chart correctly selects her name as SalesPerson, but makes unexpected selections in HireDate and SalesManager.

I’m going to say that the dimension is “improperly scoped” and correct it by adding Aggr() to the Calculated Dimension.

=Aggr(
 SalesPerson
 & chr(10) & 'Reports to ' & [Sales Manager]
 & chr(10) & 'Hire Date ' & date(HireDate,'YYYY-MMM-DD')
 ,SalesPerson)

Selections will now be correctly limited to the “SalesPerson” field.

 

We’ve seen that Aggr() can narrow selections. We can widen selections as well.  This listbox will make selections in Customer, Country, SalesPerson and Year, even though only Customer is displayed in the listbox.

=aggr(
 only({1}
 Customer
 )
 ,Customer, Country, SalesPerson, Year)

 

We don’t have to include the display field in the selections.  In what I’ll call a  “backdoor associative search” , this expression will display Customer, but selects only the OrderID values associated with the Customer.

=aggr(
only({1}Customer )
,OrderID)

It’s usually a best practice to pre-create Calculated Dimensions in the script, when possible, for performance reasons. Returning to our first example, we might create a new field in the script as:

Customer & ' -- ' & Country AS CustomerAndCountry

We can use the new field as a display value, but we want selections to be made in Customer.

=aggr(
 only({1}  CustomerAndCountry)
 ,Customer)

 

As a last example,  we can create  “bookmark” like alternatives; either new fields linked in the data model or advanced search at run time.

Here I’ve linked a hidden field named “Bookmark” into specific OrderIDs in the script.  I want selections to be reflected in the OrderID field.

=aggr(only({1}Bookmark), Bookmark,OrderID)

Here is an advanced search that presents a listbox of Customers who have placed at least one order with a value >50K.

=aggr(
only({1<OrderID={"=sum({1}OrderAmount)>50000"}>}Customer )
,Customer)

Aggr() can be a “heavy resource consumer” and has the potential to slow down your application. Use only when required and avoid using or benchmark the impact in large applications.  Calculated Dimensions can also be a source of slow performance, precalculate fields in the script when possible.

Download the  example qvw for this post .

-Rob

 

 

Share