All posts by Rob Wunderlich

Yoke Dashboard

I was chatting with a colleague recently about trends in BI and I brought up what I call the “commoditization of metrics” .  Google Analytics is an early example of this — your data crunched and delivered at the KPI level.

I recently ran across a great example of the commodity metrics idea:  Yoke.io.

Yoke let’s you build your own dashboard using metrics gleaned from cloud services such as Gmail, Twitter and Github. Here’s a portion of my Yoke dashboard. It’s all built with a few clicks and no coding.

Yoke Image

Give Yoke.io a try, it’s free!

-Rob

Join me at the upcoming “Masters Summit for Qlik” in Milan on 5-7 April. In addition to learning about all things Qlik, we’ll be talking about trends in Dashboarding and BI. 

Share

Scoping Selections with Aggr()

Summary: Selections can be made in Calculated Dimensions, although the result may not always be what is expected or desired.   The Aggr() function can be used to control what field(s) get selected.

The technique discussed in this post applies to both QlikView and Qlik Sense.  The screenshots shown are from QlikView.  Some of the visuals are a bit different in Qlik Sense, but the idea and expressions demonstrated are the same.

A downloadable example to accompany this post is available here.

Consider a listbox created with an <Expression> value of:

=Customer & ' -- ' & Country

A listbox constructed this way is useful for providing additional context or an  additional search.

Selections made in that listbox will make underlying selections in both Customer and Country.

The user is probably  expecting a selection in Customer only.  To limit the selection to Customer, add an Aggr() function to the expression:

=aggr(
 Customer & ' -- ' & Country
 ,Customer)

Only the Customer field is listed in the Aggr() dimension, so selections will be made only in Customer.

A side effect of adding Aggr() is that gray (unassociated) rows no longer display.  We can fix that with a bit of Set Analysis.

=aggr(
 only({1<Customer={"*"}>} Customer & ' -- ' & Country)
 ,Customer)

Now the listbox looks and behaves as expected.

 

Another place you may need Aggr() to control selection intent is chart Calculated Dimensions.

Hovering over a Salesrep value in the chart below gives a contextual popup that identifies Manager and Hire Date associated with the Rep.

The column was created as a Calculated Dimension:

=SalesPerson
& chr(10) & 'Reports to ' & [Sales Manager]
& chr(10) & 'Hire Date ' & date(HireDate,'YYYY-MMM-DD')

Clicking Michelle in the chart correctly selects her name as SalesPerson, but makes unexpected selections in HireDate and SalesManager.

I’m going to say that the dimension is “improperly scoped” and correct it by adding Aggr() to the Calculated Dimension.

=Aggr(
 SalesPerson
 & chr(10) & 'Reports to ' & [Sales Manager]
 & chr(10) & 'Hire Date ' & date(HireDate,'YYYY-MMM-DD')
 ,SalesPerson)

Selections will now be correctly limited to the “SalesPerson” field.

 

We’ve seen that Aggr() can narrow selections. We can widen selections as well.  This listbox will make selections in Customer, Country, SalesPerson and Year, even though only Customer is displayed in the listbox.

=aggr(
 only({1}
 Customer
 )
 ,Customer, Country, SalesPerson, Year)

 

We don’t have to include the display field in the selections.  In what I’ll call a  “backdoor associative search” , this expression will display Customer, but selects only the OrderID values associated with the Customer.

=aggr(
only({1}Customer )
,OrderID)

It’s usually a best practice to pre-create Calculated Dimensions in the script, when possible, for performance reasons. Returning to our first example, we might create a new field in the script as:

Customer & ' -- ' & Country AS CustomerAndCountry

We can use the new field as a display value, but we want selections to be made in Customer.

=aggr(
 only({1}  CustomerAndCountry)
 ,Customer)

 

As a last example,  we can create  “bookmark” like alternatives; either new fields linked in the data model or advanced search at run time.

Here I’ve linked a hidden field named “Bookmark” into specific OrderIDs in the script.  I want selections to be reflected in the OrderID field.

=aggr(only({1}Bookmark), Bookmark,OrderID)

Here is an advanced search that presents a listbox of Customers who have placed at least one order with a value >50K.

=aggr(
only({1<OrderID={"=sum({1}OrderAmount)>50000"}>}Customer )
,Customer)

Aggr() can be a “heavy resource consumer” and has the potential to slow down your application. Use only when required and avoid using or benchmark the impact in large applications.  Calculated Dimensions can also be a source of slow performance, precalculate fields in the script when possible.

Download the  example qvw for this post .

-Rob

 

 

Share

Preparing for QlikView Version 12

There are a number of reasons you may want to upgrade to QV12, I’ll be posting about them in the next few weeks. Today’s post is specifically about what you must address to upgrade to QV12.

There are two items I’m aware of that you must be aware of and consider before upgrading.

  1. QlikView version 12 Server changes the default permission for the  EXECUTE script statement More details in this post.
  2. In QlikView 12, QVD functions now respect the DIRECTORY statement. This is a breaking change that may cause some of your QV11 script to return incorrect results or take incorrect conditional actions. More information in a post here.

So again, I’ll be blogging about new features you may want to take advantage of,  but above are the only two items I know of that you must consider before upgrading.

How do you know if you are using the impacted EXECUTE statement or QVD* functions? Perhaps you already have a tool in place to answer that question. But if you don’t, my colleagues at the Masters Summit for Qlik are making the Script Repository Tool (“ScriptRepo”) available for public download.

If you have attended a Masters Summit, then you already have a copy of ScriptRepo  in your takeaways.  if not, we would love to have you attend a Summit. But the Masters Summit Team understands the universal need for a tool like ScriptRepo whether you can attend the Summit or not.

The “ScriptRepo” tool extracts script from a directory of QVWs and then allows you to do a global search for script of interest such as “Execute”. Plug in the directory path that contains qvw files,  and a temporary work directory to extract the script into:

384

Press the “Extract & Reload” button and each QVW file in the input directory will be opened and the script extracted. The open is done with “nodata”, so the extraction runs relatively fast — about one qvw per second.

After extraction, you can use your favorite search tool to scan the output directory, or use the builtin QlikView search & display from the “Script Search” sheet.

After searching for “customer” and selecting a candidate qvw:

276

 

I hope to see you at the Milan Summit in April 2016 or a USA location in Fall 2016.  If you have questions about the Summit or using the ScriptRepo tool, let us know through the QlikView Cookbook Contact form or the Masters Summit Contact page.

-Rob

Share

QV12: QVD Functions Respect Directory

Summary: QV12 introduces a “breaking change” in how file paths are interpreted by QVD File functions like QvdNoOfRecords.

In QlikView version 11 script, the five QVD file functions were not impacted by use of the Directory statement.  Relative paths given to these functions were always relative to the qvw file, not the current Directory setting.  This is different than the other file functions like FileSize() which considered paths to be relative to the current Directory setting.

In QlikView version 12, the QVD File functions have been changed to respect the Directory setting, making them consistent with the other functions.  I think this is a good idea.  However, it is a breaking change in that your QV11 script may no longer work in QV12 without changes.

Here is how you may have coded a script sequence in QV11:

DIRECTORY data;
STORE  Customer INTO Customer.qvd;
LET vCustomerRows =    QvdNoOfRecords('data\Customer.qvd');

This will not work in QV12. The function will be looking for the qvd in “data\data\Customer.qvd”.  There will be no script error, but the value of “vCustomerRows” will be null. Because QV12 respects the Directory, the correct syntax for QV12 would be “QvdNoOfRecords(‘Customer.qvd’).

This change is probably not a big deal for most customers, but it could be a silent irritant for many.  If your incremental reloads or other conditional code test for the existence of a QVD using QvdCreateTime(), the code could make the wrong decision leading to missing or corrupted data.

Next week I’ll show you a simple way to scan all your qvws for use of QVD functions or other script of interest.

-Rob

 

Share

Authorizing the Script EXECUTE Statement

Summary: QV Version 12 removes  per qvw control of the script EXECUTE Statement.  In QV12 Server, the default is to disallow all EXECUTE statements.  Any EXECUTE statement will fail unless you turn on the global setting to allow.

The QlikView  script “EXECUTE” statement provides the capability to run external programs from script.  Because EXECUTE may present a potential security risk, it’s use must be authorized. QlikView Version 12 changes how EXECUTE authorization is granted.

In QV11  Desktop,  the script editor Settings pane provides a checkbox labeled “Can Execute External Programs”. This property is set per qvw.

When an EXECUTE statement is encountered,  execution is allowed if the property is checked.  If not checked, a popup is presented asking for permission.

The “User Preference: Security: Always Override Security:  Script” may be checked on to bypass the Execute permission check.

If “Script” is checked, the execute statement will be allowed regardless of the “Can Execute External Programs” setting.

Authorizing Execute works differently in QV11 Server reloads.  In QV11 Server prior to SR11,  Execute statements were always allowed. The “Can Execute External Programs” property has no effect.

QV11 SR11+ Server introduced a new setting

"AllowExecuteCommand=0|1".

Setting the value to “0” prohibits Execute in any reload on the server, setting  to “1” allows all Execute statements. The default is “1”, allow.

The “AllowExecuteCommand” is set in the QVB settings file “C:\Windows\System32\config\systemprofile\AppData\Roaming\QlikTech\QlikViewBatch\settings.ini”.   See the SR11 or QV12 Release Notes for more information.

To summarize QV11 controls —  In desktop, it is possible to control Execute on a qvw by qvw basis. In Server SR10 and earlier, Execute is always allowed.  In SR11+, Execute may be allowed or disallowed for all qvws via the “AllowExecuteCommand” setting.

 

On to QV12.  In QV12 IR Desktop, the “Can Execute External Programs” checkbox is present but it has no impact on script execution.   The only way to allow Execute is to check on the “Always Override Security:  Script” User Preference setting. 

I hope the “Can Execute External Programs” checkbox is removed in a future SR as it no longer seems to have any use.

QV12 Server uses the “AllowExecuteCommand” ini setting. The default is “0”, disallow, which is different than QV11.

To summarize QV12 controls — In Desktop, Execute is allowed or disallowed for all qvws via the “Always Override Security:  Script” User Preference setting. In Server, Execute is allowed or disallowed for all qvws via the “”AllowExecuteCommand” setting.

A summary of controls for both versions:

 

 

To allow Execute in Qlik Sense, you must enable Legacy mode. Turning on  Legacy mode also allows  file path references in LOAD statements.  I’m guessing most Sense users would not accept this side effect, which may mean that Execute is not practical in the Sense environment.

Execute is not a commonly used script statement. If you are using Execute, I hope this post helps you plan for QV12.

-Rob

 

Share

Masters Summit 2016 — Milan

The Masters Summit for Qlik begins it’s fourth year with a date in Milan, Italy on 5-7 April. The Masters Summit is three days of advanced training designed to take your Qlik skills to the next level.

Designed for Qlik Developers who have basic skills and experience,  the Summit presents three days of intense hands-on sessions in topics such as Advanced Scripting and Data Modeling, Advanced Aggregation and Set Analysis, and Visualization Techniques.

We’ve updated our name to “Masters Summit for Qlik” to reflect that the conference now provides training and content for both QlikView and Qlik Sense.

You have some experience with Qlik, have taken the beginning courses. How do you ramp up to create more success with Qlik? Learn from seasoned experts and world class presenters Rob Wunderlich, Barry Harmsen, Oleg Troyansky and Bill Lay.

In addition to the hands-on exercises, you’ll come away with many valuable sample files and documents. You’ll also get a chance to meet and network with Qlik Developers from around the world.

In three years over 500 Qlik Developers have attended eight Summits in Europe and the US. Their feedback is overwhelming positive. Read about their experience here.

The US Summit date is in the planning stages. We anticipate the date will be sometime in September 2016.  Also stay tuned for an announcement of a brand new continent for the Summit in fall of 2016.

An early bird discount is available for Milan so register now.

Share

Q-On Courses in January

Just a heads up to get in your planning before taking a holiday break.  I’ll be teaching two on-line courses in early January:

January 7  QlikView™ Document Performance Tuning

Learn how to measure and optimize the performance of your QlikView apps. By the end of the session, you will understand how the calculation process in QlikView works, and how data model, expression and chart design impact response times and resource usage.

You will come away with the skills to analyze your own apps and make them run faster.

 

January 6  QlikView™ Components Scripting Library

Speed up your QlikView™ development workflow by making use of the powerful QlikView Components (QVC) scripting library.

QVC can help you build your QlikView™ projects quicker and ensuring a high level of quality in your scripts. The set of ready-made subroutines that QVC provides can be used to perform common script operations, implemented in a flexible way and incorporating best practices.

 

Hope to see you there !

-Rob

Share

A Better Show Frequency

I like using “Show Frequency” in a Listbox, but the feature suffers two drawbacks.

  1. If the Field is in a Dimension table, the frequency — usually 1 — is not particularly useful to the user.
  2. Excluded (gray) rows show a blank frequency value.

As an alternative to “Show Frequency”, we can use the Expression pane of the Listbox to provide a more meaningful number, such as count of orders.

num(
 count(DISTINCT OrderID)
,'#,##0')

Listbox Expressions don’t provide for a heading.   We can fake one by adding it the Listbox Title with enough spaces to align the heading.

The excluded values show zero. We can fix that by adding a Set to the expression. The Set should honor selections in other fields such as Product or Year, so we only need to ignore the selection in this field.

num(
 count({<Customer=>} DISTINCT OrderID)
,'#,##0')

Optionally, we can use the same expression as a Sort Expression if we want to sort by Order count.

There you have it.   A more useful Frequency for the QlikView Listbox.

The Qlik Sense Filter Pane does not provide an Expression property, but you can achieve a similar result in Qlik Sense by creating the Field as an expression,

aggr( 
    only({<Customer=>}Customer) 
    & repeat(chr(160),8)
    &num(
    count({<Customer=>}DISTINCT OrderID)
    ,'#,##0')
,Customer)

 

 

The “repeat(chr(160),8)” is a trick to insert 8 “non-breaking” spaces. A string of multiple regular ‘  ‘ spaces will display as only a single space when displayed in the browser.

You may want to place the numbers on the left side to make the numbers clearer.

And yes, you could dynamically calculate the number of spaces required to get multiple digits right-aligned.

Using  aggr() to create a Filter Box is a much “heavier calculation” than just using the Field. If you have a large application, test this technique for performance before deciding to use it there.

-Rob

Share

Mind The Space

Here’s a heads up on a QlikView script syntax issue.  The problem came up several times in a beginner class last week and I’ve also noted beginners on the QlikCommunity Forum struggling with this as well.

There should be a space after a script keyword, separating it from the next word, right?

Resident  MyTab    // correct
ResidentMyTab     // incorrect

If we omit the space  the syntax coloring will alert us and we may get a red error underline as well.

What if a bracket is used, such as the case when a name contains a space?  If a bracket appears immediately after the keyword, the syntax highlighter will make look it correct. But it is invalid syntax and will fail  when reloaded.

There are a number of places in script where this problem can happen. Here are a few examples.

- LOAD * INLINE[
- [F1] as[F2]
- LOAD[F1]

Good news! QlikView Version 12 corrects the problem. In QV12, the missing space will be flagged in the editor.

 

While the requirement for a space may seem obvious to an experienced developer or programmer type, it can be a problem for beginners who are copying an example from a book where the space is not so clear, and who rely on highlighting to tell them they have it right.

It can be a difficult error to debug. The resulting script error message can be something  indirect like “missing/misplaced FROM..” when the actual error is a missing space after an “AS” keyword.

I’m going to add the following slide to all my beginner trainings:

Words in expressions or script must be separated by a delimiter. Depending on context, the delimiter will be one of:

Space  Comma  (  {  <  = +  -  *   / Semicolon

[ Single-quote and Double-quote are not delimiters.

I know this incomplete, leaving off tabs, newline, =>. Just keeping it simple.

-Rob

 

 

Share

Pivot Table Grids

Using dimensions on the X and Y axis and plotting measures at the intersection is a useful visualization. The out of the box solution in QlikView is the Grid chart.

The grid chart does have limited options for representing data.

The Pivot Table can be a more robust alternative.  A Pivot Table grid (Crosstable) is created by dragging one of the dimensions to the horizontal position.

 

Now we can represent the measures as a heatmap by coloring the background of the cells.  This is frequently a more effective visualization than circles,  .

We can also display numbers in the cells.

 

We can display multiple expressions per dimension.

 

Here I show both the number and a linear gauge allowing for clear comparisons between Regions. The user can swap the dimensions if desired.

 

You can mix colors and numbers to show interesting patterns. Here’s a map of average Sunrise/Sunset/Daylight for my home town.

Maybe it’s not a design winner, but I find it interesting.

 

We can blank some cells in the grid to make meaningful shapes, in this case a rough outline of the United States.

All of the examples shown can be downloaded here.

A few styling tips:

  • Style format “Clean” is a good starting point for managing border lines and spacing.
  • “Custom Format Cell” Border before/after (double line) can be used to create white space between rows as shown in the Minimum Wage example above.  Also, setting Text Size to other than the default 100% can be useful.

Do you have any examples of Pivot Grids to share?

-Rob

 

Share