Category Archives: Masters Summit

Masters Summit for Qlik Cloud

Masters Summit New Qlik Cloud Track!

The Masters Summit Team is developing all new courses for 2024. The new content is focused on:

  • Qlik Cloud (perfect for clients who are on or moving to cloud already, or evaluating a migration).
  • New capabilities in the platform, like AutoML, Automation, etc.

Before we get too far down the road, we want your input to gauge interest.

We are opening pre-registration for our 2024 conferences:

Vienna, Austria – October 02-04 Pre-register for Vienna now

Philadelphia, USA – November 04-06 Pre-register for Philadelphia now

If you or one of colleagues would be interested in attending one of these Summits, and believe you can get approval for the expense we would ask you to kindly pre-register now.

We will not be taking any payment at this time and you should not make any travel arrangements.

Why pre-register?

If you pre-register on or before January 15, 2024, you can attend the event for a specially-discounted price of $1800 (that’s a 25% discount off our usual early-bird price)

This is your opportunity to help shape the future of Masters Summit events and attend with a deeply-discounted price. If we determine enough of you are interested in returning, based on pre-registration, we will move forward with our 2024 plans

See the new Sessions below.

Managing your tenant

  • Setting up development and promotion flows
  • Approaches for managing data.
  • Monitoring your tenant

Migrating to Qlik cloud

  • Planning your migration
  • Common pitfalls
  • Available tools, and how to use them.

Performance Tuning and Capacity Management

  • Best practices for performance and data scalability with the Qlik capacity licensing in mind.
  • Review opportunities in data modelling and visualizations to improve performance.
  • Review available tools for monitoring performance.

Real-world AutoML Applications

  • Understand machine learning predictive models and what business problems they can tackle.
  • Profiling data for ML, basic statistics, feature engineering, avoiding common pitfalls.
  • Evaluating and explaining your models using AutoML results
  • Deploying and monitoring your model

Enterprise Reporting

  • Building reports from exiting sheets or objects
  • Output presentation quality reports in various formats including annotations.
  •  Automating the distribution of reports to a variety of delivery channels.
  • Review existing report connector blocks.
  • Discover the self-serve options for subscription and alert-based reporting.

Qlik Automations

  • Automating Analytics Operations and Automation
  • Dynamic Actions and Business Process Integration
  • Going beyond: Qlik-CLI and the Qlik APIs

Qlik Cloud Integration

  • Understand what APIs are available, the necessary skill sets required to develop with them and how to correctly structure your project architecture.
  • See the art of the possible with a mix of real-world solutions and innovative demos.
  • Discover useful tools to help speed up development and reduce costs.
  • Get tips and tricks for the dark art of authorization and authentication in both client and server implementations.

We hope to see you in 2024.

Sincerely, The Masters Summit Team.

Share

Using Attributes in QVD Files

Here’s a fun Friday post.

As I was updating my incremental load materials for the upcoming October/November Masters Summit for Qlik, I ran across an old note about creating “Attributes” in QVDs using Tags. Due to a bug in QVD generation I couldn’t implement the idea at the time. But the bug is long since fixed so here I present the general idea of “QVD Attributes” (my name, not an official Qlik thing).

We can assign Tags to Qlik Tables using the script TAG statement:

Tag TABLE TranData with 'MaxId=$(vMaxId)';
Tag TABLE TranData with 'Source=Salesforce';

If the table “TranData” is subsequently written to a QVD using the STORE statement, the tag will be included in the QVD file.

We can read tags directly from a QVD with script:

LOAD String%Table as Tag
FROM [lib://mylib/TranData.qvd]
(XmlSimple, table is [QvdTableHeader/TableTags/String]);

Note that we are reading only the XML header from the beginning of the file. This is very fast.

If we use a structured tag format like “key=value” we can create a generalized “Attribute” store in a QVD that can tell us something about the contents of that QVD. That could be useful when we want to use the QVD in a subsequent script.

How do we retrieve these tags in a useful and consistent way? I think loading them as variables would be a good approach. We can create a reusable subroutine to load all “key=value” tags from a QVD and generate variables named “Tag:key”. We then call the Sub, using the path to any QVD:

Call LoadTags('lib://QVDFiles/TranData.qvd');

We might utilize the new variables in something like an incremental Load.

SQL SELECT * From Trans Where Id > $(Tag:MaxId);

Or as a control to conditionally execute a script block.

If '$(Tag:Source)' = 'Salesforce' Then
// Use the Salesforce Load statement
EndIf

What is all this stuff? Subroutines? Preceding Load? Incremental Load? Tags? Join me at the Masters Summit for Qlik in Dublin or Orlando where we will teach you all about these topics and make you a scripting Master.

At the bottom of this post is a complete sample including the subroutine. You only need to change the “QVDFiles” to your lib to run this on your system.

-Rob

// Subroutine to Load Tagname=Value pairs from a QVD as "Tag:Tagname" Variables.
Sub LoadTags(_path)
    _TempTags:
    Load
        SubField(Tag, '=', 1) as TagName,
        SubField(Tag, '=', 2) as TagValue 
    ;    
    LOAD String%Table as Tag
    FROM [$(_path)] 
    (XmlSimple, table is [QvdTableHeader/TableTags/String])
    Where String%Table like '*=*';
    
    For _i = 0 to NoOfRows('_TempTags') - 1
        Let _vname = 'Tag:' & Peek('TagName', $(_i), '_TempTags');
        Let [$(_vname)] = Peek('TagValue', $(_i), '_TempTags');
    Next _i
    Set _vname=;
    Set _i=;
    Drop Table _TempTags;
End Sub
// END of Subroutine


// Generate some sample data
TranData:
LOAD  
	RecNo()*2 as Id,
	Rand() as Amount,
    'A' as Customer
AutoGenerate 6;

// Something we may want to record from the data is the max Id value, for later incremental load. 
TempId:
LOAD Max(Id) as MaxId Resident TranData;
Let vMaxId = Peek('MaxId');
Drop Table TempId;

// We can tag with $(variable) values or literal values. 
Tag TABLE TranData with 'MaxId=$(vMaxId)';
Tag TABLE TranData with 'Source=Salesforce';
// Some other tag that is not part of our name=value scheme.
Tag TABLE TranData with "ABC";
// STORE and DROP the QVD
Store TranData Into [lib://QVDFiles/TranData.qvd] (qvd);
Drop Table TranData;


// Call our subroutine to load tags as variables, from the disk QVD.
Call LoadTags('lib://QVDFiles/TranData.qvd');

// Demonstrate how we might use these tag values.
// In a SQL Where clause.
SET select = SQL SELECT * From Trans Where Id > $(Tag:MaxId);
Trace $(select);

// In a conditional to select a block of script.
If '$(Tag:Source)' = 'Salesforce' Then
	Trace Going to use the Salesforce load statement;
EndIf
Share

Guest Speaker for Dublin Masters Summit

I’m pleased to announce that Brian Booden will join us at the October Masters Summit for Qlik in Dublin, Ireland. Brian is a seven (!) time Qlik Luminary / Partner Ambassador, co-host of The Data Mix, and a well known Qlik and Analytics expert.

Brian will share some inside tips on using Qlik Application Automation and I expect a few interesting tricks. I’m hoping he shows off the capabilities of his WhatsApp for Qlik integration!

Masters Summit for Qlik is the premier Qlik Sense & QlikView advanced education event. In three packed days of lectures and hands on exercises, you’ll learn from experts and take your Qlik Dev skills to the next level in data modeling, scripting, expressions and visualization. We’ll also cover performance tuning and Qlik API Integration for beginners. You’ll come away with a wealth of new knowledge, code samples and documentation.

Find out what over 1500 of your colleagues have learned already.

Share

Qlik App Performance Tuning — March 29

My Masters Summit for Qlik colleague Oleg Troyansky will be presenting his always valuable “Performance Tuning” course on March 29 as part of our “Masters Summit at Home” online series.

Oleg will do a deep dive into the causes and corrections for Qlik App performance issues. He’ll explain the why of good and bad performance while explaining important Qlik Engine concepts and demonstrating best practices. I highly recommend this course.

Oleg will demonstrate available performance tools including my favorite — QSDA Pro. As part of the class each student will receive a trial license key for QSDA and I’ll be available before and after class to help you get QSDA installed and configured.

Also coming up in the “Masters Summit at Home” series:

Effective Visualizations” with Bill Lay on March 15. Bill will teach you how to effectively “tell the story” using Qlik visualizations in a series of interesting and challenging scenarios. Bill is an engaging and thoughtful presenter. I always come away from Bill’s lectures with fresh ideas and useful tools.

Are you Qlik Integration / API curious? Want to understand the potential of combining the web and the Qlik platform? Join Nick Webster on April 12 for “Qlik Sense Integration“. Nick begins by showcasing Qlik embedded in intriguing and useful ways. He then moves on to teach the basics of web technologies HTML, CSS and Javascript, which you will then use to construct a fully functional, interactive Qlik web application in class. No web programming experience necessary!

Share

Roll Your Own Qlik “Connector”

Summary: I demonstrate how to use a simple Python REST server to wrap any data source for loading by the Qlik REST connector.

In my years of Qlik development I have many times needed to load data from a source not supported by available connectors. Sometimes the data resides on a factory floor controller and can only be accessed using something like telnet. One approach is to create a batch process that extracts the data to a flat file accessible to Qlik. This works, but you have to deal with issues of timing and failure monitoring.

If you want real time direct loading from Qlik script, you can write a custom connector for your data source. Writing a custom connector is a programming chore of medium complexity and there are limited languages you can work in.

As an alternative to a custom connector or batch process, I’m going to show how you can use the out-of-the-box Qlik REST connector to access your special data, wherever it lives!

REST is a popular way of exchanging data between systems. Data is requested from a REST Endpoint using a URL like:

http://localhost:3000/customers

Data is returned in JSON format like:

[{"Name": "Abc corp", "City": "New York"},
{"Name": "GeoFarm": "City": "Boston"}]

The Qlik REST connector translates JSON data into the rows and fields of a Qlik table. This URL request + JSON output makes it very easy to test REST endpoints using a browser.

Due to REST’s popularity, the tools we need to create REST endpoints are available in just about all programming languages, platforms and lowcode services. Use whatever language you are familiar with or that runs in your environment. I’m going to show a quick example using Python. If you want to create this example yourself, make sure you have Python 3+ installed along with the packages Flask and Pandas.

For my “special data source”, I’m going to use output from the Windows “Tasklist” command. The output looks like below.

Our REST Endpoint must provide three components:

  1. A web server that will handle the incoming URL.
  2. Data from our special/custom data source.
  3. A conversion of data from our native format to JSON.

Here is a complete Python program that satisfies all three requirements.

from flask import Flask, json
from io import StringIO
import pandas as pd
api = Flask(__name__)

@api.route('/tasklist', methods=['GET'])
def get_tasklist():
    import subprocess
    subprocess = subprocess.Popen("tasklist /FO CSV", shell=True, stdout=subprocess.PIPE, text=True)
    csvStringIO = StringIO(subprocess.stdout.read())
    df = pd.read_csv(csvStringIO);
    return Response(df.to_json(orient='records'), content_type='application/json')
if __name__ == '__main__':
    api.run(port=3000, host='0.0.0.0') 

Save this code in file “SimpleRest.py” and start the python task with the command:

python SimpleRest.py

In a Qlik App, create a new REST connection. The URL will be our new REST endpoint. If your Python task is running on other than “localhost”, change the host in the URL.

Give it a Name. Test and Save.

Use the “Select data” button on the connection and select the “root” table. The Qlik wizard will build the required script.

Reload and you should have data!

Let’s break down the pieces of the python program to understand what you would change and what you would re-use in a new scenario.

The first four lines import required libraries and initialize Flask, the REST library we will use. Here is the remaining code.

@api.route('/tasklist', methods=['GET'])
def get_tasklist():
    import subprocess
    subprocess = subprocess.Popen("tasklist /FO CSV", shell=True, stdout=subprocess.PIPE, text=True)
    csvStringIO = StringIO(subprocess.stdout.read())
    df = pd.read_csv(csvStringIO);
    return Response(df.to_json(orient='records'), content_type='application/json')
if __name__ == '__main__':
    api.run(port=3000, host='0.0.0.0') 

The bolded lines above start a webserver listening on port 3000 and declare what code will run when the URL path is “/tasklist”.

@api.route('/tasklist', methods=['GET'])
def get_tasklist():
    import subprocess
    subprocess = subprocess.Popen("tasklist /FO CSV", shell=True, stdout=subprocess.PIPE, text=True)
    csvStringIO = StringIO(subprocess.stdout.read())
    df = pd.read_csv(csvStringIO);
    return Response(df.to_json(orient='records'), content_type='application/json')
if __name__ == '__main__':
    api.run(port=3000, host='0.0.0.0') 

These two lines above run the tasklist /FO CSV command and capture the output. To make wrangling easier I added tasklist options “/FO CSV” which generates output in CSV format.

@api.route('/tasklist', methods=['GET'])
def get_tasklist():
    import subprocess
    subprocess = subprocess.Popen("tasklist /FO CSV", shell=True, stdout=subprocess.PIPE, text=True)
    csvStringIO = StringIO(subprocess.stdout.read())
    df = pd.read_csv(csvStringIO);
    return Response(df.to_json(orient='records'), content_type='application/json')
if __name__ == '__main__':
    api.run(port=3000, host='0.0.0.0') 

Lastly we wrangle the output from CSV to JSON format and send the data to the requestor.

I hope you can see from this example how easy it can be to expose your custom data to Qlik and have real time loading. I used Python for this example, but you could implement this same example in many languages with little effort.

if you’ve created my example in your own environment, you should now be able to create a treemap chart of Memory Usage. Tip: parse the [Mem Usage] field in the script with

Num#(SubField([Mem Usage], ' ', 1), '#,##0') * 1024 as [Mem Usage]

Want to talk more data loading techniques? Join me at the Masters Summit for Qlik in Fall 2022 where in addition to heads down learning, we’ll be kicking the tires on things like Server Side Extensions and Qlik Application Automation.

-Rob

Share

Masters Summits 2022

We’re baaaack!  After a two year break from live events, I’m pleased to announce that the Masters Summit for Qlik starts up again in Fall 2022. 

Masters Summit 2022 Madrid and New Orleans

Madrid – September 19-21

New Orleans – November 14-16

Our goal in this three day hands on education event is to “Take your Qlik skills to the next level”  to make you more productive and increase the business value of your Qlik Sense or QlikView applications.

Through lecture,  hands on activities and takeaway code samples,  the Summit will expand your knowledge with advanced techniques and a deeper understanding of the core skills required in all Qlik application development:

  • Data Modeling
  • Advanced Scripting
  • Advanced Aggregation & Set Analysis
  • Visualization.

In addition to core topics, we’ll have 1/2 day workshops on performance tuning and an introduction to creating Qlik Sense mashups using APIs.  See the complete agenda here.

Our evening guest speakers, networking events and optional lunchtime lectures fill out the program with additional content and lively discussion.

Our panel of five presenters are well known as authors, educators,  Qlik experts and members of the Qlik Luminary and MVP programs.

Have you taken basic Qlik training and/or worked with the product for a while?  Do you find yourself struggling with data modeling questions such as slowly changing dimensions and rolling time analysis?  Syntax for aggregation questions like “what products do top salesreps in each region sell?”  When do I need “$()” and when do I not?  

I hope you can join us in Madrid or New Orleans to take your Qlik skills to the next level.  The early bird registration discount is available until August 19.  Event details and online registration.

See you there!

-Rob

Share

Load Master Items in Script?

At the Masters Summit for Qlik, I always try to show something new and cool.  At past events I’ve shown things like Alexa selections in Qlik Sense and the new Document Analyzer for Qlik Sense.

So what do I have lined up for the upcoming Amsterdam and Washington DC events?

In QlikView we use variables in a similar way to Master Measures in Qlik Sense. That is, as a single version of a chart expression.  In QlikView, we typically use script to load variables from an external source to facilitate reuse and portability.

Qlik Sense does not have the ability to create Master Measures using script. But wouldn’t that be useful if it could…

I’ve been working on “loading Master Measures in script”  and believe I have a working solution.  I’ll be demonstrating at the Summit!

I hope you can join us at the upcoming summit. Our goal at this three day  hands on education event is to “Take your Qlik skills to the next level”  to make you more productive and increase the business value of your QlikView or Qlik Sense applications.

More information and registration here.

-Rob

 

Share

A Common SSE Plugin Project

Summary: I introduce qcb-qlik-sse, a community Server Side Extension to share custom Qlik functions. 

At the Masters Summit for Qlik, I dive into several different methods of creating reusable script and custom functions.  In QlikView we have the ability to write custom functions using VbScript/Jscript in the qvw Module.

Custom functions have been useful for things like regular expressions, geo calculations, url encoding, encryption and others.  I’ll call them “edge functions” — some of us need them, some of us don’t.

Qlik Sense does not have the module facility. How can we satisfy the requirement for custom function in Qlik Sense?  The Server Side Extension (SSE) facility can fill the need and is available to both Qlik Sense and QlikView.

An SSE Plugin runs as a separate task and provides communication with a Qlik Script or chart Expression via a TCP port. The same SSE Plugin can serve both QS and QV.

Anyone can write an SSE. The Qlik team provides the SSE base and you write a plugin that wires your new functions to Qlik. The new functions can be used in both Script and Charts.  A number of plugins have already been produced.

SSE seems to be the ideal place to provide a collection of edge functions.  Rather than a bunch of one-offs, I’m thinking a good idea would be to pool resources into a single effort that could be shared, much like QlikView Components did for Script.

I’ve implemented this idea as qcb-qlik-sse.  This server uses as it’s base the qlik-sse package created by Miralem Drek.

qcb-qlik-sse is written in javascript and runs in node.js.  At startup, the server scans it’s “/functions” directory and discovers what functions are available.  The general idea is that you can add new function by creating a new js file.  You can remove function you don’t want available by deleting the corresponding js file.

See what functions I’ve already implemented in the doc here. I’ve also provided an example qvf and qvw that exercise the functions.

If you want to try it out,  download the project and define the plugin to QS or QV as documented here.  You will also need node.js 8 or later installed.

Defined functions will show up in the suggestion list in both the script and expression editors.

 

If you want to add functions, some javascript skills are required.  Follow the directions in the readme and submit a PR.

I’ve labeled the project as “experimental” at this stage because I anticipate there could be some significant restructuring as I get feedback.

Let me know your ideas and if you find this useful!

-Rob

Want to kick the tires on reusable code and make your Qlik team more efficient? Come to the Masters Summit for Qlik, a three day advanced training event for Qlik Developers. 

 

 

Share

Masters Summit Fall 2019

Fall 2019 dates and location for the Masters Summit for Qlik have been announced .  Our goal at this  hands on education event is to “Take your Qlik skills to the next level”  to make you more productive and increase the business value of your QlikView or Qlik Sense applications.

When registering for either location, choose one of two tracks:

Our traditional “Qlik Sense and QlikView” track is designed for Qlik Developers who create applications using the out-of-the box clients: Qlik Sense Hub or the QlikView Desktop. Through lecture,  hands on activities and takeaway code samples,  the Summit will expand your knowledge with advanced techniques and a deeper understanding of the core skills required in all Qlik application development:

  • Data Modeling
  • Advanced Scripting
  • Advanced Aggregation & Set Analysis
  • Visualization.
  • Performance

See the complete agenda here.

Our Qlik API track is designed for Web Developers who create Qlik applications and mashups using the Qlik Engine API, Qlik Core, QAP and custom visualizations.

  • Qlik Core
  • Generic Objects
  • Loading & Visualizing Data
  • Architecture and Performance

The API track is led by Nick Webster and Speros Kokenes. You can see the agenda for the API track here.

Our evening guest speakers, networking events and optional lunchtime lectures fill out the program with additional content and lively discussion.

Our panel of six presenters are well known as authors, educators,  Qlik experts and members of the Qlik Luminary and MVP programs.

Learn more about determining if the Summit is right for you and choosing a track in our Frequently Asked Questions.  If you have any unanswered questions or want to learn more, reach out to registration@masterssummit.com .

I hope you can join us in Amsterdam or Washington D.C. to take your Qlik skills to the next level.  Event details and online registration.

See you there!

-Rob

Share

Masters Summit Stockholm

The 16th edition of the Masters Summit for Qlik will take place 1-5 April 2019  at the Stockholm City Conference Centre .  Our goal at this  hands on education event is to “Take your Qlik skills to the next level”  to make you more productive and increase the business value of your QlikView or Qlik Sense applications.

 

New for this summit, you can choose from two tracks:

Our traditional track, 1-3 April, is designed for Qlik Developers who create applications using the out-of-the box clients: Qlik Sense Hub or the QlikView Desktop. Through lecture,  hands on activities and takeaway code samples,  the Summit will expand your knowledge with advanced techniques and a deeper understanding of the core skills required in all Qlik application development:

  • Data Modeling
  • Advanced Scripting
  • Advanced Aggregation & Set Analysis
  • Visualization.
  • Performance

See the complete agenda here.

Our new API track, 3-5 April, is designed for Developers who will create Qlik applications and mashups using Qlik APIs, Qlik Core, QAP and custom visualizations.

  • Qlik Core
  • Generic Objects
  • Loading & Visualizing Data
  • Architecture and Performance

The API track is led by Nick Webster and Speros Kokenes. You can see the agenda for the API track here.

Our evening guest speakers, networking events and optional lunchtime lectures fill out the program with additional content and lively discussion.

Our panel of six presenters are well known as authors, educators,  Qlik experts and members of the Qlik Luminary and MVP programs.

Learn more about determining if the Summit is right for you and choosing a track in our Frequently Asked Questions.  If you have any unanswered questions or want to learn more, reach out to registration@masterssummit.com .

I hope you can join us in Stockholm to take your Qlik skills to the next level.  The early bird registration discount is available until 28 February.  Event details and online registration.

See you there!

-Rob

Share