My Masters Summit for Qlik colleague Oleg Troyansky will be presenting his always valuable “Performance Tuning” course on March 29 as part of our “Masters Summit at Home” online series.
Oleg will do a deep dive into the causes and corrections for Qlik App performance issues. He’ll explain the why of good and bad performance while explaining important Qlik Engine concepts and demonstrating best practices. I highly recommend this course.
Oleg will demonstrate available performance tools including my favorite — QSDA Pro. As part of the class each student will receive a trial license key for QSDA and I’ll be available before and after class to help you get QSDA installed and configured.
Also coming up in the “Masters Summit at Home” series:
“Effective Visualizations” with Bill Lay on March 15. Bill will teach you how to effectively “tell the story” using Qlik visualizations in a series of interesting and challenging scenarios. Bill is an engaging and thoughtful presenter. I always come away from Bill’s lectures with fresh ideas and useful tools.
Are you Qlik Integration / API curious? Want to understand the potential of combining the web and the Qlik platform? Join Nick Webster on April 12 for “Qlik Sense Integration“. Nick begins by showcasing Qlik embedded in intriguing and useful ways. He then moves on to teach the basics of web technologies HTML, CSS and Javascript, which you will then use to construct a fully functional, interactive Qlik web application in class. No web programming experience necessary!
Summary: I demonstrate how to use a simple Python REST server to wrap any data source for loading by the Qlik REST connector.
In my years of Qlik development I have many times needed to load data from a source not supported by available connectors. Sometimes the data resides on a factory floor controller and can only be accessed using something like telnet. One approach is to create a batch process that extracts the data to a flat file accessible to Qlik. This works, but you have to deal with issues of timing and failure monitoring.
If you want real time direct loading from Qlik script, you can write a custom connector for your data source. Writing a custom connector is a programming chore of medium complexity and there are limited languages you can work in.
As an alternative to a custom connector or batch process, I’m going to show how you can use the out-of-the-box Qlik REST connector to access your special data, wherever it lives!
REST is a popular way of exchanging data between systems. Data is requested from a REST Endpoint using a URL like:
http://localhost:3000/customers
Data is returned in JSON format like:
[{"Name": "Abc corp", "City": "New York"}, {"Name": "GeoFarm": "City": "Boston"}]
The Qlik REST connector translates JSON data into the rows and fields of a Qlik table. This URL request + JSON output makes it very easy to test REST endpoints using a browser.
Due to REST’s popularity, the tools we need to create REST endpoints are available in just about all programming languages, platforms and lowcode services. Use whatever language you are familiar with or that runs in your environment. I’m going to show a quick example using Python. If you want to create this example yourself, make sure you have Python 3+ installed along with the packages Flask and Pandas.
For my “special data source”, I’m going to use output from the Windows “Tasklist” command. The output looks like below.
Our REST Endpoint must provide three components:
A web server that will handle the incoming URL.
Data from our special/custom data source.
A conversion of data from our native format to JSON.
Here is a complete Python program that satisfies all three requirements.
from flask import Flask, json
from io import StringIO
import pandas as pd
api = Flask(__name__)
@api.route('/tasklist', methods=['GET'])
def get_tasklist():
import subprocess
subprocess = subprocess.Popen("tasklist /FO CSV", shell=True, stdout=subprocess.PIPE, text=True)
csvStringIO = StringIO(subprocess.stdout.read())
df = pd.read_csv(csvStringIO);
return Response(df.to_json(orient='records'), content_type='application/json')
if __name__ == '__main__':
api.run(port=3000, host='0.0.0.0')
Save this code in file “SimpleRest.py” and start the python task with the command:
python SimpleRest.py
In a Qlik App, create a new REST connection. The URL will be our new REST endpoint. If your Python task is running on other than “localhost”, change the host in the URL.
Give it a Name. Test and Save.
Use the “Select data” button on the connection and select the “root” table. The Qlik wizard will build the required script.
Reload and you should have data!
Let’s break down the pieces of the python program to understand what you would change and what you would re-use in a new scenario.
The first four lines import required libraries and initialize Flask, the REST library we will use. Here is the remaining code.
These two lines above run the tasklist /FO CSV command and capture the output. To make wrangling easier I added tasklist options “/FO CSV” which generates output in CSV format.
Lastly we wrangle the output from CSV to JSON format and send the data to the requestor.
I hope you can see from this example how easy it can be to expose your custom data to Qlik and have real time loading. I used Python for this example, but you could implement this same example in many languages with little effort.
if you’ve created my example in your own environment, you should now be able to create a treemap chart of Memory Usage. Tip: parse the [Mem Usage] field in the script with
Want to talk more data loading techniques? Join me at the Masters Summit for Qlik in Fall 2022 where in addition to heads down learning, we’ll be kicking the tires on things like Server Side Extensions and Qlik Application Automation.
We’re baaaack! After a two year break from live events, I’m pleased to announce that the Masters Summit for Qlik starts up again in Fall 2022.
Madrid – September 19-21
New Orleans – November 14-16
Our goal in this three day hands on education event is to “Take your Qlik skills to the next level” to make you more productive and increase the business value of your Qlik Sense or QlikView applications.
Through lecture, hands on activities and takeaway code samples, the Summit will expand your knowledge with advanced techniques and a deeper understanding of the core skills required in all Qlik application development:
Data Modeling
Advanced Scripting
Advanced Aggregation & Set Analysis
Visualization.
In addition to core topics, we’ll have 1/2 day workshops on performance tuning and an introduction to creating Qlik Sense mashups using APIs. See the complete agenda here.
Our evening guest speakers, networking events and optional lunchtime lectures fill out the program with additional content and lively discussion.
Our panel of five presenters are well known as authors, educators, Qlik experts and members of the Qlik Luminary and MVP programs.
Have you taken basic Qlik training and/or worked with the product for a while? Do you find yourself struggling with data modeling questions such as slowly changing dimensions and rolling time analysis? Syntax for aggregation questions like “what products do top salesreps in each region sell?” When do I need “$()” and when do I not?
I hope you can join us in Madrid or New Orleans to take your Qlik skills to the next level. The early bird registration discount is available until August 19. Event details and online registration.
In QlikView we use variables in a similar way to Master Measures in Qlik Sense. That is, as a single version of a chart expression. In QlikView, we typically use script to load variables from an external source to facilitate reuse and portability.
Qlik Sense does not have the ability to create Master Measures using script. But wouldn’t that be useful if it could…
I’ve been working on “loading Master Measures in script” and believe I have a working solution. I’ll be demonstrating at the Summit!
I hope you can join us at the upcoming summit. Our goal at this three day hands on education event is to “Take your Qlik skills to the next level” to make you more productive and increase the business value of your QlikView or Qlik Sense applications.
Summary: I introduce qcb-qlik-sse, a community Server Side Extension to share custom Qlik functions.
At the Masters Summit for Qlik, I dive into several different methods of creating reusable script and custom functions. In QlikView we have the ability to write custom functions using VbScript/Jscript in the qvw Module.
Custom functions have been useful for things like regular expressions, geo calculations, url encoding, encryption and others. I’ll call them “edge functions” — some of us need them, some of us don’t.
Qlik Sense does not have the module facility. How can we satisfy the requirement for custom function in Qlik Sense? The Server Side Extension (SSE) facility can fill the need and is available to both Qlik Sense and QlikView.
An SSE Plugin runs as a separate task and provides communication with a Qlik Script or chart Expression via a TCP port. The same SSE Plugin can serve both QS and QV.
Anyone can write an SSE. The Qlik team provides the SSE base and you write a plugin that wires your new functions to Qlik. The new functions can be used in both Script and Charts. A number of plugins have already been produced.
SSE seems to be the ideal place to provide a collection of edge functions. Rather than a bunch of one-offs, I’m thinking a good idea would be to pool resources into a single effort that could be shared, much like QlikView Components did for Script.
qcb-qlik-sse is written in javascript and runs in node.js. At startup, the server scans it’s “/functions” directory and discovers what functions are available. The general idea is that you can add new function by creating a new js file. You can remove function you don’t want available by deleting the corresponding js file.
If you want to try it out, download the project and define the plugin to QS or QV as documented here. You will also need node.js 8 or later installed.
Defined functions will show up in the suggestion list in both the script and expression editors.
If you want to add functions, some javascript skills are required. Follow the directions in the readme and submit a PR.
I’ve labeled the project as “experimental” at this stage because I anticipate there could be some significant restructuring as I get feedback.
Let me know your ideas and if you find this useful!
-Rob
Want to kick the tires on reusable code and make your Qlik team more efficient? Come to the Masters Summit for Qlik, a three day advanced training event for Qlik Developers.
Fall 2019 dates and location for the Masters Summit for Qlik have been announced . Our goal at this hands on education event is to “Take your Qlik skills to the next level” to make you more productive and increase the business value of your QlikView or Qlik Sense applications.
When registering for either location, choose one of two tracks:
Our traditional “Qlik Sense and QlikView” track is designed for Qlik Developers who create applications using the out-of-the box clients: Qlik Sense Hub or the QlikView Desktop. Through lecture, hands on activities and takeaway code samples, the Summit will expand your knowledge with advanced techniques and a deeper understanding of the core skills required in all Qlik application development:
Our Qlik API track is designed for Web Developers who create Qlik applications and mashups using the Qlik Engine API, Qlik Core, QAP and custom visualizations.
Our evening guest speakers, networking events and optional lunchtime lectures fill out the program with additional content and lively discussion.
Our panel of six presenters are well known as authors, educators, Qlik experts and members of the Qlik Luminary and MVP programs.
Learn more about determining if the Summit is right for you and choosing a track in our Frequently Asked Questions. If you have any unanswered questions or want to learn more, reach out to registration@masterssummit.com .
The 16th edition of the Masters Summit for Qlik will take place 1-5 April 2019 at the Stockholm City Conference Centre . Our goal at this hands on education event is to “Take your Qlik skills to the next level” to make you more productive and increase the business value of your QlikView or Qlik Sense applications.
New for this summit, you can choose from two tracks:
Our traditional track, 1-3 April, is designed for Qlik Developers who create applications using the out-of-the box clients: Qlik Sense Hub or the QlikView Desktop. Through lecture, hands on activities and takeaway code samples, the Summit will expand your knowledge with advanced techniques and a deeper understanding of the core skills required in all Qlik application development:
Our new API track, 3-5 April, is designed for Developers who will create Qlik applications and mashups using Qlik APIs, Qlik Core, QAP and custom visualizations.
Our evening guest speakers, networking events and optional lunchtime lectures fill out the program with additional content and lively discussion.
Our panel of six presenters are well known as authors, educators, Qlik experts and members of the Qlik Luminary and MVP programs.
Learn more about determining if the Summit is right for you and choosing a track in our Frequently Asked Questions. If you have any unanswered questions or want to learn more, reach out to registration@masterssummit.com .
I hope you can join us in Stockholm to take your Qlik skills to the next level. The early bird registration discount is available until 28 February. Event details and online registration.
The 15th edition of the Masters Summit for Qlik will take place October 3-5 2018 at the Bellevue Hotel in Philadelphia PA . Our goal in this three day hands on education event is to “Take your Qlik skills to the next level” to make you more productive and increase the business value of your QlikView or Qlik Sense applications.
Through lecture, hands on activities and takeaway code samples, the Summit will expand your knowledge with advanced techniques and a deeper understanding of the core skills required in all Qlik application development:
Data Modeling
Advanced Scripting
Advanced Aggregation & Set Analysis
Visualization.
In addition to core topics, we’ll have 1/2 day workshops on performance tuning and an introduction to creating Qlik Sense mashups using APIs. See the complete agenda here.
Our evening guest speakers, networking events and optional lunchtime lectures fill out the program with additional content and lively discussion.
Our panel of five presenters are well known as authors, educators, Qlik experts and members of the Qlik Luminary and MVP programs.
In addition to our regular presenters, we’ve got some exciting evening guest speakers in Speros Kokenes and Paul Van Siclen.
Have you taken basic Qlik training and/or worked with the product for a while? Do you find yourself struggling with data modeling questions such as slowly changing dimensions and rolling time analysis? Syntax for aggregation questions like “what products do top salesreps in each region sell?” When do I need “$()” and when do I not? More self assessment to determine if the summit is right for you can be found here.
I hope you can join us in Philadelphia to take your Qlik skills to the next level. The early bird registration discount is available until August 17. Event details and online registration.
The 14th edition of the Masters Summit for Qlik will take place in Prague on 3-5 April 2018. In this three day hands on education event our goal is to “Take your Qlik skills to the next level” making you more productive and increasing the business value of your QlikView or Qlik Sense applications.
Through lecture, hands on activities and takeaway code samples, the Summit will expand your knowledge with advanced techniques and understanding of the core skills required in all Qlik application creation:
Data Modeling
Scripting
Advanced Aggregation & Set Analysis
Visualization.
In addition to core topics, we’ll have 1/2 day workshops on performance tuning and an introduction to creating Qlik Sense mashups using APIs. See the complete agenda here.
Our evening guest speakers, networking events and optional lunchtime lectures fill out the program with additional content and lively discussion.
Our panel of five presenters are well known as authors, educators, Qlik experts and members of the Qlik Luminary and MVP programs.
Have you taken basic Qlik training and/or worked with the product for a while? Do you find yourself struggling with data modeling questions such as slowly changing dimensions and rolling time analysis? Syntax for aggregation questions like “what products do top salesreps in each region sell?” When do I need “$()” and when do I not? More self assessment to determine if the summit is right for you can be found here.
I hope you can join us in Prague to take your Qlik skills to the next level. The early bird registration discount is available until 2 March. Event details and online registration.
I travel a lot for business and personal reasons. I’ve used TripIt (owned by Concur) for several years to plan and manage my trips.
It’s a great tool, can’t image living without it.
Today TriptIt sent me a lovely graphic email summarizing my 2017 travel. The design was a very fashionable card layout with share-to-social links.
Pretty cool, but for one problem. It’s not my data.
Yes, those are definitely not my travel stats.
When I teach Qlik application development, a common question raised by students or other newcomers is: “What’s the most important skill to master? Scripting? Expression writing? SQL? Pre-attentive attributes? Color rules?”
My answer is always the same: “First, get the numbers right. Second, keep the numbers right”.
It doesn’t matter how it looks or how fast it calculates if the results are wrong.
Join me at the Masters Summit for Qlik in Prague 3-5 April where in addition to a full range of Qlik Dev topics, we’ll be discussing automating quality. I’ll be demonstrating tools to automate dashboard validation and continuous quality monitoring.
-Rob
BTW I emailed the TripIt folks about the error, they’ve promised to look into it.
