When working on Qlik Sense performance issues I frequently find I want to measure the performance of specific expressions. I might want to know how variations of an expression may perform against each other. In a slow chart with many measures I want calculation time individually for each measure to focus my efforts. Or perhaps I’m just satisfying a general curiosity or trying to settle a bet.
You can measure the performance of expression variations by modifying the chart and measuring the overall chart response time with something like Chrome Add Sense or QS Document Analyzer. That approach can get kind of clunky especially when you are focused on a subset of measures in the chart.
I prefer a more structured approach to testing expressions. The tool I reach for is CubeTester.
CubeTester is an open source Nodejs command line tool for testing the performance of Qlik HyperCubes (Dimensions and Measures). The test specification is written in a json file as either a HyperCubeDef or the “simplified” Dimension/Measure syntax.
Here’s a sample test written in simplified syntax that tests three variations of a cube (chart) containing one Dimension and three Measures.
I’ll run CubeTester specifying the file that holds this test:
node index.js test tests/columns.json
And receive this output:
There is no significant difference in performance between the variations. Importantly, I can also see that all three return identical total values as well.
CubeTester supports two commands:
- test : Run tests.
- extract: Extract app charts into a test file.
There are a number of options that can be specified on the command line or in the test definition. See the readme for more information on available options.
in addition to testing variations or trying out a theory, here are some other cases where I’ve used CubeTester.
- When working with a mashup where my HyperCube exists only in code, there is no chart to test.
- In a slow rendering chart I can test individual measures, combinations of measures and non-data expressions (like color expressions) to find the culprit.
Using CubeTester I can quickly try out ideas and document my progress as I work through an issue. I’ve made some interesting discoveries!
Some notes:
- Testing against a server uses certificates for authentication. (Pull request welcome if you want more auth options).
- Make sure you specify “wss” when using a server endpoint eg
wss://your.server:4747
- You’ll need to test with enough data to get calculation times of sufficient magnitude. Two results of 5 milliseconds vs 7 milliseconds are not precise enough to draw conclusions from.
- Calculation time is affected by the capacity of the target machine and what else is running. I recommend to repeat tests until you see a stable pattern. Use the –repeat option and take the lowest result from each repeat.
CubeTester is free to use. Have fun!
-Rob
Very nice Rob, thanks for sharing!
I love the idea of this to test the performance of different approaches during development without having to actually implement them. Great idea, Rob.
Very well done, thanks!
One question, running my test on a 100ml row table i get back only 50.000 rows. is this coming fromthe engine or is inside the tool?
Thanks!
The returned row count is from the engine. 50k should be the number of dimension combinations in this case.
Hi Rob,
cache can skew the results? Or does CubeTester frees the cache before any test?
Thanks,
Giuseppe
You can control how cached results are used with the –useCache switch. The default is false, meaning cache will not be used — the cube will get calculated fresh each time.
This is great tool, Thanks. I’m fan of document analyzer developed by you. Looking forward add this in my must have tool sets for QS.
I’m trying to run the commands, but stcuk with the JS compilation error.
Version of NodeJS isntalled on my machine is 6.4.1. Trying to connect with QS desktop.
Any help is really appreciated.
The required minimum nodejs is V8. I recommend V10 as that’s what I currently use, although later versions may work as well. I’ll open an issue to add this requirement into the package.
Hi ,
when i am trying to execute this comment, im getting below error.
C:\Users\localhost\Desktop\Office Work\Monitoring apps\CubeTester-master\CubeTester-master>node index.js test tests/columns.json
undefined
ErrorEvent {
target: WebSocket {
_events: [Object: null prototype] {
open: [Function],
close: [Function],
error: [Function],
message: [Function]
},
_eventsCount: 4,
_maxListeners: undefined,
readyState: 3,
protocol: ”,
_binaryType: ‘nodebuffer’,
_closeFrameReceived: false,
_closeFrameSent: false,
_closeMessage: ”,
_closeTimer: null,
_closeCode: 1006,
_extensions: {},
_isServer: false,
_receiver: null,
_sender: null,
_socket: null,
url: ‘ws://localhost:9076//app/engineData’,
_req: null,
[Symbol(kCapture)]: false
},
type: ‘error’,
message: ‘connect ECONNREFUSED 192.168.0.105:9076’,
error: Error: connect ECONNREFUSED 192.168.0.105:9076
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) {
errno: ‘ECONNREFUSED’,
code: ‘ECONNREFUSED’,
syscall: ‘connect’,
address: ‘192.168.0.105’,
port: 9076
}
}
Please help me to find.
Hello,
Please use the github issue tracker https://github.com/RobWunderlich/CubeTester/issues to report problems with CubeTester.