This is the multi-page printable view of this section. Click here to print.
Overview of the actions Butler can take when a reload task fails, is aborted or succeeds.
Note that client-managed and cloud versions of Qlik Sense
Info
Client-managed and cloud versions of Qlik Sense support different source triggers (reload failed/stopped/succeeded) and alert destinations (email, Teams, Slack, etc).
Check each section below for information about which triggers and destinations are supported for each version.
Below follows a list of destinations to which Butler can send notifications when a reload task fails or is aborted.
A comparison of the different alert destinations can be found here.
It might not be obvious at first, but there are several kinds of reloads in Qlik Sense Enterprise on Windows:
The reload failure notifications described here work by looking at log entries written by the scheduling service. When that service writes information to the logs about a failed reload, your logging appender will detect it and send a UDP message to Butler - who will forward the message to all the notification destinations configured in the config file.
It’s also possible to have the log appender send emails without using Butler. It works perfectly fine, but the emails will be very basic when it comes to formatting and you will not get any of the features offered by Butler (last few lines of the reload script log included in the email, customizable email subjects etc).
Butler can send two kinds of alert emails:
Alert emails can be formatted using HTML, use CSS styling, emojis etc.
There’s no reason an alert email can’t look good!
Alert emails viewed on a mobile phone give direct insight into what has happened:
Failed reload alert email on mobile home screen. |
Failed reload alert email viewed on mobile. |
In a regular email client a reload failed email could look like below.
Note the end of the script - the last few lines of the reload log are often very useful when it comes to understanding what caused the reload failure.
Qlik Sense Enterprise on Windows uses the log4net logging framework to create log files. Log4net is quite flexible and can - among other things - send emails when events such as reload failures occur. There is however little flexibility when it comes to layout and contents of those emails. They are text only (no formatting, tables, different fonts, colors etc) and the email subjects cannot contain any dynamic fields (for example the name of the failed reload task).
The goal of Butler’s alert emails is to address these limitations and offer a flexible foundation not only for emails, but for all kinds of alerts.
If you want to explore what’s possible using just the features offered by log4net, Christof Schwarz has a good post on sending basic notification emails when scheduled reloads fail, with links to Levi Turner’s great examples.
Qlik Sense can be configured in many ways. In some companies all apps are owned by a central service account.
Other companies set the developer as app owner also for published apps.
In the latter case it might be relevant to send the app owner a notification email when a reload task fails or is aborted. That way the developer is immediately made aware of the issue and can act on it as needed.
This feature assumes the app owner’s user account (in the Sense user directory) has an email address associated with it. When syncing users from Active Directory the users’ emails are often brought along into Sense, but there is no guarantee for this.
If an email address is available for a Sense user, the QMC user section can look like this:
Sometimes there is a desire to only have email alerts for some tasks.
One example can be a Sense cluster that hosts both development and production apps, maybe separated on different servers.
As of Butler 7.4.0 it is possible to control per task if an alert email should be sent when the task fails or is aborted from the QMC.
Conceptually it works like this:
Instructions for how to configure this feature is available here.
Note: This feature is similar to - but independent from - the “task specific email recipients” feature below. Either feature can be enabled or disabled independently of the other in Butler’s config file.
They may be cases where all alert emails should normally go to for example a Sense administrator, but some alerts should instead (or also) go to some other recipients.
An example could be a sales related Sense app. If it fails reloading the standard alert email should go to the Sense administrator, but there should also be an alert email sent to the sales operations team, to notify them that they won’t find updated numbers in the Sales app.
Butler handles this scenario by using a custome propperty (its name is configurable in the Butler config file) to set alert email recipients on a per-task basis.
Conceptually it works like this:
Instructions for how to configure this feature is available here.
Note: This feature is similar to - but independent from - the “alert emails only for some tasks” feature below. Either feature can be enabled or disabled independently of the other in Butler’s config file.
Butler uses a templating engine called Handlebars. It is used when Butler sense alert emails.
The high-level system overview below shows how email (and other alert types) are sent by Butler:
The Handlebars templating engine looks for template fields in the template files you create.
A complete list of template fields - including descriptions - is available in the Reference section.
While not obvious at first, there are different kinds of reloads taking place in a Qlik Sense Enterprise environment:
Reloads started by the Sense Scheduler service. These reloads always have a task associated with them.
Reloads started from Sense’s standard script editor. These reloads are not started by the Sense scheduler, but rather directly in the Sense engine. Progress for such reloads will therefore go to the engine logs.
The log appenders that drive Butler’s alerts rely on the Scheduler logs - not the engine logs.
This is an intentional design decision.
It is certainly possible to add log appenders also for engine logs and that way get notified when any reload fail. The question is whether that’s an interesting use case. In most cases sys admins aren’t very interested in reloads that fail during app development - they only care about failures caused by apps in production - i.e. app reload tasks managed by the Sense Scheduler. Thus, Butler currently doesn’t deal with reload failures reported from the Sense engine.
Qlik’s documenation around log appenders and how to hook into the Sense logs is somewhat brief, but does provide a starting point if you want to dive deeper into this topic.
The main log4net documentation (log4net is the logging framework used by Qlik Sense Enterprise) can also be useful.
These links describe how emails can be sent from the log4net logging framework itself, directly to the recipient. Butler includes sameple XML files for this use case too, but Butler takes things further by using the data in the Sense logs to pull in more data around the failed or stopped reload.
In other words - Butler’s alert emails are significantly more flexible and contain information (such as script logs) that are not availble using purely log4net.
The video below is available at Ptarmigan Labs’ YouTube channel and also in the Butler playlist.
Microsoft Teams, Slack and email are all notification destinations.
Alert messages/notifications come in two variants: “basic” and “formatted”.
These messages take full advantage of the formatting available in each notification destination.
Slack has its own way for creating messages with rich layout and formatting - as does Teams and email.
Formatted messages are created using template files.
Each notification destination has its own set of template files. It’s therefore possible to take advantage of each destination’s specific features when it comes to formatting the messages sent by Butler.
Message templates can include “template fields”. These are placeholders that are replaced with actual event values before the message is sent.
The GitHub repository includes fully functional template files for all destinations.
Basic message formats are available for all notification destinations.
This message type is useful if you only want a short, basic notification that a task failed or was aborted. The basic formats are pre-defined and are thus easy to set up.
Basic and formatted reload task failure notifications can look like this in Teams:
The configuration needed for setting this up is described here.
Basic and formatted reload task failure notifications can look like this in Teams:
The configuration needed for setting this up is described here.
The video below is available at Ptarmigan Labs’ YouTube channel and also in the Butler playlist.
When doing a scheduled reload or a reload started from the QMC, Sense will create a detailed log file that includes all the commands executed during the reload.
If a reload for reason fails it can be very useful to look at these reload logs.
The latest reload log file for each reload task is available via the QMC, but logs for previous reload attempts are not available via the QMC.
Using the same mechanism used by reload failure alerts in general, Butler can be configured to store the reload logs of all failed reloads to disk.
The reload logs are stored in the directory configured in the Butler config file, with separate directories for each date:
.
└── scriptlog
├── qscloud
│ └── 2024-10-14
│ └── 2024-10-14T11-41-31_appId=86ee4ae7-7ae7-4dd4-98a1-ebea989f78fb_reloadId=670d0369dededd0781e18ade.log
└── qseow
└── 2024-10-10
└── 2024-10-10_15-35-25_appId=8f1d1ecf-97a6-4eb5-8f47-f9156300b854_taskId=22b106a8-e7ed-4466-b700-014f060bef16.log
5 directories, 2 files
All in all this makes it a lot easier to find log files for failed reloads.
Configuration of this feature is described here.
When a reload fails, Butler can send information about the failed reload to InfluxDB.
The data stored in InfluxDB is described here.
Once the data is in InfluxDB it can be visualized in Grafana or similar tools.
Grafana has a good log viewer that can be used to visualize the data.
Note how even the script log is stored in InfluxDB, so you can see the last few lines of the reload script log in Grafana.
This makes it easy to right away see what went wrong, especially when dealing with reloads that happened a while back.
Configuration of this feature is described here.
When investigating reload failures it can often be useful to have access to the entire reload log, as this usually tells immediately what went wrong.
Butler forwards a very comprehensive set of data to New Relic when a reload fails, including the failing part of the app’s script.
This means that companies using New Relic as their enterprise monitoring solution can now also use it to monitor their Qlik Sense reloads, as well as all the Sense real-time metrics provided by Butler’s sibling tool Butler SOS.
As of this wrtiting, New Relic also offers a free tier that will be more than enough for most Butler users.
It is thus possible to get started with monitoring Sense reloads in New Relic without any additional cost.
More information about how Butler integrates with New Relic can be found here.
Configuration of this feature is described here.
When a reload fails, Butler can send information about the failed reload to an MQTT broker.
MQTT is a lightweight messaging protocol that is commonly used in IoT applications, but it is a mature and versatile protocol that can be used in many different scenarios.
In short, MQTT works by having a broker that clients can connect to. Clients can publish messages to the broker, and clients can subscribe to messages from the broker.
This makes MQTT a great way to integrate different systems in a publish/subscribe pattern.
By sending information about failed reloads to an MQTT broker, Butler can be integrated with any system that can consume MQTT messages - which is a lot of systems.
The information included in the MQTT message is described here.
Here is an example of how the information about a failed reload can be viewed in MQTT Explorer:
Configuration of this feature is described here.
Signl4 has a great mobile app that can be used to receive reload-failed alerts from Butler.
It can look something like this:
More information about how Butler integrates with Signl4 can be found here.
Configuration of this feature is described here.
Webhooks may be somewhat limited in terms of what they can do, but their simplicity is also their strength.
When you need to send information about failed/aborted reloads to a system that doesn’t have a dedicated integration with Butler, webhooks may be a good solution.
Butler offers a lot of flexibility in terms of how each webhook is configured.
You can…
Sending reload-failed informnation to a GET http webhook provided by Node-RED can look like this in the Node-RED editor:
Configuration of this feature is described here.
Qlik Sense Cloud has a few different ways to reload apps:
Note
The reload failure notifications described here work in all cases except the “manual reloads started from the script editor”.
There is currently no way around that, it’s just how Qlik Cloud works.
Butler can send the following alert emails:
Alert emails can be formatted using HTML, use CSS styling, emojis etc.
There’s no reason an alert email can’t look good!
Here is an example of how a failed reload alert email could look like:
All apps in Qlik Sense Cloud has an app owner, typically the user who created the app.
If there is an email address associated with the app owner, Butler can send an alert email to the app owner when a reload task fails or is aborted.
This feature can be turned on/off in the Butler config file.
It is also possible to only send alert emails to some app owners but not others, this page describes how to configure that.
Sometimes there is a desire to only have email alerts for some aps.
Maybe some apps are critical and should always trigger an alert email when they fail, while other apps are less critical and should not trigger any alert emails.
This is possible using a tag set on apps in Qlik Sense Cloud:
More info in the setup section.
Butler uses a templating engine called Handlebars. It is used when sending all kinds of alert emails supported by Butler.
For an overview of how Butler deals with alert messages for Qlik Sense Cloud, please see the setup section.
The Handlebars templating engine looks for template fields in the template files you create.
A complete list of template fields - including descriptions - is available in the Reference section.
Microsoft Teams, Slack and email are all notification destinations.
Alert messages/notifications come in two variants: “basic” and “formatted”.
These messages take full advantage of the formatting available in each notification destination.
Slack has its own way for creating messages with rich layout and formatting - as does Teams and email.
Formatted messages are created using template files.
Each notification destination has its own set of template files. It’s therefore possible to take advantage of each destination’s specific features when it comes to formatting the messages sent by Butler.
Message templates can include “template fields”. These are placeholders that are replaced with actual event values before the message is sent.
The GitHub repository includes fully functional template files for all destinations.
Basic message formats are available for all notification destinations.
This message type is useful if you only want a short, basic notification that a task failed or was aborted. The basic formats are pre-defined and are thus easy to set up.
Basic and formatted reload task failure notifications can look like this in Teams:
The configuration needed for setting this up is described here.
Basic and formatted reload task failure notifications can look like this in Teams:
The configuration needed for setting this up is described here.
When investigating reload failures it can often be useful to have access to the entire reload log.
Butler can store complete script logs for failed app reloads on local disk, making it easy to figure out what happened without having to log into Qlik Cloud.
Butler can be configured to retrieve and store the reload logs of all failed app reloads to disk.
The feature is identical to that of client-managed Qlik Sense, but those features can be individually enabled/disabled and configured.
The reload logs are stored in the directory configured in the Butler config file, with separate directories for each date:
.
└── scriptlog
├── qscloud
│ └── 2024-10-14
│ └── 2024-10-14T11-41-31_appId=86ee4ae7-7ae7-4dd4-98a1-ebea989f78fb_reloadId=670d0369dededd0781e18ade.log
└── qseow
└── 2024-10-10
└── 2024-10-10_15-35-25_appId=8f1d1ecf-97a6-4eb5-8f47-f9156300b854_taskId=22b106a8-e7ed-4466-b700-014f060bef16.log
5 directories, 2 files
All in all this makes it a lot easier to find log files for failed reloads.
Configuration of this feature is described here.