Config file syntax
Description of Butler’s config file.
Main Butler config file
The production_template.yaml config file looks like this (sorry for the incorrect syntax coloring, the issue is noted and is being worked on):
Info
Starting with Butler version 9.0 there is a check that the config file has the correct format.
This means that if you forget to add or change some setting in the main YAML config file, Butler will tell you what’s missing and refuse to start.
A consequence of this is that all settings are now mandatory, even if you don’t use them.
Trying to start Butler without some mandatory settings in the config file will result in an error message like this:
Adding the missing settings and restarting Butler will result in a successful startup.
---
Butler:
# General notes:
# - File and directory paths in this sample config file use Linux/Mac syntax, i.e. using forward slashes.
# Windows paths work just as well, just make sure to quote them with single or double quotes.
# - All entries in the config file are mandatory in the send that they must be present.
# However, if a feature is not used the corresponding config entries can contain
# any value (for example the provided default ones).
# Logging configuration
logLevel: info # Log level. Possible log levels are silly, debug, verbose, info, warn, error
fileLogging: false # true/false to enable/disable logging to disk file
logDirectory: log # Directory where log files are stored (no trailing / )
anonTelemetry: true # Can Butler send anonymous telemetry data?
# More info on whata data is collected: https://butler.ptarmiganlabs.com/docs/about/telemetry/
# Please consider leaving this at true - it really helps future development of Butler!
# Heartbeats can be used to send "I'm alive" messages to any other tool, e.g. an infrastructure monitoring tool
heartbeat:
enable: false
remoteURL: http://my.monitoring.server/some/path/
frequency: every 30 seconds # https://bunkat.github.io/later/parsers.html
# Docker health checks are used when running Butler as a Docker container.
# The Docker engine will call the container's health check REST endpoint with a set interval to determine
# whether the container is alive/well or not.
# If you are not running Butler in Docker you can safely disable this feature.
dockerHealthCheck:
enable: false # Control whether a REST endpoint will be set up to serve Docker health check messages
port: 12398 # Port the Docker health check service runs on (if enabled)
# Uptime monitor
uptimeMonitor:
enable: false # Should uptime messages be written to the console and log files?
frequency: every 15 minutes # https://bunkat.github.io/later/parsers.html
logLevel: verbose # Starting at what log level should uptime messages be shown?
storeInInfluxdb:
enable: false # Should Butler memory usage be logged to InfluxDB?
storeNewRelic:
enable: false
destinationAccount:
- First NR account
- Second NR account
# There are different URLs depending on whther you have an EU or US region New Relic account.
# The available URLs are listed here: https://docs.newrelic.com/docs/accounts/accounts-billing/account-setup/choose-your-data-center/
# As of this writing the options for the New Relic metrics API are
# https://insights-collector.eu01.nr-data.net/metric/v1
# https://metric-api.newrelic.com/metric/v1
url: https://insights-collector.eu01.nr-data.net/metric/v1 # Where should uptime data be sent?
header: # Custom http headers
- name: X-My-Header
value: Header value
metric:
dynamic:
butlerMemoryUsage:
enable: true # Should Butler's memory/RAM usage be sent to New Relic?
butlerUptime:
enable: true # Should Butler's uptime (how long since it was started) be sent to New Relic?
attribute:
static: # Static attributes/dimensions to attach to the data sent to New Relic.
- name: metricType
value: butler-uptime
- name: service
value: butler
- name: environment
value: prod
dynamic:
butlerVersion:
enable: true # Should the Butler version be included in the data sent to New Relic?
# Credentials for third party systems that Butler integrate with.
# These can also be specified via command line parameters when starting Butler.
# Command line options takes precedence over settings in this config file.
thirdPartyToolsCredentials:
newRelic: # Array of New Relic accounts/insert keys. Any data sent to New Relic will be sent to both accounts.
- accountName: First NR account
insertApiKey: <API key 1 (with insert permissions) from New Relic>
accountId: <New Relic account ID 1>
- accountName: Second NR account
insertApiKey: <API key 2 (with insert permissions) from New Relic>
accountId: <New Relic account ID 2>
# InfluxDB settings
influxDb:
enable: false # Master switch for InfluxDB integration. If false, no data will be sent to InfluxDB.
hostIP: influxdb.mycompany.com # IP or FQDN of Influxdb server
hostPort: 8086 # Port where Influxdb is listening. Default=8086
auth:
enable: false # Does InfluxDB require login?
username: user_joe
password: joesecret
dbName: butler # Name of database in InfluxDB to which Butler's data is written
instanceTag: DEV # Tag that can be used to differentiate data from multiple Butler instances
# Default retention policy that should be created in InfluxDB when Butler creates a new database there.
# Any data older than retention policy threshold will be purged from InfluxDB.
retentionPolicy:
name: 10d
duration: 10d
reloadTaskFailure:
enable: true
tailScriptLogLines: 20
tag:
static: # Static tags to attach to data stored in InflixDB
- name: butler_instance
value: prod-1
dynamic:
useAppTags: true # Should app tags be stored in InfluxDB as tags?
useTaskTags: true # Should task tags be stored in InfluxDB as tags?
reloadTaskSuccess:
enable: true
allReloadTasks:
enable: false
byCustomProperty:
enable: true
customPropertyName: 'Butler_SuccessReloadTask_InfluxDB'
enabledValue: 'Yes'
tag:
static: # Static attributes/dimensions to attach to events sent to InfluxDb
# - name: event-specific-tag 1
# value: abc 123
dynamic:
useAppTags: true # Should app tags be sent to InfluxDb as tags?
useTaskTags: true # Should task tags be sent to InfluxDb as tags?
# Store script logs of failed reloads on disk.
# The script logs will be stored in daily directories under the specified main directory below
# NOTE: Use an absolute path when running Butler as a standalone executable!
scriptLog:
storeOnDisk:
reloadTaskFailure:
enable: false
logDirectory: /path/to/scriptlogs
# Qlik Sense related links used in notification messages
qlikSenseUrls:
qmc: <Link to Qlik Sense QMC>
hub: <Link to Qlik Sense Hub>
# Settings for monitoring Qlik Sense licenses
qlikSenseLicense:
licenseMonitor:
enable: false
frequency: every 6 hours # https://bunkat.github.io/later/parsers.html#text
destination:
influxDb: # Store license data in InfluxDB
enable: true
tag:
static: # Static attributes/tags to attach to the data sent to InflixDB
- name: foo
value: bar
licenseRelease:
enable: false # true/false. If true, Butler will release unused licenses according to settings below
dryRun: true # true/false. If true, Butler will not actually release any licenses, just log what it would have done.
frequency: every 6 hours # https://bunkat.github.io/later/parsers.html#text
neverRelease: # Various ways of defining which users should never have their licenses released
user: # Users who should never have their licenses released
- userDir: 'INTERNAL'
userId: 'sa_repository'
- userDir: 'INTERNAL'
userId: 'sa_api'
- userDir: 'USERDIR'
userId: 'qs_admin_account'
tag: # Users with these tags will never have their licenses released
- License do not release
- some other tag
customProperty: # Users with these custom properties will never have their licenses released
- name: LicenseManage
value: do-not-release
userDirectory: # List of user directories whose users should never have their licenses released
- INTERNAL
- ADMIN
inactive: Ignore # Ignore/Yes/No. The value is case insensitive
# No = Don't release licenses for users marked as "Inactive=No" in the QMC
# Yes = Don't release licenses for users marked as "Inactive=Yes" in the QMC
# Ignore = Disregard this setting
blocked: Ignore # Ignore/Yes/No, No = Don't release licenses for users marked as "Blocked=No" in the QMC
removedExternally: ignore # Ignore/Yes/No, No = Don't release licenses for users marked as "Removed externally=No" in the QMC
licenseType: # License types to monitor and release
analyzer:
enable: true # Monitor and release Analyzer licenses
releaseThresholdDays: 30 # Number of days a license can be unused before it is released
professional:
enable: true # Monitor and release Professional licenses
releaseThresholdDays: 30 # Number of days a license can be unused before it is released
destination:
influxDb: # Store info about released licenses in InfluxDB
enable: true
tag:
static: # Static attributes/tags to attach to the data sent to InflixDB
- name: foo
value: bar
# Settings for notifications and messages sent to MS Teams
teamsNotification:
enable: false
reloadTaskFailure:
enable: false
webhookURL: <web hook URL from MS Teams>
messageType: formatted # formatted / basic. Formatted means that template file below will be used to create the message.
basicMsgTemplate: 'Qlik Sense reload failed: "{{taskName}}"' # Only needed if message type = basic
rateLimit: 300 # Min seconds between emails for a given taskID. Defaults to 5 minutes.
headScriptLogLines: 10
tailScriptLogLines: 10
templateFile: /path/to/teams/template/directory/failed-reload.handlebars
reloadTaskAborted:
enable: false
webhookURL: <web hook URL from MS Teams>
messageType: formatted # formatted / basic. Formatted means that template file below will be used to create the message.
basicMsgTemplate: 'Qlik Sense reload aborted: "{{taskName}}"' # Only needed if message type = basic
rateLimit: 300 # Min seconds between emails for a given taskID. Defaults to 5 minutes.
headScriptLogLines: 10
tailScriptLogLines: 10
templateFile: /path/to/teams/template/directory/aborted-reload.handlebars
serviceStopped:
webhookURL: <web hook URL from MS Teams>
messageType: formatted # formatted / basic. Formatted means that template file below will be used to create the message.
basicMsgTemplate: 'Windows service stopped: "{{serviceName}}" on host "{{host}}"' # Only needed if message type = basic
rateLimit: 30 # Min seconds between messages for a given Windows service. Defaults to 5 minutes.
templateFile: /path/to/teams/template/directory/service-stopped.handlebars
serviceStarted:
webhookURL: <web hook URL from MS Teams>
messageType: formatted # formatted / basic. Formatted means that template file below will be used to create the message.
basicMsgTemplate: 'Windows service started: "{{serviceName}}" on host "{{host}}"' # Only needed if message type = basic
rateLimit: 30 # Min seconds between messages for a given Windows service. Defaults to 5 minutes.
templateFile: /path/to/teams/template/directory/service-started.handlebars
# Settings for notifications and messages sent to Slack
slackNotification:
enable: false
restMessage:
webhookURL: <web hook URL from Slack> # Webhook to use when sending basic Slack messages via Butler's REST API
reloadTaskFailure:
enable: false
webhookURL: <web hook URL from Slack>
channel: sense-task-failure # Slack channel to which task failure notifications are sent
messageType: formatted # formatted / basic. Formatted means that template file below will be used to create the message.
basicMsgTemplate: 'Qlik Sense reload failed: "{{taskName}}"' # Only needed if message type = basic
rateLimit: 300 # Min seconds between emails for a given taskID. Defaults to 5 minutes.
headScriptLogLines: 10
tailScriptLogLines: 10
templateFile: /path/to/slack/template/directory/failed-reload.handlebars
fromUser: Qlik Sense
iconEmoji: ':ghost:'
reloadTaskAborted:
enable: false
webhookURL: <web hook URL from Slack>
channel: sense-task-aborted # Slack channel to which task stopped notifications are sent
messageType: formatted # formatted / basic. Formatted means that template file below will be used to create the message.
basicMsgTemplate: 'Qlik Sense reload aborted: "{{taskName}}"' # Only needed if message type = basic
rateLimit: 300 # Min seconds between emails for a given taskID. Defaults to 5 minutes.
headScriptLogLines: 10
tailScriptLogLines: 10
templateFile: /path/to/slack/template/directory/aborted-reload.handlebars
fromUser: Qlik Sense
iconEmoji: ':ghost:'
serviceStopped:
webhookURL: <web hook URL from Slack>
channel: qliksense-service-alert # Slack channel to which Windows service stopped notifications are sent
messageType: formatted # formatted / basic. Formatted means that template file below will be used to create the message.
basicMsgTemplate: 'Windows service stopped: "{{serviceName}}" on host "{{host}}"' # Only needed if message type = basic
rateLimit: 30 # Min seconds between messages for a given Windows service. Defaults to 5 minutes.
templateFile: /path/to/slack/template/directory/service-stopped.handlebars
fromUser: Qlik Sense
iconEmoji: ':ghost:'
serviceStarted:
webhookURL: <web hook URL from Slack>
channel: qliksense-service-alert # Slack channel to which Windows service stopped notifications are sent
messageType: formatted # formatted / basic. Formatted means that template file below will be used to create the message.
basicMsgTemplate: 'Windows service started: "{{serviceName}}" on host "{{host}}"' # Only needed if message type = basic
rateLimit: 30 # Min seconds between messages for a given Windows service. Defaults to 5 minutes.
templateFile: /path/to/slack/template/directory/service-started.handlebars
fromUser: Qlik Sense
iconEmoji: ':ghost:'
# Settings needed to send email notifications when for example reload tasks fail.
# Reload failure notifications assume a log appender is configured in Sense AND that the UDP server in Butler is running.
emailNotification:
enable: false
reloadTaskAborted:
enable: false
appOwnerAlert:
enable: true # Should app owner get notification email (assuming email address is available in Sense user directory)
includeOwner:
includeAll: true # true = Send notification to all app owners except those in exclude list
# false = Send notification to app owners in the include list
user:
- directory: <Sense user directory>
userId: <userId>
- directory: <Sense user directory>
userId: <userId>
excludeOwner:
user:
- directory: <Sense user directory>
userId: <userId>
- directory: <Sense user directory>
userId: <userId>
# Custom property used to control which aborted tasks will cause alert emails to be sent
# If this setting is true, alerts will not be sent for all tasks, but *only* for tasks with the CP set to the enabledValue.
# If this setting is false, alerts will be sent for all aborted reload tasks.
alertEnableByCustomProperty:
enable: true
customPropertyName: 'Butler_AbortedAlertEnableEmail'
enabledValue: 'Yes'
# Custom property used to say that alerts for a certain task should be sent to zero or more recipients
# These alerts will be sent irrespective of the alertEnableByCustomProperty.enable setting.
alertEnabledByEmailAddress:
customPropertyName: 'Butler_AbortedAlertSendToEmail'
rateLimit: 600 # Min seconds between emails for a given taskID. Defaults to 5 minutes.
headScriptLogLines: 15 # Number of lines from start of script to include in email
tailScriptLogLines: 15 # Number of lines from end of script to include in email
priority: high # high/normal/low
subject: 'Qlik Sense reload aborted: "{{taskName}}"' # Email subject. Can use template fields
bodyFileDirectory: path/to/email_templates # Directory where email body template files are stored
htmlTemplateFile: aborted-reload # Name of email body template file to use
fromAdress: Qlik Sense (no-reply) <qliksense-noreply@mydomain.com>
recipients: # Array of email addresses to which the notification email will be sent
- <Email address 1>
- <Email address 2>
reloadTaskFailure:
enable: false
appOwnerAlert:
enable: true # Should app owner get notification email (assuming email address is available in Sense user directory)
includeOwner:
includeAll: true # true = Send notification to all app owners except those in exclude list
# false = Send notification to app owners in the include list
user:
- directory: <Sense user directory>
userId: <userId>
- directory: <Sense user directory>
userId: <userId>
excludeOwner:
user:
- directory: <Sense user directory>
userId: <userId>
- directory: <Sense user directory>
userId: <userId>
# Custom property used to control which task failures will cause alert emails to be sent
# If this setting is true, alerts will not be sent for all tasks, but *only* for tasks with the CP set to the enabledValue.
# If this setting is false, alerts will be sent for all failed reload tasks.
alertEnableByCustomProperty:
enable: false
customPropertyName: 'Butler_FailedAlertEnableEmail'
enabledValue: 'Yes'
# Custom property used to say that alerts for a certain task should be sent to zero or more recipients
# These alerts will be sent irrespective of the alertEnableByCustomProperty.enable setting.
alertEnabledByEmailAddress:
customPropertyName: 'Butler_FailedAlertSendToEmail'
rateLimit: 600 # Min seconds between emails for a given taskID. Defaults to 5 minutes.
headScriptLogLines: 15 # Number of lines from start of script to include in email
tailScriptLogLines: 15 # Number of lines from end of script to include in email
priority: high # high/normal/low
subject: 'Qlik Sense reload failed: "{{taskName}}"' # Email subject. Can use template fields
bodyFileDirectory: path/to/email_templates # Directory where email body template files are stored
htmlTemplateFile: failed-reload # Name of email body template file to use
fromAdress: Qlik Sense (no-reply) <qliksense-noreply@mydomain.com>
recipients: # Array of email addresses to which the notification email will be sent
- <Email address 1>
- <Email address 2>
serviceStopped:
rateLimit: 30 # Min seconds between emails for a given service. Defaults to 5 minutes.
priority: high # high/normal/low
subject: '❌ Windows service stopped on host {{host}}: "{{serviceDisplayName}}"'
bodyFileDirectory: path/to/email_templates/email_templates
htmlTemplateFile: service-stopped
fromAdress: Qlik Sense (no-reply) <qliksense-noreply@mydomain.com>
recipients:
- <Email address 1>
- <Email address 2>
serviceStarted:
rateLimit: 30 # Min seconds between emails for a given service. Defaults to 5 minutes.
priority: high # high/normal/low
subject: '✅ Windows service started on host {{host}}: "{{serviceDisplayName}}"'
bodyFileDirectory: path/to/email_templates/email_templates
htmlTemplateFile: service-started
fromAdress: Qlik Sense (no-reply) <qliksense-noreply@mydomain.com>
recipients:
- <Email address 1>
- <Email address 2>
smtp: # Email server settings. See https://nodemailer.com/smtp/ for details on the meaning of these fields.
host: <FQDN or IP or email server, e.g. smtp.gmail.com>
port: <port on which SMTP server is listening>
secure: true # true/false
tls:
serverName: # If specified the serverName field will be used for TLS verification instead of the host field.
ignoreTLS: false
requireTLS: true
rejectUnauthorized: false
auth:
enable: true
user: <Username, email address etc>
password: <your-secret-password>
# Incident management tools integration
# Used to trigger incidents in these tools when task reloads fail or are aborted.
incidentTool:
signl4:
enable: false # Enable/disable Signl4 integration as a whole
url: https://connect.signl4.com/webhook/abcde12345
reloadTaskFailure:
enable: false # Enable/disable reload failed handling in Signl4
rateLimit: 15 # Min seconds between emails for a given taskID. Defaults to 5 minutes
serviceName: Qlik Sense # Signl4 "service name" to use
severity: 1 # Signl4 severity level for failed reloads
includeApp:
includeAll: false
appId:
# - 47d38f73-628f-44e1-a62c-841604b123ff
reloadTaskAborted:
enable: false # Enable/disable reload aborted handling in Signl4
rateLimit: 15 # Min seconds between emails for a given taskID. Defaults to 5 minutes
serviceName: Qlik Sense # Signl4 "service name" to use
severity: 10 # Signl4 severity level for aborted reloads
includeApp:
includeAll: false
appId:
# - 47d38f73-628f-44e1-a62c-841604b123ff
newRelic:
enable: false
destinationAccount:
event: # Failed/aborted reload tasks are sent as events to these New Relic accounts
- First NR account
- Second NR account
log: # Failed/aborted reload tasks are sent as log entries to these New Relic accounts
- First NR account
- Second NR account
# New Relic uses different API URLs for different kinds of data (metrics, events, logs, ...)
# There are different URLs depending on whther you have an EU or US region New Relic account.
# The available URLs are listed here: https://docs.newrelic.com/docs/accounts/accounts-billing/account-setup/choose-your-data-center/
url:
# As of this writing the valid options are
# https://insights-collector.eu01.nr-data.net
# https://insights-collector.newrelic.com
event: https://insights-collector.eu01.nr-data.net
# Valid options are (1) EU/rest of world and 2) US)
# https://log-api.eu.newrelic.com/log/v1
# https://log-api.newrelic.com/log/v1
log: https://log-api.eu.newrelic.com/log/v1
reloadTaskFailure:
destination:
event:
enable: false
sendToAccount: # Which reload task failures are sent to New Relic as events
byCustomProperty:
enable: false # Control using a task custom property which reload task failures are sent as events
customPropertyName: 'Butler_FailedTask_Event_NewRelicAccount'
always:
enable: false # Controls which New Relic accounts ALL failed reload tasks are sent to (as events)
account:
- First NR account
- Second NR account
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: event-specific-attribute 1 # Example
value: abc 123 # Example
dynamic:
useAppTags: true # Should app tags be sent to New Relic as attributes?
useTaskTags: true # Should task tags be sent to New Relic as attributes?
log:
enable: false
tailScriptLogLines: 20
sendToAccount: # Which reload task failures are sent to New Relic as log entries
byCustomProperty:
enable: false # Control using a task custom property which reload task failures are sent as log entries
customPropertyName: 'Butler_FailedTask_Log_NewRelicAccount'
always:
enable: false # Controls which New Relic accounts ALL failed reload tasks are sent to (as logs)
account:
- First NR account
- Second NR account
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: log-specific-attribute 1 # Example
value: def 123 # Example
dynamic:
useAppTags: true # Should app tags be sent to New Relic as attributes?
useTaskTags: true # Should task tags be sent to New Relic as attributes?
sharedSettings:
rateLimit: 15 # Min seconds between events sent to New Relic for a given taskID. Defaults to 5 minutes.
header: # Custom http headers
- name: X-My-Header # Example
value: Header value 1 # Example
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: service # Example
value: butler # Example
- name: environment # Example
value: prod # Example
reloadTaskAborted:
destination:
event:
enable: false
sendToAccount: # Which reload task aborts are sent to New Relic as events
byCustomProperty:
enable: false # Control using a task custom property which reload task aborts are sent as events
customPropertyName: 'Butler_AbortedTask_Event_NewRelicAccount'
always:
enable: false # Controls which New Relic accounts ALL aborted reload tasks are sent to (as events)
account:
- First NR account
- Second NR account
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: event-specific-attribute 2 # Example
value: abc 123 # Example
dynamic:
useAppTags: true # Should app tags be sent to New Relic as attributes?
useTaskTags: true # Should task tags be sent to New Relic as attributes?
log:
enable: false
tailScriptLogLines: 20
sendToAccount: # Which reload task aborts are sent to New Relic as log entries
byCustomProperty:
enable: true # Control using a task custom property which reload task aborts are sent as log entries
customPropertyName: 'Butler_AbortedTask_Log_NewRelicAccount'
always:
enable: false # Controls which New Relic accounts ALL aborted reload tasks are sent to (as logs)
account:
- First NR account
- Second NR account
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: log-specific-attribute 2 # Example
value: def 123 # Example
dynamic:
useAppTags: true # Should app tags be sent to New Relic as attributes?
useTaskTags: true # Should task tags be sent to New Relic as attributes?
sharedSettings:
rateLimit: 15 # Min seconds between events sent to New Relic for a given taskID. Defaults to 5 minutes.
header: # Custom http headers
- name: X-My-Header # Example
value: Header value 2 # Example
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: service # Example
value: butler # Example
- name: environment # Example
value: prod # Example
serviceMonitor:
destination:
event:
enable: false
sendToAccount: # Windows service events are sent to these New Relic accounts
- First NR account
- Second NR account
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: event-specific-attribute
value: abc 123
dynamic:
serviceHost: true # Should host where service is running be sent to New Relic as attribute?
serviceName: true # Should service name be sent to New Relic as attribute?
serviceDisplayName: true # Should service display name be sent to New Relic as attribute?
serviceState: true # Should service state be sent to New Relic as attribute?
log:
enable: false
sendToAccount: # Windows service log entries are sent to these New Relic accounts
- First NR account
- Second NR account
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: log-specific-attribute
value: def 456
dynamic:
serviceHost: true # Should host where service is running be sent to New Relic as attribute?
serviceName: true # Should service name be sent to New Relic as attribute?
serviceDisplayName: true # Should service display name be sent to New Relic as attribute?
serviceState: true # Should service state be sent to New Relic as attribute?
monitorServiceState: # Control whih service states are sent to New Relic
running:
enable: true
stopped:
enable: true
sharedSettings:
rateLimit: 5 # Min seconds between events/logs sent to New Relic for a given host+service. Defaults to 5 minutes.
header: # Custom http headers
- name: X-My-Header # Example
value: Header value 2 # Example
attribute:
static: # Static attributes/dimensions to attach to events sent to New Relic.
- name: service # Example
value: butler # Example
- name: environment # Example
value: prod # Example
# Settings for notifications and messages sent using outgoing webhooks
webhookNotification:
enable: false
reloadTaskFailure:
rateLimit: 300 # Min seconds between outgoing webhook calls for a given taskID. Defaults to 5 minutes.
webhooks:
- description: 'This outgoing webhook is used to...' # Informational only
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: POST # GET/POST/PUT
- description: 'This outgoing webhook is used to...' # Informational only
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: PUT # GET/POST/PUT.
- description: 'This outgoing webhook is used to...' # Informational only
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: GET # GET/POST/PUT
reloadTaskAborted:
rateLimit: 300 # Min seconds between outgoing webhook calls for a given taskID. Defaults to 5 minutes.
webhooks:
- description: 'This outgoing webhook is used to...' # Informational only
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: PUT # GET/POST/PUT
- description: 'This outgoing webhook is used to...' # Informational only
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: POST # GET/POST/PUT
- description: 'This outgoing webhook is used to...' # Informational only
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: GET # GET/POST/PUT
serviceMonitor:
rateLimit: 15 # Min seconds between outgoing webhook calls, per Windows service that is monitored. Defaults to 5 minutes.
webhooks:
- description: 'This outgoing webhook is used to...'
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: POST # GET/POST/PUT. Note that the body and URL query parameters differs depending on which method is used
- description: 'This outgoing webhook is used to...'
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: PUT # GET/POST/PUT. Note that the body and URL query parameters differs depending on which method is used
- description: 'This outgoing webhook is used to...'
webhookURL: http://host.my.domain:port/some/path # outgoing webhook that Butler will call
httpMethod: GET # GET/POST/PUT. Note that the body and URL query parameters differs depending on which method is used
# Scheduler for Qlik Sense tasks
scheduler:
enable: false # Should Butler's reload task scheduler be started?
configfile: config/schedule.yaml # Path to file containing task start schedules
# Key-value store
keyValueStore:
enable: false # Should Butler's key-value store be enabled?
maxKeysPerNamespace: 1000 # Max keys that can be stored per namespace. Defaults to 1000 if not specified in this file.
mqttConfig:
enable: false # Should Qlik Sense events be forwarded as MQTT messages?
brokerHost: <FQDN or IP of MQTT server>
brokerPort: 1883
azureEventGrid:
enable: false # If set to true, Butler will connect to an Azure Event Grid MQTT Broker, using brokerHost and brokerPort above
clientId: <client ID>
clientCertFile: <path to client certificate file>
clientKeyFile: <path to client key file>
taskFailureSendFull: true
taskAbortedSendFull: true
subscriptionRootTopic: qliksense/# # Topic that Butler will subscribe to
taskStartTopic: qliksense/start_task # Topic for incoming messages used to start Sense tasks. Should be subtopic to subscriptionRootTopic
taskFailureTopic: qliksense/task_failure
taskFailureFullTopic: qliksense/task_failure_full
taskFailureServerStatusTopic: qliksense/butler/task_failure_server
taskAbortedTopic: qliksense/task_aborted
taskAbortedFullTopic: qliksense/task_aborted_full
serviceRunningTopic: qliksense/service_running
serviceStoppedTopic: qliksense/service_stopped
serviceStatusTopic: qliksense/service_status
udpServerConfig:
enable: false # Should the UDP server responsible for receving task failure/aborted events be started?
serverHost: <FQDN or IP (or localhost) of server where Butler is running>
portTaskFailure: 9998
restServerConfig:
enable: false # Should Butler's REST API be started? Must be true if *any* API endpoints are to be used.
serverHost: <FQDN or IP (or localhost) of server where Butler is running> # Use 0.0.0.0 to listen on all network interfaces (e.g. when running in Docker!).
serverPort: 8080 # Port where Butler's REST is available. Any free port on the server where Butler is running can bse used.
backgroundServerPort: 8081 # Port used internally by Butler's REST API. Any free port on the server where Butler is running can bse used.
# List of directories between which file copying via the REST API can be done.
# Butler will try to clean up messy paths like this one, which resolves to /Users/goran/butler-test-dir1
# How? First you have /Users/goran/butler-test-dir1//abc which cleans up to /Users/goran/butler-test-dir1/abc,
# then up one level (..).
fileCopyApprovedDirectories:
- fromDirectory: /Users/goran/butler-test-dir1//abc//..
toDirectory: /Users/goran/butler-test-dir2
- fromDirectory: /Users/goran/butler-test-dir2
toDirectory: /Users/goran/butler-test-dir1
- fromDirectory: /from/some/directory2
toDirectory: /to/some/directory2
# List of directories between which file moves via the REST API can be done.
fileMoveApprovedDirectories:
- fromDirectory: /Users/goran/butler-test-dir1//abc//..
toDirectory: /Users/goran/butler-test-dir2
- fromDirectory: /Users/goran/butler-test-dir2
toDirectory: /Users/goran/butler-test-dir1
- fromDirectory: /from/some/directory2
toDirectory: /to/some/directory2
# List of directories in which file deletes via the REST API can be done.
fileDeleteApprovedDirectories:
- /Users/goran/butler-test-dir1
- /Users/goran/butler-test-dir1//abc//..
- /Users/goran/butler-test-dir2
# If set to true, Butler will be started with a focus on creating an API documentation file
# All configuration relating to outbound connetions (to Sense, email servers, MQTT broker etc) will be disabled.
# NOTE: This setting should always be false (or just deleted), unless you want to regenerate the API doc files.
restServerApiDocGenerate: false
# Enable/disable individual REST API endpoints. Set config item below to true to enable that endpoint.
restServerEndpointsEnable:
apiListEnbledEndpoints: false
base62ToBase16: false
base16ToBase62: false
butlerping: false
createDir: false
createDirQVD: false
fileDelete: false
fileMove: false
fileCopy: false
keyValueStore: false
mqttPublishMessage: false
newRelic:
postNewRelicMetric: false
postNewRelicEvent: false
scheduler:
createNewSchedule: false
getSchedule: false
getScheduleStatusAll: false
updateSchedule: false
deleteSchedule: false
startSchedule: false
stopSchedule: false
senseAppReload: false
senseAppDump: false
senseListApps: false
senseStartTask: false
slackPostMessage: false
restServerEndpointsConfig:
newRelic:
postNewRelicMetric: # Setings used by post metric to New Relic API endpoint
destinationAccount:
- First NR account
- Second NR account
# As of this writing the valid options are
# https://insights-collector.eu01.nr-data.net/metric/v1
# https://insights-collector.newrelic.com/metric/v1
url: https://insights-collector.eu01.nr-data.net/metric/v1
header: # Custom http headers
- name: X-My-Header
value: Header value
attribute:
static: # Static attributes/dimensions to attach to the metrics data sent to New Relic.
- name: env
value: prod
postNewRelicEvent: # Setings used by post event to New Relic API endpoint
destinationAccount:
- First NR account
- Second NR account
# Note that the URL path should *not* be included in the url setting below!
# As of this writing the valid options are
# https://insights-collector.eu01.nr-data.net
# https://insights-collector.newrelic.com
url: https://insights-collector.eu01.nr-data.net/
header: # Custom http headers
- name: X-My-Header
value: Header value
attribute:
static: # Static attributes/dimensions to attach to the metrics data sent to New Relic.
- name: env
value: prod
# Controls which tasks can be started via Butler's REST API.
# Enabling this feature gives Qlik Sense sysadmins a way to control which tasks can be started by third party systems and applications.
# If this feature is disabled all tasks can be started via the API (assuming the start task API itself is enabled).
# Note that the taskId, tag and customProperty sections below are additive. I.e. a task only has to appear in one of those sections to be on the "allowed" list.
startTaskFilter:
enable: false
allowTask:
taskId:
# Zero or more approved/allowed task IDs
# If Butler.startTaskFilter.enable is true, only task IDs listed below will be started by Butler
- e3b27f50-b1c0-4879-88fc-c7cdd9c1cf3e
- 7552d9fc-d1bb-4975-9a38-18357de531ea
- fb0f317d-da91-4b86-aafa-0174ae1e8c8f
tag:
# Zero or more tags used to indicate that a task is approved to be started by Butler.
# Use the Qlik Sense QMC to set tags on tasks.
# If Butler.startTaskFilter.enable is true, only tasks with the tags below will be started by Butler
- startTask1
- startTask2
customProperty:
# Zero or more custom properties name/value pairs used to indicate that a task is approved to be started by Butler.
# Use the Qlik Sense QMC to set custom properties on tasks.
# If Butler.startTaskFilter.enable is true, only tasks with the custom property values below will be started by Butler
- name: taskGroup
value: tasks1
- name: taskGroup
value: tasks2
# Monitor Windows services.
# This feature only works when Butler is running on Windows Server or desktop.
# On other OSs service monitoring will be automatically disabled.
serviceMonitor:
enable: false # Main on/off switch for service monitoring
frequency: every 30 seconds # https://bunkat.github.io/later/parsers.html
monitor:
- host: <hostname or IP> # Host name of Windows computer where services are running
services: # List of services to monitor
- name: postgresql-x64-12 # Posgress/repository db
friendlyName: Repository DB
- name: QlikSenseEngineService
friendlyName: Engine
- name: QlikSensePrintingService
friendlyName: Printing
- name: QlikSenseProxyService
friendlyName: Proxy
- name: QlikSenseRepositoryService
friendlyName: Repository
- name: QlikSenseSchedulerService
friendlyName: Scheduler
- name: QlikSenseServiceDispatcher
friendlyName: Service Dispatcher
alertDestination: # Control to thich destinations service related alerts are sent
influxDb: # Send service alerts to InfluxDB
enable: true
newRelic: # Send service alerts to New Relic
enable: true
email: # Send service alerts as emails
enable: true
mqtt: # Send service alerts as MQTT messages
enable: true
teams: # Send service alerts as MS Teams messages
enable: true
slack: # Send service alerts as Slack messages
enable: true
webhook: # Send service alerts as outbound webhooks/http calls
enable: true
# Certificates to use when connecting to Sense. Get these from the Certificate Export in QMC.
cert:
clientCert: <path/to/cert/client.pem>
clientCertKey: <path/to/cert/client_key.pem>
clientCertCA: <path/to/cert/root.pem>
# If running Butler in a Docker container, the cert paths MUST be the following
# clientCert: /nodeapp/config/certificate/client.pem
# clientCertKey: /nodeapp/config/certificate/client_key.pem
# clientCertCA: /nodeapp/config/certificate/root.pem
configEngine:
engineVersion: 12.612.0 # Qlik Associative Engine version to use with Enigma.js. Works with Feb 2020 and others
host: <FQDN or IP of Sense server where Sense Engine is running>
port: <Port to connect to, usually 4747>
useSSL: true
headers:
X-Qlik-User: UserDirectory=Internal;UserId=sa_repository
rejectUnauthorized: false # Set to false to ignore warnings/errors caused by Qlik Sense's self-signed certificates.
# Set to true if the Qlik Sense root CA is available on the computer where Butler SOS is running.
configQRS:
authentication: certificates
host: <FQDN or IP of Sense server where QRS is running>
useSSL: true
port: <Port to connect to, usually 4242>
headerKey: X-Qlik-User # Header used to identify what user connection to QRS is made as
headerValue: UserDirectory=Internal; UserId=sa_repository # What user connection to QRS is made as
rejectUnauthorized: false # Set to false to ignore warnings/errors caused by Qlik Sense's self-signed certificates.
# Set to true if the Qlik Sense root CA is available on the computer where Butler SOS is running.
configDirectories:
qvdPath: <Path to folder under which QVDs are stored>
Comments
-
Note that you can enable/disable most features independently of each other. This makes it easy to have Butler do exactly what’s needed - no more - no less.
-
The default location cert/key files are found in (assuming a standard install of Qlik Sense)
C:\ProgramData\Qlik\Sense\Repository\Exported Certificates\<name specified during certificate export>The files needed by Butler are
client.pem,client_key.pemandroot.pem.