Slack update from pdi

Data-transform your slack channels

In my humble opinion Slack is a fantastic tool for keeping development teams aligned and informed. It has (irc supported) chat, voice calls, proper code snippet sharing but it really shines in its ability to integrate all that is important to a team.

For example we integrate:

  • updates on github, Pull requests, comments, …
  • updates on tickets, new, open, closed tickets
  • updates from ci, builds, tests, the occasional #FAIL
  • updates from deploys instantiated from ci onto the mesos cluster
  • updates on failing services and network components from nagios
  • bash scripts synchronising in the background or via cron

One thing was missing

Updates from pesky long running etl-jobs, while being away from the vpn-ed carte status page. And a convenient way of updating fellow are-we-there-yet-are-we-there-yet-are-we-there-yet-are-we-there-yet- colleagues ;-).

Enter the slack incoming webhook

Slacks incoming-webhook is simple, easy to understand and can be extended to support a lot of feedback:

curl -X POST \
--data-urlencode 'payload={"text":"This is a line of text.\nAnd this is another one."}' \
https://hooks.slack.com/services/T00000000/B00000000/XXXXXXXXXXXXXXXXXXXXXXXX

Pdi can do this nicely, and posting it to a slack endpoint seems a no brainer. So for something to be usefull it has to:

  • update on starting a job
  • update on sub jobs / transformations
  • escalate on #FAIL-ed data transformations

Do it

A reporting transformation that delivers the very same json, but with some additions:

report to slack

Script the json with javascript:

var payload= '{' +
	'"channel": "' + channel + '",' +
    '"username": "' + username + '",' +
	'"text": "' + env + '",' +
    '"attachments": [' +
        '{' +
            '"title": "' + title + '",' +
            '"text": "' + text + '",' +
			'"fallback": "' + text + '",' +
            '"color": "' + color + '"' +
        '}' +
    ']' +
'};';

Incorperate the slack reporting into a bigger picture:

main job

Et voila, some slack updates:

on slack

Thin foil hat

This thing is updating from internal sources onto semi-public services, save to say to not stick any comprising data in those shiny new status updates…

Source kjb / ktr

link

Rogier Wessel

Opensource Data dude, from linux to docker into the magical world of data and Pentaho. Might have eaten all red pills when it came to docker...

Amersfoort, The Netherlands https://blog.blijblijblij.com