Learn from experts and connect with global tech leaders at POST/CON 24. Register by March 26 to save 30%.

Learn more →
X

Stream Any Log File to Slack Using curl

Avatar

When was the last time you had to execute a long-running script and wished you could monitor the progress remotely without setting up a plethora of systems? For us, it was yesterday. And doing that was so simple, we had to share it.

At Postman, we use Slack for a lot of stuff, having it report the progress of an arbitrary script was simply natural. The following piece of code tails a file and for every new line added to the file, streams it to a slack incoming webhook using curl.

#!/bin/bash

tail -n0 -F "$1" | while read LINE; do
  (echo "$LINE" | grep -e "$3") && curl -X POST --silent --data-urlencode \
    "payload={\"text\": \"$(echo $LINE | sed "s/\"/'/g")\"}" "$2";
done

To use this script, save it as an executable script and simply pass the path to the log file and a webhook url to this script.

./tail-slack.sh "file.log" "https://hooks.slack.com/services/...";

If you want this script to keep running even after you have logged out of your SSH terminal, use nohup command.

You can even make this script send the line to slack only when a particular keyword is found by sending the keyword as a third parameter.

./tail-slack.sh "/var/log/ngix/access.log" \
  "https://hooks.slack.com/services/..." " 404 ";

The above example tails an nginx access log and streams the line to slack when it has a 404 response. How simple and cool is that!

Tags:

Comment

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.

16 thoughts on “Stream Any Log File to Slack Using curl

  • Avatar

    genius – thanks for sharing!

  • Avatar

    This is awesome…question – Is it possible to capture and stream more than one type of error (e.g., 403, 404, 500)?

    • Avatar

      Sure. Check the syntax of grep command. You could do “\ 404 \ |\ 302\ ” or something similar.

  • Avatar

    This post is amazing.

    I’d like a recommendation: how would you make this job persist across system reboots? Should I set it in a cron job for @reboot or do you have another suggestion? I’m currently running as:

    @reboot ./tail-slack.sh “/var/log/nginx/error.log” “https://hooks.slack.com/services/…”

    • Avatar

      cron would be a bit unreliable and messy. Try using `upstart`. Assuming you’re on Ubuntu.

      • Avatar

        Thanks. Apparently Ubuntu 16.04 moved to systemd as default, just as I was getting confident with upstart. Any tips on how this could work from systemd?

        • Avatar

          I haven’t spent much time with systemd to comment on it off hand. It should be similar in theory though.

  • Avatar

    The script work as designed. Thank you.

    I’m late to this show and just recently discovered #slack.

    The script is missing some customization. For example: the script is missing the options to:

    *override the username (payload={“username”: “robot”)
    *channel (payload={“channel”: “off-topic”)
    *icon (payload={“icon_url”: “http://somewebsite.com/somecoolimage.jpg”)

    Was wondering if you could provide some guidance.

    • Avatar

      In the script, simply replace the JSON text I have set with a customised variant of your own and it should work just fine.

  • Avatar

    This. Is. Awesome!!!

    Any way to have an ignore list, rather than a “whitelist”? I would like to parse just *some* information from an error_log, not all.

    Thank you so much!

    • Avatar

      You could try piping the tail through a grep before sending it to the loop.

      • Avatar

        Yup, solved it, also improved it a bit (check my other comment).

    • Avatar

      You could invert grep. Replace the `grep` in my code using `grep -v` or `grep -L`. Try running `grep –help | grep invert` to see more on how inversion switches work.

  • Avatar

    Note that, using your script, an attacker could inject code.

    Here’s an updated version of your script: https://github.com/Razva/newbadmin/blob/master/slack/local-monitor.sh

  • Avatar

    Super helpful… thanks!

  • Avatar

    You may use egrep for $3 value