2016/11/10 Leave a comment
NodePing now offers delayed notifications for your uptime monitoring. This is a powerful new feature that will help make your notifications actionable. There are two primary use cases for delayed notifications: flapping services and escalating notifications.
Not all services or networks are rock solid. Sometimes three or even two nines is “good enough”. Some locations have inherently lower expectations for availability or are just prone to frequent, short-lived outages. When a check often fails but recovers by itself quickly (flaps), it’s difficult to get actionable notifications.
Adjusting the check sensitivity setting down is useful to give your check more time to recover but if unassisted recovery takes longer than a minute the check will still likely fail. Use delayed notifications for flapping checks to receive alerts only when services are ‘really’ down. You can configure NodePing to send a alerts if your check remains down after say 5 minutes. Set the delay (from 1 minute to 1 hour) to your tolerance and receive only alerts when human intervention is required.
Not everyone needs to know about every outage right away. If the sysadmin on call can get the site back up within a few minutes, there’s no action required by senior staff or for the help desk to be informed. If an outage lasts longer, however, you may need to let your boss know things are still offline or give a heads up to the help desk that there are issues on the website and to expect some calls. Use delayed notifications to set escalating alerts to others if an outage continues.
You can even escalate alerts to yourself. I have several checks set to email me immediately and then send me an SMS if they’re still failing after 5 minutes and a voice call if the outage lasts longer than 10 minutes.
Setting Notification Delays
When editing a check, you’ll see the contact method drop down in the ‘Notifications’ section of each check. Choose a contact method and the ‘Delay’ and ‘Schedule’ dropdowns will also appear. You can set different delays on the same contact method by adding additional lines with the same contact method.
Delayed notifications can be useful to make all your alerts more actionable. If your contacts are ignoring NodePing notifications, they’ll succumb to alert fatigue and eventually ignore a truly important notification.
If you need any help tuning your checks to avoid flapping or adjusting your notifications to make them more actionable, please reach out to us at firstname.lastname@example.org. We really are happy to help.
If you aren’t using NodePing for uptime monitoring yet, please sign up for our 15-day, free trial and let us help you increase your uptime.