When 200 Isn’t Enough

2011/12/09 by Leave a comment

Website monitoring seems like a simple thing.  Most HTTP checks look for a response in the 200-399 range, 200 being the most common HTTP response meaning “Server responded correctly – here’s the resource you were looking for.”  But if you’re one of the millions of people or businesses that use Joomla, checking for 200 just isn’t enough.

Now don’t get me wrong, I like Joomla.  It’s a fine CMS and I’ve built many custom modules and plugins for it.  There are plenty of great websites running on it.  But you have to keep a close eye out for errors.

Today, one of our clients had an issue with the network storage on their server and the file system was quickly remounted in read-only mode.  This is expected and even prescribed behavior for mount point errors.  But MySQL, the database, didn’t like not being able to write to its tables and logs and quickly failed.  That, in turn, took the public website offline.  Joomla presented a big white page with the error message Database Error: Unable to connect to the database:Could not connect to MySQL.  The site was not working as expected… surely the website monitoring service would send an alert!  Nope.  The database access error was returned with an HTTP status of 200.  The web server was saying “Everything’s OK.”  No alert was sent from the basic HTTP check.  How embarrassing is it to get the news that your website is down on a call from a customer?

With NodePing, you can use our HTTP Content checks to make sure this doesn’t happen to your Joomla (or other CMS) site.  It works by simply searching for particular text within the returned HTML page.  If you add a check that searches for your copyright footer, it will fail as expected if the database is unavailable and the above error message was returned on the check.  You’ll receive your email or SMS notification that it’s down and can react quickly.  Conversely, if you know the exact error you want to avoid, you can run a negative content check so that it will alert you when the response of the check DOES contain the text ‘Database Error’.

With today’s complex web applications depending on many moving parts, it’s not enough to just check for a 200 response from a web server.  You have to dig deeper, and NodePing has just the shovel you need.

Filed under Monitoring, NodePing, Using NodePing Tagged with , ,

DNS Monitoring for Both Sides

2011/11/11 by 4 Comments

DNS monitoring, like a coin, has two sides: “What does my DNS server say?” and “What does ‘public’ DNS say?”  With NodePing server monitoring, you can ask both questions.

Our DNS check allows you to send a query of a specific type to your DNS server (or a public DNS server) and test the response against a string you define.  For example, you can verify that your website domain resolves to your web server’s static IP address and have NodePing send you an email or SMS alert when either the server or the response fails.

DNS queries can be made for the following types and the response verified:

  • A
  • CNAME
  • MX
  • NS
  • PTR
  • SOA
  • TXT

You can find more info on the DNS checks and our other check types in our documentation.

If you don’t have a NodePing account yet, try out our new DNS monitoring checks for free with a 15-day trial.

Filed under DNS, Monitoring, NodePing, NodePing Updates, Using NodePing Tagged with ,

Now with Unlimited International SMS

2011/11/02 by 3 Comments

Until today, NodePing  offered SMS notifications only to US-based numbers. But do not despair, our friends across the great ponds, we’re happy to announce that we’ve added unlimited international SMS notifications to our server monitoring service. Just add your mobile number to your contact record in plus format (example for UK ‘+445555555555’), and then set it as the notification number in your checks. As always, here at NodePing, it doesn’t cost any extra.  Same great service, same great price – you’ve leveled up!

Globally, mobile networks have a lot of variability and it is possible that some carriers won’t work with our SMS initially. If that’s the case with your carrier please let us know and we’ll do what we can to resolve it.

Give us a digital high-five in the comments if you’re glad to see international SMS added to NodePing!

Filed under Monitoring, NodePing, NodePing Updates, Using NodePing Tagged with ,

Eight things you could do with monitoring checks on 1000 targets

2011/09/08 by 1 Comment

With NodePing, you get checks on up to 1000 targets or services for one flat rate. NodePing’s 1000 service limit is designed to take the lid off of the kinds of limitations you might face with other service providers that charge more for adding checks or services. Once more checks don’t cost you more, what could you do with them? Every once in a while one of our customers has that moment where they realize how much they can do with NodePing that they couldn’t do when adding check targets raised the price. Here are eight things 1000 check targets can allow you to do that you might not do on other services.

  1. Monitor all your web sites and all basic services. OK, this one isn’t very creative, but it has to be said. If you are responsible for your business’s web sites then you need to know if they are down. Web sites that are down are not generating revenue, or if they are internal sites are not enabling your business to operate. If a site is worth having, then it is worth monitoring. This is the main reason people use monitoring in the first place. This goes beyond just web sites. If you are responsible to make sure that a service is available to customers or employees, you should monitor it so that you know immediately if it is unavailable, before someone complains.
  2. Our tongue-in-cheek tag line is “All your nodes are pinged by us,” but why not? With NodePing, now you can ping them all. If you don’t need notifications on all of them, just turn that off on a host by host basis, but you’ll have availability and uptime stats on everything.
  3. Monitor that a web page is showing the right information. This is called a web content check. Some web applications and content systems don’t return a proper 404 error, so to a normal HTTP check the page might appear to be up. A HTTP Content check makes sure the site is up by checking that it contains what you are expecting it to contain. It is often good to set the content to be checked as something that appears in all your pages, such as your copyright statement. This way if the text on the page changes during the normal course of business, your check will still pass.
  4. Monitor that the wrong text isn’t appearing on the page. Some web pages contain dynamic text. This is particularly the case for pages that show feeds, or your most recent news items. We’ve all gone to a site that should have a page with a list of articles or posts, but instead shows a database error or some kind of “No articles found” message. If that’s not what you want people to see, but you don’t know what text to check for because you don’t know what articles will appear, a check that makes sure the page does not contain specific text is the way to go.
  5. Along the same lines, since you have plenty of checks you might want more than one check on the same URL. If you need to watch for more than one error message, or check that multiple widgets or blocks on the page are populating correctly, why not check them all?
  6. Simple cron replacement. Many times web applications have a process that needs to run every so often, maybe every hour or every minute. These are often accessible by hitting a URL. This is often done by using curl or wget in a cron job, but it is easier to set up a check to hit the URL at the right interval. We use this to keep couchdb views fresh. Similarly, it can be used to replace Drupal’s cron job requirements.
  7. Check API’s and other HTTP interfaces. These often don’t get monitored, but they can be a key piece of your business. The HTTP Content check doesn’t care what kind of body the response has, and it will happily check for your text in JSON or XML as well as in HTML. You can monitor that a CouchDb server is saying “Welcome,” for example, or hit a URL that returns a reduced view and look for the value you expect in the results. The same idea applies to SOAP interfaces as well.
  8. Monitor other monitoring. Many systems have a status page that says how services on that host are doing. Frequently they’ll have an OK message, or an ERROR message will appear when things go wrong. HTTP Content checks can be used to watch these pages and send notifications if the wrong thing appears or does not appear on those pages. Both the “Contains” and “Does not contain” options for content checks are useful on this one.

There are many more things you could do with 1000 checks that you might not even consider doing with other services. We plan to add more check types to increase the utility of the service even more. What other things could you think of doing if you aren’t limited by artificial constraints imposed by services that charge by the target service or URL?

Filed under Monitoring, NodePing, Using NodePing Tagged with , ,

Multi-site checks added to NodePing

2011/09/08 by Leave a comment

This week we’ve launched pinghosts in two additional locations with automatic rechecking across locations. So if the New Jersey pinghost reports a monitored site is down, we now automatically recheck it from Texas and California before we send notifications.

How many times we do this depends on your “Sensitivity” setting for the check. The default setting of High rechecks once from each of the other two locations. These rechecks take a few seconds each, and the notification will be sent off in around 30 seconds. A setting of “Very Low” rechecks the service ten times, and with the extra rechecking the notification gets out in around 2 minutes. A setting of “Very High” doesn’t do a recheck at all, so if any check shows the service is not responding as expected we send the notice immediately.

This addition of the multi-site rechecks is one of the features we get asked about most often, and we’re very happy to get it rolled out to our production service. We’d love to hear from you about additional features that you’d like to see in our service.

Filed under Monitoring, NodePing, NodePing Updates, Using NodePing Tagged with ,

Website monitoring with a backflip

2011/09/02 by 2 Comments

A standard website monitoring check will fail when the page isn’t returned at all or the web server reports a page missing. What happens when your site is running but there is a problem with dynamic content, like a feed is missing or a list of recent posts is empty?

In those cases the page might be “working” from the web server’s point of view (and so not reported as a 404 or 500 error), but not displaying what you want. You don’t want your visitors to see messages like “Error establishing a database connection” or “0 articles found“.

NodePing HTTP Content check tests if particular text shows up on a given webpage.  Use the setting ‘Contains‘ to be alerted when specific text does NOT appear on a page.  But in this case, we want to receive an alert when our error messages DO appear on the page.  Use the ‘Does not contain‘ setting and the error message text as the search term to be notified when that happens.

For example, if you had an article list that was dynamic, so you never knew exactly what was going to show up there but you know something is wrong if the text ‘0 articles found‘ appears.  Maybe the database is offline or you haven’t written anything recently enough.  You’ll want to receive an alert.

Simply configure a HTTP Content check for the page and switch the text setting to ‘Does not contain‘ and add ‘0 articles found‘ to the text area.  This will check the webpage and as long as it does NOT contain the words ‘0 articles found‘, the check will pass.  If that text ever shows up, the check will fail and you’ll receive an alert, as expected.

There’s a thousand other uses for the HTTP Content check.  Get creative and make sure you’re alerted when errors happen.

Filed under Monitoring, NodePing, NodePing Updates, Using NodePing

5 Basic Questions About Web Site Monitoring

2011/08/26 by 1 Comment

The other day I was looking for a place to get some good Mexican food. That’s fairly easy in my part of the world, but I was looking for somewhere I hadn’t eaten before. I found a place the same way I always do, on the web. I typed my search into a search engine, pulled up a map of my area, and started clicking on web sites. I looked through the menu for each place I found and picked based on my impression of the restaurant from the web site.

I do this same kind of thing for all kinds of businesses at least several times a week. Plumbing parts, accountants, property management companies, mechanics, toys, web design, banks… pretty much everything. If I am looking for a business, I find them on the web. If it’s not on the web, I will probably not find it.

Increasingly business relies on the Internet, even for non-Internet businesses. If your site is down, you lose business. If your email doesn’t go through, you lose business. If your business is technology or web related, this is doubly crucial. People’s impression depends largely on how you come across on the Internet, and if it doesn’t work you are in trouble.

That seems obvious, but how do you ensure that everything works all the time? If you are a large enough business to have a highly skilled IT department, they are using monitoring tools or services. If you are not a large enough business to keep IT staff on the payroll, this largely falls to you. How do you make sure your Internet presence is a plus for your business, bringing in new customers instead of driving them away? How do you do that without spending too much money and too much time?

Monitoring services exist specifically for this purpose. Technology professionals use monitoring services to make sure that the services they are responsible for are always available. This has been the normal way to do business for tech professionals for many years. There are sophisticated tools to help with this job that can monitor all kinds of things and notify someone immediately if there are problems. Until more recently, doing this inexpensively without spending a lot of time was out of reach for most people. Not any more.

A few years ago monitoring as a service started to pop up on the Internet. Now there are a number of companies out there providing these types of services. Some of them are easy to use, some are not. Many of them cost a lot, but a few do not. Some of them sell snake oil and fancy gadgets that don’t really tell you what you need to know. Increasingly, smart IT technical people are realizing that they can save time and money by using outside services to do things they had to do before themselves, and these same services are available to everybody without requiring a significant investment or a lot of knowledge.

If you are not a tech professional, and you are thinking about finding a way to make sure that your Internet presence is always there when your customers are looking for you, you might be asking questions like these:

  1. Is it easy? There is absolutely no reason monitoring should be hard to set up or use. If it is hard or takes you more than a few minutes to get going, chose a different service provider. You don’t need to know a lot to use a good monitoring service. In most cases all you need is the address of your web site or email service.
  2. How does it work? Monitoring services mostly all work about the same way. You login to the web site and create checks for the monitoring service. Setting up the check generally consists of typing in the address of your web sites and how often you want them checked. Sometimes there are a few other simple questions, but it doesn’t need to be more complicated than that. You also enter in email addresses or phone numbers to notify when your site or service is down. Typically the service takes it from there and starts monitoring right away. Monitoring services just connect to your site and log what happened. It’s all automated.
  3. How do I know what I need? Just about any monitoring service will do the checks that most businesses need. If you have specific needs in your monitoring, this might be something to shop around for. However, most businesses need HTTP checks, which is the basic check that makes sure a web site is up, and SMTP, which checks email services. Just about all monitoring companies do HTTP checks, and most of them do SMTP.
  4. How often should it check? This is up to you, but if you are using a service that checks every 10 or 15 minutes, your site could be down for several minutes before you know about it. The better services check as frequently as every minute. This is not a lot more expensive for the service to provide, and it should not cost you a lot more either.
  5. How much will it cost? This is currently the biggest differentiator in the monitoring business. Some monitoring services cost a lot, especially if you have more than a couple of web sites to watch. It doesn’t have to be expensive. NodePing costs a flat rate of $10 per month to check up to a thousand sites or services every minute. If you’re paying more than that, you’re paying too much. If a service needs a special calculator or a talk with a sales person to tell you how much it will cost, it’s too much. If prices are per check, read the fine print. It should be inexpensive, and it should be simple.

If you are not doing website and email monitoring yet you should start today. We think that NodePing is a great choice, but there are other good providers out there. Shop around. It is important to your success, it is easy, and it is inexpensive. You just have to do it.

Filed under Monitoring, NodePing, Using NodePing Tagged with , ,

10 Common Server Monitoring Mistakes

2011/08/17 by 8 Comments

Server monitoring is an essential part of any business environment that has services.  Even if you don’t have your own servers and use cloud-based services, you’ll want to know about downtime.  You don’t want to find out your web site is down from customers and you don’t want your boss to be the one to point out the email server has wandered off into the weeds.  Done properly, server monitoring alerts those responsible for the services the minute they’re unavailable, allowing them to respond quickly, getting things back up and running.

David and I have been responsible for servers and server monitoring for years and have probably made nearly all the mistakes possible while trying to do it properly.  So listen to the war stories from a couple of guys with scars and learn from our mistakes.

Here are 10 common server monitoring mistakes we’ve made.

1. Not checking all my servers

Yeah it seems like a no-brainer but when I have so many irons in the fire, it’s hard to remember to configure server monitoring for all of them.  Some more commonly forgotten servers are:

  • Secondary DNS and MX servers.  This ‘B’ squad of servers usually gets in the game when the primary servers are offline for maintenance or have failed.  If I don’t keep my eye on them too, they may not be working when I need them the most.
  • New servers.  Ah, the smell of fresh pizza boxes from Dell!  After all the fun stuff (OS install, configuration, hardening, testing, etc) the two most forgotten ‘must-haves’ on a new server are the asset tag (anybody still use those?) and setting up server monitoring.
  • Temporary/Permanent servers.  You know the ones I’m talking about.  The ‘proof of concept’ development box that was thrown together from retired hardware that has suddenly been dubbed as ‘production’.  It needs monitoring too.

2. Not checking all services on a host

We know most failures take the whole box down but if I don’t watch each service on a host, I could have a running website while FTP has flatlined.

The most common one I forget is to check both HTTP and HTTPS.  Sure, it’s the same ‘service’ but the apache configuration is separate, the firewall rules are likely separate, and of course HTTPS needs a valid SSL certificate.  I’ve gotten the embarrassing calls about the site being ‘down’ only to find out that the cert had expired.  Oh, yeah… I was supposed to renew that, wasn’t I.

3. Not checking often enough

Users and bosses have very little tolerance for downtime.  A lesson learned when trying to use a cheap monitoring service  that only provided 10 minute check intervals.  That’s up to 9.96 minutes of risk (pretty good math, huh?) that my server might be down before I’m alerted.  Configure 1 minute check intervals on all services.  Even if I don’t need to respond to them right away (a development box that goes down in the middle of the night), I’ll know ‘when’ it went down to within 60 seconds which could be helpful information when slogging through the logs for root cause analysis later.

4. Not checking HTTP content

Standard HTTP check is good… but the ‘default’, ‘under-construction’ Apache server page has given me that happy 200 response code and a green ‘PASS’ in my monitoring service just like my real site should.  Choose something in the footer of the page that doesn’t change and do an HTTP content matching check on that.  Don’t use the domain name though – that may show up in the ‘default’ page too and make that check less useful.

5. Not setting the correct timeout

Timeouts for a service are very subjective and should be configurable on your monitoring service.  Web guys tell me our public website should load under 2 seconds or our visitors will go elsewhere. If my HTTP service check is taking 3.5 seconds, that should be considered a FAIL result and someone should be notified.  Likewise, if I had a 4 second ‘helo’ delay configured in my sendmail, I’d want to move that timeout above that.

Timeouts set to high let my performance issues go unnoticed; timeouts set too low just increase my notification noise. It takes time to tweak these on a per-service level.

6. Not realizing external and internal monitoring are different

When having an external monitoring service watch servers behind my firewalls, I may need to punch some holes in said firewall for that monitoring to work properly.  This can be a real challenge sometimes as many monitoring services use multiple locations and then dynamically pick one to monitor my servers making it hard to maintain a white-list of their IPs or hostnames to let in my network.

Another gotcha I’ve run into is resolution of internal and external DNS views.  If these aren’t configured properly, you’ll most likely get lots of ‘down’ notifications for hosts that are simply unreachable.

7. Sensitivity too low/high

Some servers or services seem more prone to having little hiccups that don’t take the server down but may intermittently cause checks to fail due to traffic or routing or maybe the phase of the moon. Nothing’s more annoying than a 3AM ‘down’ SMS for a host that really isn’t down.  Some folks call this a false positive or flapping- I call it a nuisance.  Of course I should jump every time a single ping looses its way around the interwebs and every SMTP helo goes unanswered – but reality sets in and a more dangerous condition might occur – I may be tempted to start ignoring notifications because of all the false positives.

A good monitoring service handles this nicely by allowing me to adjust the sensitivity of  each check.  Set this too low and my notifications for legitimate down events take too long to reach me but set it too high and I’m swamped with useless false positive notifications.  Again, this is something that should be configured per service and will take time to tweak.

8. Notifying the wrong person

Nothing ruins a vacation like a ‘host down’ notification.  Sure, I’ve got backup sysadmins that are covering it but I forgot to change the service so notifications get delivered to them and not me.

Another thing I’ve forgotten to take into consideration is notification time windows.  John’s always the first in the office at 6AM, he should get the alerts until Billy shows up at 9AM because we all know Billy is useless until he’s had that first hit of coffee.

9. Not choosing the correct notification type

Quick on the heels of #8 is knowing which type of notification to send. Yeah, I’ve made the mistake of configuring it to send email alerts when the email server is down.  Critical server notifications should almost always send via SMS.

10. Not whitelisting the notification system’s email address

Quick on the heels of #9 (we’ve got lots of heels around here) is recognizing that if I don’t whitelist the monitoring service’s email address – it may end up in the bit bucket.  Mental note – dang, all out of mental note paper.

Bonus!

11. Paying too much

I’ve paid hundreds of dollars a month for a mediocre monitoring service for a couple dozen servers before.  That’s just stupid.  NodePing costs $10 a month for 1000 servers/services at 1 minute intervals and we’re not the only cost effective monitoring service out there.  Be sure to shop around to find one that fits your needs well.  Know that most services are charging way too much though.

They say a wise man learns from his mistakes but a wiser man learns from the mistakes of the wise man.  Nuff said, true believer.

Filed under Monitoring, Using NodePing Tagged with , ,

Newer posts →