Monitoring results now available in JSON and CSV formats

Many of our customers monitor their own customers’ websites, or they monitor services about which they have to periodically report availability and incident statistics. Many of them would like to include status information in their own dash boards or integrate results with other data in reports. All of that calls for the data to be available in formats that can be easily used by scripts, or easily imported into spreadsheets.

NodePing has now added the ability to retrieve monitoring results in JSON and CSV format. JSON is ideal for scripting situations, such as creating your own dashboards or views of the data. CSV is great for loading the results data into your spreadsheets for analysis or creating reports for customers or management.

Both of these formats are available as extensions to our public reporting feature, which we added a while back. When you turn on public reporting, you are given a UUID that identifies that check. The same UUID can be used to access the data as a public report (you need to know the UUID, which gives you some control over who sees it, but does not require a login), as JSON output, or as a CSV download.

You can read more about these features and see some sample URLs on our Reporting page in the NodePing Documentation.

Monitoring password protected websites

Some of our customers have asked us to add basic authentication to our HTTP checks. They want to be able to check the availability of web pages that are protected from public access by a login. So, we have enhanced our HTTP and HTTP Content checks to support basic authentication.

This means that when you set up HTTP or HTTP Content checks in NodePing’s website monitoring service, you can now include a username and password in the URL. The format is username:password@host. We already supported both http and https requests and arbitrary ports in the URL. The following URL examples are all in a valid format for NodePing HTTP and HTTP Content checks (although these are fictitious, don’t actually use these URL’s in your checks), with this enhancement rounding out the list:

People who use this feature should be aware that HTTP basic authentication is not secure, which is one of the reasons we had not included it until now. This has required a small change to our Terms of Service to point out that we aren’t responsible for confidential information that is included in checks such as this. If you choose to include a username and password in your checks, you should take normal precautions to protect your data, including making sure that the login used for the checks is limited to no more access than is needed for the check, and avoid reusing passwords.

If you have any questions about basic authentication in HTTP or HTTP content checks, or about any aspect of our website monitoring, please don’t hesitate to ask us by emailing info@nodeping.com.

Server Monitoring from Europe

NodePing is happy to announce we’ve added a new region to our check locations. You can now choose to run your checks from our North American or new European regions… or both!

Website Monitoring

We’ve heard from many of our customers with a presence in Europe that a check location on that side of the pond would be a huge benefit. Thanks to our great providers, IntroVPS and RobHost, we added 4 new probe servers in the following locations:

  • London, England
  • Amsterdam, Netherlands
  • Falkenstein, Germany
  • Bucharest, Romania
Check out our FAQ for ip addresses and more information on our check locations.

We’ve also introduced the idea of a ‘default region‘ for your NodePing account. For new customers, you’ll set your default region when you sign up. Current customers have their default region set to ‘North America’. The default region can be changed in the ‘Scheduling‘ tab. When you create checks, they will be automatically run from your default region. You may, of course, decide to run any check from any region (including multiple regions and ‘worldwide’) by simply choosing the desired region from the check configuration when you create it. You can also change an existing check by selecting a different region in the check configuration at any time.

When you assign your checks to a region, they will be run from a random server in that region. If an ‘up/down’ event is detected, NodePing will immediately and automatically recheck from other servers in that same region. The number of rechecks is based on your configuration of the check – the default is 2 rechecks. After verifying the ‘up/down’ status on other servers in the same region, we’ll send out your configured email and SMS notifications.

Everything is included with NodePing so the new European region checking is already part of your flat-rate $10 a month subscription which includes your 1000 site/server checks, unlimited logins, contacts, contact groups, emails and international SMS. If you’re not already a customer, sign up for your free 15-day trial.

We’re already planning to add even more check regions to NodePing – Oceania maybe? South America? East Asia? Let us know in the comments what new region you’d like to see added next.

Contact Groups added to NodePing

We have been getting a lot of great feedback and suggestions from our users over the months since we launched the NodePing monitoring service. One feature that many people have asked for is the ability to group contacts and assign notifications to groups of contacts rather than just to individuals. Today we’re excited to announce that we’ve added Contact Groups to the service.

You can now create a Group, assign individual addresses or phone numbers to each contact group, and then set the Group to receive notifications for individual checks. So, for example, you could create a Developers group and a Systems group and have each assigned to different sites or servers you are monitoring. Then if the makeup of that part of your team changes you can change who belongs to the group without having to change each of the notifications.

One way we use this feature in the NodePing team is to assign SMS numbers to a group called SMS. Then we can easily set notifications to all of our phones together. For us, SMS notifications generally mean “Do something about this now!” and it makes it easy to assign key services to this notifications contact group. You could also use it to group all of a person’s contact methods easily for notifications that should get to them wherever they are. So for example you could create a “Shawn” group and include all of Shawn’s email addresses and phone numbers in that group. Then notifications will automatically be sent to all of his addresses.

As with contacts, there is no limit on the number of contact groups you can use in NodePing. You can mix and match contact methods in all kinds of ways, to suit your particular needs. We’re happy to now get to use this feature ourselves, and we hope it will further enhance the usefulness of our service to you as well.

There are a few smaller enhancements and adjustments in the release today as well, and we have more coming. Thank you to everybody who has chosen NodePing for their server and website monitoring. Thank you also to everybody who has given us feedback and suggestions. Please keep them coming!

When 200 Isn’t Enough

Website monitoring seems like a simple thing.  Most HTTP checks look for a response in the 200-399 range, 200 being the most common HTTP response meaning “Server responded correctly – here’s the resource you were looking for.”  But if you’re one of the millions of people or businesses that use Joomla, checking for 200 just isn’t enough.

Now don’t get me wrong, I like Joomla.  It’s a fine CMS and I’ve built many custom modules and plugins for it.  There are plenty of great websites running on it.  But you have to keep a close eye out for errors.

Today, one of our clients had an issue with the network storage on their server and the file system was quickly remounted in read-only mode.  This is expected and even prescribed behavior for mount point errors.  But MySQL, the database, didn’t like not being able to write to its tables and logs and quickly failed.  That, in turn, took the public website offline.  Joomla presented a big white page with the error message Database Error: Unable to connect to the database:Could not connect to MySQL.  The site was not working as expected… surely the website monitoring service would send an alert!  Nope.  The database access error was returned with an HTTP status of 200.  The web server was saying “Everything’s OK.”  No alert was sent from the basic HTTP check.  How embarrassing is it to get the news that your website is down on a call from a customer?

With NodePing, you can use our HTTP Content checks to make sure this doesn’t happen to your Joomla (or other CMS) site.  It works by simply searching for particular text within the returned HTML page.  If you add a check that searches for your copyright footer, it will fail as expected if the database is unavailable and the above error message was returned on the check.  You’ll receive your email or SMS notification that it’s down and can react quickly.  Conversely, if you know the exact error you want to avoid, you can run a negative content check so that it will alert you when the response of the check DOES contain the text ‘Database Error’.

With today’s complex web applications depending on many moving parts, it’s not enough to just check for a 200 response from a web server.  You have to dig deeper, and NodePing has just the shovel you need.

Making sure a web site is solid

There are a lot of things that go into making an Internet business successful. Layout, design, content wording, the effectiveness of graphics and how they are placed, the call to action, SEO, marketing and ads, product placement, pricing, visitor experience, check out processing… not to mention having the right product at the right time. None of that matters at all if your web site isn’t up.

Once you do all the work to get it all working together, you want to make sure it stays that way. Monitoring is a required part of running a solid business. It is a basic part of the infrastructure necessary to make sure your investment is going to continue to work for you. It isn’t something that is really optional when it comes to protecting the work and financial investment you put into building your site.

A server monitoring service checks to make sure things are working properly at all times. It isn’t just about making sure the server is up (although that is important). Monitoring should be comprehensive, and touch each part of your critical business infrastructure. Yes, you need to know if your server is sown. You also need to know if people are seeing database errors, or if their email to your customer service aren’t going through, and if your DNS is working properly. Each piece of your site should be checked to make sure everything is as it should be.

That means that one check on your front page isn’t really enough. Typically, a web site is running some kind of software that provides the framework of the site. Joomla, Drupal, and WordPress are all popular tools for this piece, but there are many others. Some businesses write their own platforms for this piece, but just about everybody uses something (even if it is custom built) that provides the framework for their content. If this piece breaks, it most often won’t show as down to simplistic site checks. The web server will still respond to requests, people will just see an ugly error message instead of your beautiful design. Your monitoring should check to make sure that your page is not just up, but working properly.

In fact, if your site uses several components that all need to be working together to pull off the overall effect, you should monitor each piece. That might consist of a shopping cart component (like osCommerce), a database and module for your inventory and product placements. Your blog needs to be working smoothly. If you serve images or media from a separate service or server you need to keep an eye on that too.

What does this look like in practice? Typically, you’ll want a basic HTTP check to monitor the entry page to your site. Then, you’ll want a content check on an image and other key elements that might break independently. For dynamic content served through modules or integration, you often don’t know in advance what that content will be. For those, a negative content check is in order, watching for an error or for messages like “0 items found.” For each of these checks, you’ll want to tune the timeouts to levels appropriate for your hosting and configuration.

All of this should be easy, and it shouldn’t cost you too much. You should be able to set it up, and then it should just work. This isn’t where you want to spend your time and energy. It should be boring stuff. It should be simple and dependable. If you are a small to medium sized business with a typical web site or three, and you don’t have a team of people dedicated to doing all the stuff we’ve discussed here, you should not have to spend more than twenty dollars a month on monitoring for your entire business infrastructure, and the whole monitoring setup should not take more than a few minutes a month of your time. If your monitoring provider is costing more than that or is more trouble than that, switch to someone else.

Many people think they can get monitoring for free. You can’t. The businesses that offer “free” services are paying ad costs around $10 per visitor to their site to get your “free” business. Don’t believe them when they say you can get a good value for that “free” plan. The “free” plans typically are for very few (usually one) check with a 15 minute or even one hour interval. That doesn’t do anything useful in making sure your site is working for your customers. Monitoring should be inexpensive, but it isn’t really free.

The biggest player on the block is Pingdom. Pingdom’s service is fairly similar to NodePing’s, but the five sites Pingdom provides for $9.95 (as of the date of this post) aren’t enough for your business. As we’ve discussed here, a small business with a web site already needs their 30 site plan for $39.95. As compared to many providers, Pingdom is a good deal. Compare that plan to NodePing’s $10 flat rate plan, and we think the value choice is clear.

DNS Monitoring for Both Sides

DNS monitoring, like a coin, has two sides: “What does my DNS server say?” and “What does ‘public’ DNS say?”  With NodePing server monitoring, you can ask both questions.

Our DNS check allows you to send a query of a specific type to your DNS server (or a public DNS server) and test the response against a string you define.  For example, you can verify that your website domain resolves to your web server’s static IP address and have NodePing send you an email or SMS alert when either the server or the response fails.

DNS queries can be made for the following types and the response verified:

  • A
  • CNAME
  • MX
  • NS
  • PTR
  • SOA
  • TXT

You can find more info on the DNS checks and our other check types in our documentation.

If you don’t have a NodePing account yet, try out our new DNS monitoring checks for free with a 15-day trial.

Now with Unlimited International SMS

Until today, NodePing  offered SMS notifications only to US-based numbers. But do not despair, our friends across the great ponds, we’re happy to announce that we’ve added unlimited international SMS notifications to our server monitoring service. Just add your mobile number to your contact record in plus format (example for UK ‘+445555555555’), and then set it as the notification number in your checks. As always, here at NodePing, it doesn’t cost any extra.  Same great service, same great price – you’ve leveled up!

Globally, mobile networks have a lot of variability and it is possible that some carriers won’t work with our SMS initially. If that’s the case with your carrier please let us know and we’ll do what we can to resolve it.

Give us a digital high-five in the comments if you’re glad to see international SMS added to NodePing!

US SMS? Yeah, we got that…

You asked for it…

For our US customers NodePing now offers unlimited SMS as part of the $10/month plan.  And you thought we couldn’t pack any more value in there!

All you need to do is add your 10-digit phone number to your contact record and to your check notifications and those important UP/DOWN messages will flow directly to your phone.

For customers outside the US, we’re working on direct SMS integration and hope to implement it in the near future.  Until then, please continue to use the email-to-SMS gateway addresses provided by your mobile carrier.  If your mobile operator isn’t listed at the link provided, please contact them directly and ask what your email-to-SMS address is.

Eight things you could do with monitoring checks on 1000 targets

With NodePing, you get checks on up to 1000 targets or services for one flat rate. NodePing’s 1000 service limit is designed to take the lid off of the kinds of limitations you might face with other service providers that charge more for adding checks or services. Once more checks don’t cost you more, what could you do with them? Every once in a while one of our customers has that moment where they realize how much they can do with NodePing that they couldn’t do when adding check targets raised the price. Here are eight things 1000 check targets can allow you to do that you might not do on other services.

  1. Monitor all your web sites and all basic services. OK, this one isn’t very creative, but it has to be said. If you are responsible for your business’s web sites then you need to know if they are down. Web sites that are down are not generating revenue, or if they are internal sites are not enabling your business to operate. If a site is worth having, then it is worth monitoring. This is the main reason people use monitoring in the first place. This goes beyond just web sites. If you are responsible to make sure that a service is available to customers or employees, you should monitor it so that you know immediately if it is unavailable, before someone complains.
  2. Our tongue-in-cheek tag line is “All your nodes are pinged by us,” but why not? With NodePing, now you can ping them all. If you don’t need notifications on all of them, just turn that off on a host by host basis, but you’ll have availability and uptime stats on everything.
  3. Monitor that a web page is showing the right information. This is called a web content check. Some web applications and content systems don’t return a proper 404 error, so to a normal HTTP check the page might appear to be up. A HTTP Content check makes sure the site is up by checking that it contains what you are expecting it to contain. It is often good to set the content to be checked as something that appears in all your pages, such as your copyright statement. This way if the text on the page changes during the normal course of business, your check will still pass.
  4. Monitor that the wrong text isn’t appearing on the page. Some web pages contain dynamic text. This is particularly the case for pages that show feeds, or your most recent news items. We’ve all gone to a site that should have a page with a list of articles or posts, but instead shows a database error or some kind of “No articles found” message. If that’s not what you want people to see, but you don’t know what text to check for because you don’t know what articles will appear, a check that makes sure the page does not contain specific text is the way to go.
  5. Along the same lines, since you have plenty of checks you might want more than one check on the same URL. If you need to watch for more than one error message, or check that multiple widgets or blocks on the page are populating correctly, why not check them all?
  6. Simple cron replacement. Many times web applications have a process that needs to run every so often, maybe every hour or every minute. These are often accessible by hitting a URL. This is often done by using curl or wget in a cron job, but it is easier to set up a check to hit the URL at the right interval. We use this to keep couchdb views fresh. Similarly, it can be used to replace Drupal’s cron job requirements.
  7. Check API’s and other HTTP interfaces. These often don’t get monitored, but they can be a key piece of your business. The HTTP Content check doesn’t care what kind of body the response has, and it will happily check for your text in JSON or XML as well as in HTML. You can monitor that a CouchDb server is saying “Welcome,” for example, or hit a URL that returns a reduced view and look for the value you expect in the results. The same idea applies to SOAP interfaces as well.
  8. Monitor other monitoring. Many systems have a status page that says how services on that host are doing. Frequently they’ll have an OK message, or an ERROR message will appear when things go wrong. HTTP Content checks can be used to watch these pages and send notifications if the wrong thing appears or does not appear on those pages. Both the “Contains” and “Does not contain” options for content checks are useful on this one.

There are many more things you could do with 1000 checks that you might not even consider doing with other services. We plan to add more check types to increase the utility of the service even more. What other things could you think of doing if you aren’t limited by artificial constraints imposed by services that charge by the target service or URL?