NodePing Tips – Using the Status Filter

NodePing allows you to watch up to 1000 web sites or services. Some of our customers monitor dozens or even hundreds of web sites. Others monitor several web sites, plus DNS, FTP, SSH, and a variety of other services. The sites and services being monitored are surprisingly varied, and scattered across the globe.

Lately we’ve been working on our UI to make it work better for all of the variations in how the service is used, especially for customers with a larger number of checks. Several enhancements have been made to the main Status list. Putting more information at your fingertips.

One enhancement that is surprisingly powerful is the dynamic filtering of the status list. If you type in the Search box at the top right of the list, the list of checks is dynamically filtered as you type. The filter continues to be applied as the information is updated. This means, for example, that you can type “FAIL” in the search box and it will list only the services that are currently listed as failing. As services change from PASS to FAIL and back, they will appear and disappear from the filtered list dynamically. This gives you an instant list of those services that are failing that you can just keep on your screen and it keeps itself updated.

Another use of the search filter is viewing groups of checks with common attributes. For example, you can view all web site monitoring by typing http in the search, which will display all of the checks with an HTTP type or with “http” in the label. This is even more useful tied to careful use of check labels. If you include the word “router” in the label for all your checks of routers, that makes it very easy to list them together using the search. You could also use this to list monitoring checks of a specific network, client, or location.

The search filter is made possible by the capabilities of the DataTables plugin. Recent versions of DataTables have added some key features that have made our lives much easier in developing a rich interface for our web application.

We hope the new enhancements to our UI make the service even more useful, and we’re looking forward to hearing about the ways people use it that we didn’t think of.

Monitoring results now available in JSON and CSV formats

Many of our customers monitor their own customers’ websites, or they monitor services about which they have to periodically report availability and incident statistics. Many of them would like to include status information in their own dash boards or integrate results with other data in reports. All of that calls for the data to be available in formats that can be easily used by scripts, or easily imported into spreadsheets.

NodePing has now added the ability to retrieve monitoring results in JSON and CSV format. JSON is ideal for scripting situations, such as creating your own dashboards or views of the data. CSV is great for loading the results data into your spreadsheets for analysis or creating reports for customers or management.

Both of these formats are available as extensions to our public reporting feature, which we added a while back. When you turn on public reporting, you are given a UUID that identifies that check. The same UUID can be used to access the data as a public report (you need to know the UUID, which gives you some control over who sees it, but does not require a login), as JSON output, or as a CSV download.

You can read more about these features and see some sample URLs on our Reporting page in the NodePing Documentation.

Monitoring password protected websites

Some of our customers have asked us to add basic authentication to our HTTP checks. They want to be able to check the availability of web pages that are protected from public access by a login. So, we have enhanced our HTTP and HTTP Content checks to support basic authentication.

This means that when you set up HTTP or HTTP Content checks in NodePing’s website monitoring service, you can now include a username and password in the URL. The format is username:password@host. We already supported both http and https requests and arbitrary ports in the URL. The following URL examples are all in a valid format for NodePing HTTP and HTTP Content checks (although these are fictitious, don’t actually use these URL’s in your checks), with this enhancement rounding out the list:

People who use this feature should be aware that HTTP basic authentication is not secure, which is one of the reasons we had not included it until now. This has required a small change to our Terms of Service to point out that we aren’t responsible for confidential information that is included in checks such as this. If you choose to include a username and password in your checks, you should take normal precautions to protect your data, including making sure that the login used for the checks is limited to no more access than is needed for the check, and avoid reusing passwords.

If you have any questions about basic authentication in HTTP or HTTP content checks, or about any aspect of our website monitoring, please don’t hesitate to ask us by emailing info@nodeping.com.

Contact Groups added to NodePing

We have been getting a lot of great feedback and suggestions from our users over the months since we launched the NodePing monitoring service. One feature that many people have asked for is the ability to group contacts and assign notifications to groups of contacts rather than just to individuals. Today we’re excited to announce that we’ve added Contact Groups to the service.

You can now create a Group, assign individual addresses or phone numbers to each contact group, and then set the Group to receive notifications for individual checks. So, for example, you could create a Developers group and a Systems group and have each assigned to different sites or servers you are monitoring. Then if the makeup of that part of your team changes you can change who belongs to the group without having to change each of the notifications.

One way we use this feature in the NodePing team is to assign SMS numbers to a group called SMS. Then we can easily set notifications to all of our phones together. For us, SMS notifications generally mean “Do something about this now!” and it makes it easy to assign key services to this notifications contact group. You could also use it to group all of a person’s contact methods easily for notifications that should get to them wherever they are. So for example you could create a “Shawn” group and include all of Shawn’s email addresses and phone numbers in that group. Then notifications will automatically be sent to all of his addresses.

As with contacts, there is no limit on the number of contact groups you can use in NodePing. You can mix and match contact methods in all kinds of ways, to suit your particular needs. We’re happy to now get to use this feature ourselves, and we hope it will further enhance the usefulness of our service to you as well.

There are a few smaller enhancements and adjustments in the release today as well, and we have more coming. Thank you to everybody who has chosen NodePing for their server and website monitoring. Thank you also to everybody who has given us feedback and suggestions. Please keep them coming!

Making sure a web site is solid

There are a lot of things that go into making an Internet business successful. Layout, design, content wording, the effectiveness of graphics and how they are placed, the call to action, SEO, marketing and ads, product placement, pricing, visitor experience, check out processing… not to mention having the right product at the right time. None of that matters at all if your web site isn’t up.

Once you do all the work to get it all working together, you want to make sure it stays that way. Monitoring is a required part of running a solid business. It is a basic part of the infrastructure necessary to make sure your investment is going to continue to work for you. It isn’t something that is really optional when it comes to protecting the work and financial investment you put into building your site.

A server monitoring service checks to make sure things are working properly at all times. It isn’t just about making sure the server is up (although that is important). Monitoring should be comprehensive, and touch each part of your critical business infrastructure. Yes, you need to know if your server is sown. You also need to know if people are seeing database errors, or if their email to your customer service aren’t going through, and if your DNS is working properly. Each piece of your site should be checked to make sure everything is as it should be.

That means that one check on your front page isn’t really enough. Typically, a web site is running some kind of software that provides the framework of the site. Joomla, Drupal, and WordPress are all popular tools for this piece, but there are many others. Some businesses write their own platforms for this piece, but just about everybody uses something (even if it is custom built) that provides the framework for their content. If this piece breaks, it most often won’t show as down to simplistic site checks. The web server will still respond to requests, people will just see an ugly error message instead of your beautiful design. Your monitoring should check to make sure that your page is not just up, but working properly.

In fact, if your site uses several components that all need to be working together to pull off the overall effect, you should monitor each piece. That might consist of a shopping cart component (like osCommerce), a database and module for your inventory and product placements. Your blog needs to be working smoothly. If you serve images or media from a separate service or server you need to keep an eye on that too.

What does this look like in practice? Typically, you’ll want a basic HTTP check to monitor the entry page to your site. Then, you’ll want a content check on an image and other key elements that might break independently. For dynamic content served through modules or integration, you often don’t know in advance what that content will be. For those, a negative content check is in order, watching for an error or for messages like “0 items found.” For each of these checks, you’ll want to tune the timeouts to levels appropriate for your hosting and configuration.

All of this should be easy, and it shouldn’t cost you too much. You should be able to set it up, and then it should just work. This isn’t where you want to spend your time and energy. It should be boring stuff. It should be simple and dependable. If you are a small to medium sized business with a typical web site or three, and you don’t have a team of people dedicated to doing all the stuff we’ve discussed here, you should not have to spend more than twenty dollars a month on monitoring for your entire business infrastructure, and the whole monitoring setup should not take more than a few minutes a month of your time. If your monitoring provider is costing more than that or is more trouble than that, switch to someone else.

Many people think they can get monitoring for free. You can’t. The businesses that offer “free” services are paying ad costs around $10 per visitor to their site to get your “free” business. Don’t believe them when they say you can get a good value for that “free” plan. The “free” plans typically are for very few (usually one) check with a 15 minute or even one hour interval. That doesn’t do anything useful in making sure your site is working for your customers. Monitoring should be inexpensive, but it isn’t really free.

The biggest player on the block is Pingdom. Pingdom’s service is fairly similar to NodePing’s, but the five sites Pingdom provides for $9.95 (as of the date of this post) aren’t enough for your business. As we’ve discussed here, a small business with a web site already needs their 30 site plan for $39.95. As compared to many providers, Pingdom is a good deal. Compare that plan to NodePing’s $10 flat rate plan, and we think the value choice is clear.

Adding Notification Only Contacts

NodePing’s monitoring service is gaining popularity with companies that in turn provide online services such as hosting and network services to small businesses. NodePing works really well for these companies, because they can monitor a large number of their own customers sites and networks at one incredibly low cost. In fact, the cost of doing the same thing with most other services would run them several times as much. This enables our customers to bundle our monitoring as a part of the services that they can provide to their own customers in a very economical way, increasing their service value at low cost. That’s a win for everybody. This group of our customers have been asking us to add features to our service that makes this work even better for them, and we have been working to do just that.

The other day we added public reports to our list of features. Today we’re adding the ability to create contacts that receive notifications, but do not have a login to the NodePing app. Together, these features allow a provider to assign their customers to receive notifications and see uptime reports without access to the providers NodePing account information.

Seeing our monitoring services in use by other service providers to enhance their own offerings is exciting for us, and we plan to continue to add enhancements for this audience. As always, your feedback is welcome.

Public status reports of monitoring results

One of the features that we get asked about fairly often is public reports for monitoring results. It has been on our todo list from the start, and starting today they are available.

Of course, not every check on your list is something you want out in public. So this feature can be turned on individually for each check. Each check that you enable for public reports will have its own URL.

We’ve intentionally made these reports fairly plain, without much branding. This makes them suitable for framing or other methods you could use to embed them in your own site.

These reports will see further enhancements in the near future. As always your feedback is welcome.

Eight things you could do with monitoring checks on 1000 targets

With NodePing, you get checks on up to 1000 targets or services for one flat rate. NodePing’s 1000 service limit is designed to take the lid off of the kinds of limitations you might face with other service providers that charge more for adding checks or services. Once more checks don’t cost you more, what could you do with them? Every once in a while one of our customers has that moment where they realize how much they can do with NodePing that they couldn’t do when adding check targets raised the price. Here are eight things 1000 check targets can allow you to do that you might not do on other services.

  1. Monitor all your web sites and all basic services. OK, this one isn’t very creative, but it has to be said. If you are responsible for your business’s web sites then you need to know if they are down. Web sites that are down are not generating revenue, or if they are internal sites are not enabling your business to operate. If a site is worth having, then it is worth monitoring. This is the main reason people use monitoring in the first place. This goes beyond just web sites. If you are responsible to make sure that a service is available to customers or employees, you should monitor it so that you know immediately if it is unavailable, before someone complains.
  2. Our tongue-in-cheek tag line is “All your nodes are pinged by us,” but why not? With NodePing, now you can ping them all. If you don’t need notifications on all of them, just turn that off on a host by host basis, but you’ll have availability and uptime stats on everything.
  3. Monitor that a web page is showing the right information. This is called a web content check. Some web applications and content systems don’t return a proper 404 error, so to a normal HTTP check the page might appear to be up. A HTTP Content check makes sure the site is up by checking that it contains what you are expecting it to contain. It is often good to set the content to be checked as something that appears in all your pages, such as your copyright statement. This way if the text on the page changes during the normal course of business, your check will still pass.
  4. Monitor that the wrong text isn’t appearing on the page. Some web pages contain dynamic text. This is particularly the case for pages that show feeds, or your most recent news items. We’ve all gone to a site that should have a page with a list of articles or posts, but instead shows a database error or some kind of “No articles found” message. If that’s not what you want people to see, but you don’t know what text to check for because you don’t know what articles will appear, a check that makes sure the page does not contain specific text is the way to go.
  5. Along the same lines, since you have plenty of checks you might want more than one check on the same URL. If you need to watch for more than one error message, or check that multiple widgets or blocks on the page are populating correctly, why not check them all?
  6. Simple cron replacement. Many times web applications have a process that needs to run every so often, maybe every hour or every minute. These are often accessible by hitting a URL. This is often done by using curl or wget in a cron job, but it is easier to set up a check to hit the URL at the right interval. We use this to keep couchdb views fresh. Similarly, it can be used to replace Drupal’s cron job requirements.
  7. Check API’s and other HTTP interfaces. These often don’t get monitored, but they can be a key piece of your business. The HTTP Content check doesn’t care what kind of body the response has, and it will happily check for your text in JSON or XML as well as in HTML. You can monitor that a CouchDb server is saying “Welcome,” for example, or hit a URL that returns a reduced view and look for the value you expect in the results. The same idea applies to SOAP interfaces as well.
  8. Monitor other monitoring. Many systems have a status page that says how services on that host are doing. Frequently they’ll have an OK message, or an ERROR message will appear when things go wrong. HTTP Content checks can be used to watch these pages and send notifications if the wrong thing appears or does not appear on those pages. Both the “Contains” and “Does not contain” options for content checks are useful on this one.

There are many more things you could do with 1000 checks that you might not even consider doing with other services. We plan to add more check types to increase the utility of the service even more. What other things could you think of doing if you aren’t limited by artificial constraints imposed by services that charge by the target service or URL?

Multi-site checks added to NodePing

This week we’ve launched pinghosts in two additional locations with automatic rechecking across locations. So if the New Jersey pinghost reports a monitored site is down, we now automatically recheck it from Texas and California before we send notifications.

How many times we do this depends on your “Sensitivity” setting for the check. The default setting of High rechecks once from each of the other two locations. These rechecks take a few seconds each, and the notification will be sent off in around 30 seconds. A setting of “Very Low” rechecks the service ten times, and with the extra rechecking the notification gets out in around 2 minutes. A setting of “Very High” doesn’t do a recheck at all, so if any check shows the service is not responding as expected we send the notice immediately.

This addition of the multi-site rechecks is one of the features we get asked about most often, and we’re very happy to get it rolled out to our production service. We’d love to hear from you about additional features that you’d like to see in our service.

Monitoring Services Are Poised for a Shake Up

Server monitoring and website monitoring services cost too much and are overly complicated.

Over the past several months we have built and launched NodePing’s site and server monitoring service. Part of that process involved looking at the other companies in this market niche, and finding the opportunities for offering a service that fills a gap in what is currently being provided to customers. What we have found has confirmed our original reasons for starting NodePing.

There are a lot of companies offering site and server monitoring services. However, our experience as consumers of these services was that it was hard to find a provider that did what we needed at a reasonable price, and I think our experience is probably typical. Where’s the disconnect? We wanted a service that would allow us to watch twenty to thirty sites and services for a reasonable price. It is easy for a small to medium business to get to a couple of dozen services needing monitoring. Most companies have at least one or two web sites that need to be available all the time for their customers. Many also have two or three web sites used internally for collaboration and sharing or publishing information to employees (Intranets). Throw in a DNS server or two, a mail service, an accounting system, a key router or two, and you are quickly into double digits on the number of services that need to be checked.

IT departments used to run software like Nagios for this type of thing, and that is still a good option in many cases. Nagios provides a wider set of checks than a typical SaaS monitoring service, there are lots of specialized plugins available, and it is not all that difficult to write custom plugins. If you need specialized checks, a system like Nagios is probably the best bet. On the other hand, while Nagios is free software, running it is not free. It requires a server to run on. Typically you want monitoring to run on separate infrastructure from your normal servers, which often means leasing a server or using a VPS service. Doing this inexpensively typically runs $50-100 a month, and involves a non-trivial amount of technical expertise and work to setup, tune, and maintain. That’s not a huge amount of money, but it is not free.

External providers offer similar services. The majority of companies need HTTP, SMTP, and PING checks. These are the primary checks provided by the bulk of the monitoring as a service industry. These types of services don’t cost much to run. With today’s opportunities to build and deploy cloud based services in cost effective ways that scale well, the cost of these types of services should be fairly low. That’s not currently the case.

A quick search turns up a lot of companies offering these services. Many of them offer “free” or inexpensive services. “Free” monitoring is typically provided for one to five URLs, often with fifteen to thirty minute intervals. That is basically useless. If it is ok for a service to be down for 30 minutes without getting a notification, you probably don’t need monitoring. In my opinion, a price “plan” isn’t a serious offer unless they offer the service in intervals of five minutes or less at that price. Getting beyond that unhelpful “Free” level, many providers start charging by the URL or address you want to monitor. One company prominently advertises checks starting at $1, but again that’s one URL in thirty minute intervals, and it costs $11 for that URL check if you want to do check it every minute. Paying per check or per URL quickly gets expensive. It is not uncommon to find special price calculators on the sites of this kind of provider, which is itself a hint that the pricing is too complicated. At these prices, a fairly typical small to mid-sized company could easily find themselves spending hundreds of dollars a month on monitoring.

There are more competitive options out there. These companies typically cost $40-$60 for a reasonable number of addresses and services. These prices probably save you money compared to running monitoring yourself using something like Nagios. Plus, you don’t need to deal with setting up and maintaining the software. That’s a pretty good deal.

However, it still doesn’t need to cost that much. With modern hosting and technology, the cost per check and even per customer to run these types of services is very low. In fact, just about the only cost of running a service like this that is attributable to an individual account is the credit card processing. All the rest of the costs scale, and are spread in ways that actually decrease per account as you scale up. Unless they are just running very inefficient systems, the total overhead for the companies charging $40-$60 per month (not to mention the ones costing hundreds) should be less than $4 per customer. Of course, the companies advertising “Free” services are also spending dollars a click to get those accounts, and that easily becomes the biggest expense. Meanwhile, allowing their customers to add additional checks or URLs to an account costs the provider pennies. Pricing based on adding checks or URLs is a model completely detached from the economics of running the service.

Experience in running IT departments and talking to system administrators tells us that there are a lot of services that should be checked if best practices were followed that aren’t getting checked. Many companies that use external providers check their company’s primary site, but when adding checks means adding overhead costs (or just the work load), secondary and internal sites don’t get checked. This means that there are millions of services that should be monitored that aren’t getting monitored at all. Companies are just reacting to complaints when something goes down.

To us, this smelled like opportunity. It is not simple to set up a solid monitoring service. However, once the technology, infrastucture and processes are in place, it is a service that scales. The margin stays fairly stable even if you let customers use it as much as they need. This calls for a flat rate model.

Our biggest problem is that we have entered a market that is saturated by misinformation. Buyers assume that this type of service costs at least $40 for a reasonable level of monitoring, and often lots more. They expect to see low entry prices that don’t really meet anybody’s needs, followed by much higher prices for the real service. This becomes a marketing challenge. When shopping for these services, NodePing’s price of a flat $10 for monitoring sounds like one of the entry point bait ads. We say “$10 to monitor up to 1000 services in 1 minute intervals” and people ask “Yes, but what do we really get, and what’s it cost if we actually need to do real world monitoring?”

NodePing’s services really cost $10 a month. Period. There are no add-ons, no “X is available at additional cost”. We set 1000 services as the maximum because we don’t want to monitor IBM’s network (no offense to IBM intended). Our target is small to medium sized businesses, and we want them to monitor everything they want to monitor for one reasonable price. If this model works, maybe others will also move to flat rates. That’s great. We’d be happy to help make the monitoring world make more sense and be more cost effective for businesses. We think we have a solid technology stack and a great service, and we can do quite well even if other providers compete with us directly on price. Until then, there are few if any major providers that really provide the services that our customers need anywhere close to our price.

Monitoring services cost too much and are too complicated. We think this market is set for a change, similar to how the cloud has impacted other technology services. This shift will be a significant benefit to small and medium sized companies that need these services, and it is a fantastic opportunity to providers poised to provide the services the customers need at truly competitive scale and rates. NodePing has positioned itself to provide the services businesses need at a fantastic, flat-rate price.