Multi-site checks added to NodePing

This week we’ve launched pinghosts in two additional locations with automatic rechecking across locations. So if the New Jersey pinghost reports a monitored site is down, we now automatically recheck it from Texas and California before we send notifications.

How many times we do this depends on your “Sensitivity” setting for the check. The default setting of High rechecks once from each of the other two locations. These rechecks take a few seconds each, and the notification will be sent off in around 30 seconds. A setting of “Very Low” rechecks the service ten times, and with the extra rechecking the notification gets out in around 2 minutes. A setting of “Very High” doesn’t do a recheck at all, so if any check shows the service is not responding as expected we send the notice immediately.

This addition of the multi-site rechecks is one of the features we get asked about most often, and we’re very happy to get it rolled out to our production service. We’d love to hear from you about additional features that you’d like to see in our service.

Monitoring Services Are Poised for a Shake Up

Server monitoring and website monitoring services cost too much and are overly complicated.

Over the past several months we have built and launched NodePing’s site and server monitoring service. Part of that process involved looking at the other companies in this market niche, and finding the opportunities for offering a service that fills a gap in what is currently being provided to customers. What we have found has confirmed our original reasons for starting NodePing.

There are a lot of companies offering site and server monitoring services. However, our experience as consumers of these services was that it was hard to find a provider that did what we needed at a reasonable price, and I think our experience is probably typical. Where’s the disconnect? We wanted a service that would allow us to watch twenty to thirty sites and services for a reasonable price. It is easy for a small to medium business to get to a couple of dozen services needing monitoring. Most companies have at least one or two web sites that need to be available all the time for their customers. Many also have two or three web sites used internally for collaboration and sharing or publishing information to employees (Intranets). Throw in a DNS server or two, a mail service, an accounting system, a key router or two, and you are quickly into double digits on the number of services that need to be checked.

IT departments used to run software like Nagios for this type of thing, and that is still a good option in many cases. Nagios provides a wider set of checks than a typical SaaS monitoring service, there are lots of specialized plugins available, and it is not all that difficult to write custom plugins. If you need specialized checks, a system like Nagios is probably the best bet. On the other hand, while Nagios is free software, running it is not free. It requires a server to run on. Typically you want monitoring to run on separate infrastructure from your normal servers, which often means leasing a server or using a VPS service. Doing this inexpensively typically runs $50-100 a month, and involves a non-trivial amount of technical expertise and work to setup, tune, and maintain. That’s not a huge amount of money, but it is not free.

External providers offer similar services. The majority of companies need HTTP, SMTP, and PING checks. These are the primary checks provided by the bulk of the monitoring as a service industry. These types of services don’t cost much to run. With today’s opportunities to build and deploy cloud based services in cost effective ways that scale well, the cost of these types of services should be fairly low. That’s not currently the case.

A quick search turns up a lot of companies offering these services. Many of them offer “free” or inexpensive services. “Free” monitoring is typically provided for one to five URLs, often with fifteen to thirty minute intervals. That is basically useless. If it is ok for a service to be down for 30 minutes without getting a notification, you probably don’t need monitoring. In my opinion, a price “plan” isn’t a serious offer unless they offer the service in intervals of five minutes or less at that price. Getting beyond that unhelpful “Free” level, many providers start charging by the URL or address you want to monitor. One company prominently advertises checks starting at $1, but again that’s one URL in thirty minute intervals, and it costs $11 for that URL check if you want to do check it every minute. Paying per check or per URL quickly gets expensive. It is not uncommon to find special price calculators on the sites of this kind of provider, which is itself a hint that the pricing is too complicated. At these prices, a fairly typical small to mid-sized company could easily find themselves spending hundreds of dollars a month on monitoring.

There are more competitive options out there. These companies typically cost $40-$60 for a reasonable number of addresses and services. These prices probably save you money compared to running monitoring yourself using something like Nagios. Plus, you don’t need to deal with setting up and maintaining the software. That’s a pretty good deal.

However, it still doesn’t need to cost that much. With modern hosting and technology, the cost per check and even per customer to run these types of services is very low. In fact, just about the only cost of running a service like this that is attributable to an individual account is the credit card processing. All the rest of the costs scale, and are spread in ways that actually decrease per account as you scale up. Unless they are just running very inefficient systems, the total overhead for the companies charging $40-$60 per month (not to mention the ones costing hundreds) should be less than $4 per customer. Of course, the companies advertising “Free” services are also spending dollars a click to get those accounts, and that easily becomes the biggest expense. Meanwhile, allowing their customers to add additional checks or URLs to an account costs the provider pennies. Pricing based on adding checks or URLs is a model completely detached from the economics of running the service.

Experience in running IT departments and talking to system administrators tells us that there are a lot of services that should be checked if best practices were followed that aren’t getting checked. Many companies that use external providers check their company’s primary site, but when adding checks means adding overhead costs (or just the work load), secondary and internal sites don’t get checked. This means that there are millions of services that should be monitored that aren’t getting monitored at all. Companies are just reacting to complaints when something goes down.

To us, this smelled like opportunity. It is not simple to set up a solid monitoring service. However, once the technology, infrastucture and processes are in place, it is a service that scales. The margin stays fairly stable even if you let customers use it as much as they need. This calls for a flat rate model.

Our biggest problem is that we have entered a market that is saturated by misinformation. Buyers assume that this type of service costs at least $40 for a reasonable level of monitoring, and often lots more. They expect to see low entry prices that don’t really meet anybody’s needs, followed by much higher prices for the real service. This becomes a marketing challenge. When shopping for these services, NodePing’s price of a flat $10 for monitoring sounds like one of the entry point bait ads. We say “$10 to monitor up to 1000 services in 1 minute intervals” and people ask “Yes, but what do we really get, and what’s it cost if we actually need to do real world monitoring?”

NodePing’s services really cost $10 a month. Period. There are no add-ons, no “X is available at additional cost”. We set 1000 services as the maximum because we don’t want to monitor IBM’s network (no offense to IBM intended). Our target is small to medium sized businesses, and we want them to monitor everything they want to monitor for one reasonable price. If this model works, maybe others will also move to flat rates. That’s great. We’d be happy to help make the monitoring world make more sense and be more cost effective for businesses. We think we have a solid technology stack and a great service, and we can do quite well even if other providers compete with us directly on price. Until then, there are few if any major providers that really provide the services that our customers need anywhere close to our price.

Monitoring services cost too much and are too complicated. We think this market is set for a change, similar to how the cloud has impacted other technology services. This shift will be a significant benefit to small and medium sized companies that need these services, and it is a fantastic opportunity to providers poised to provide the services the customers need at truly competitive scale and rates. NodePing has positioned itself to provide the services businesses need at a fantastic, flat-rate price.

Website monitoring with a backflip

A standard website monitoring check will fail when the page isn’t returned at all or the web server reports a page missing. What happens when your site is running but there is a problem with dynamic content, like a feed is missing or a list of recent posts is empty?

In those cases the page might be “working” from the web server’s point of view (and so not reported as a 404 or 500 error), but not displaying what you want. You don’t want your visitors to see messages like “Error establishing a database connection” or “0 articles found“.

NodePing HTTP Content check tests if particular text shows up on a given webpage.  Use the setting ‘Contains‘ to be alerted when specific text does NOT appear on a page.  But in this case, we want to receive an alert when our error messages DO appear on the page.  Use the ‘Does not contain‘ setting and the error message text as the search term to be notified when that happens.

For example, if you had an article list that was dynamic, so you never knew exactly what was going to show up there but you know something is wrong if the text ‘0 articles found‘ appears.  Maybe the database is offline or you haven’t written anything recently enough.  You’ll want to receive an alert.

Simply configure a HTTP Content check for the page and switch the text setting to ‘Does not contain‘ and add ‘0 articles found‘ to the text area.  This will check the webpage and as long as it does NOT contain the words ‘0 articles found‘, the check will pass.  If that text ever shows up, the check will fail and you’ll receive an alert, as expected.

There’s a thousand other uses for the HTTP Content check.  Get creative and make sure you’re alerted when errors happen.

5 Basic Questions About Web Site Monitoring

The other day I was looking for a place to get some good Mexican food. That’s fairly easy in my part of the world, but I was looking for somewhere I hadn’t eaten before. I found a place the same way I always do, on the web. I typed my search into a search engine, pulled up a map of my area, and started clicking on web sites. I looked through the menu for each place I found and picked based on my impression of the restaurant from the web site.

I do this same kind of thing for all kinds of businesses at least several times a week. Plumbing parts, accountants, property management companies, mechanics, toys, web design, banks… pretty much everything. If I am looking for a business, I find them on the web. If it’s not on the web, I will probably not find it.

Increasingly business relies on the Internet, even for non-Internet businesses. If your site is down, you lose business. If your email doesn’t go through, you lose business. If your business is technology or web related, this is doubly crucial. People’s impression depends largely on how you come across on the Internet, and if it doesn’t work you are in trouble.

That seems obvious, but how do you ensure that everything works all the time? If you are a large enough business to have a highly skilled IT department, they are using monitoring tools or services. If you are not a large enough business to keep IT staff on the payroll, this largely falls to you. How do you make sure your Internet presence is a plus for your business, bringing in new customers instead of driving them away? How do you do that without spending too much money and too much time?

Monitoring services exist specifically for this purpose. Technology professionals use monitoring services to make sure that the services they are responsible for are always available. This has been the normal way to do business for tech professionals for many years. There are sophisticated tools to help with this job that can monitor all kinds of things and notify someone immediately if there are problems. Until more recently, doing this inexpensively without spending a lot of time was out of reach for most people. Not any more.

A few years ago monitoring as a service started to pop up on the Internet. Now there are a number of companies out there providing these types of services. Some of them are easy to use, some are not. Many of them cost a lot, but a few do not. Some of them sell snake oil and fancy gadgets that don’t really tell you what you need to know. Increasingly, smart IT technical people are realizing that they can save time and money by using outside services to do things they had to do before themselves, and these same services are available to everybody without requiring a significant investment or a lot of knowledge.

If you are not a tech professional, and you are thinking about finding a way to make sure that your Internet presence is always there when your customers are looking for you, you might be asking questions like these:

  1. Is it easy? There is absolutely no reason monitoring should be hard to set up or use. If it is hard or takes you more than a few minutes to get going, chose a different service provider. You don’t need to know a lot to use a good monitoring service. In most cases all you need is the address of your web site or email service.
  2. How does it work? Monitoring services mostly all work about the same way. You login to the web site and create checks for the monitoring service. Setting up the check generally consists of typing in the address of your web sites and how often you want them checked. Sometimes there are a few other simple questions, but it doesn’t need to be more complicated than that. You also enter in email addresses or phone numbers to notify when your site or service is down. Typically the service takes it from there and starts monitoring right away. Monitoring services just connect to your site and log what happened. It’s all automated.
  3. How do I know what I need? Just about any monitoring service will do the checks that most businesses need. If you have specific needs in your monitoring, this might be something to shop around for. However, most businesses need HTTP checks, which is the basic check that makes sure a web site is up, and SMTP, which checks email services. Just about all monitoring companies do HTTP checks, and most of them do SMTP.
  4. How often should it check? This is up to you, but if you are using a service that checks every 10 or 15 minutes, your site could be down for several minutes before you know about it. The better services check as frequently as every minute. This is not a lot more expensive for the service to provide, and it should not cost you a lot more either.
  5. How much will it cost? This is currently the biggest differentiator in the monitoring business. Some monitoring services cost a lot, especially if you have more than a couple of web sites to watch. It doesn’t have to be expensive. NodePing costs a flat rate of $10 per month to check up to a thousand sites or services every minute. If you’re paying more than that, you’re paying too much. If a service needs a special calculator or a talk with a sales person to tell you how much it will cost, it’s too much. If prices are per check, read the fine print. It should be inexpensive, and it should be simple.

If you are not doing website and email monitoring yet you should start today. We think that NodePing is a great choice, but there are other good providers out there. Shop around. It is important to your success, it is easy, and it is inexpensive. You just have to do it.

Dear PHP, I’m leaving and yes, she’s sexier

Dear PHP,

I know this letter won’t come as much of a surprise to you.  We’ve been growing apart for a while now but today we officially part ways.

It wasn’t easy to write this.  You and I have a lot of history.  Hard to believe it was over 10 years ago when you welcomed me into your arms.  You were young, sexy, and a breath of fresh air compared to my ex, Perl (shudder – lets not go there).  It didn’t take long for our relationship to start paying the bills.  In fact, every job I’ve had in the last decade had you on a pedestal.

We had plenty of good times.  Remember how we survived a front-page CNN link and pulled in $500k in 14 days?  And all the dynamic PDF creation over the years still brings a smile on my face.

But we had our rough times too.  I could go the rest of my life without ever hearing the words ‘register globals‘ again.  And you know quite well that I still bear the scars from creating SOAP clients with you.  While neither of us has been truly faithful (what ever happened to V6 and UTF-8?), we’ve always been able to work out our differences up to now.

But starting tomorrow, for the first time in 10 years, my ‘day job’ won’t include you.  I’m leaving you for node.js.  We were introduced by our mutual friend, JQuery.  At first I thought she was just the latest flavor of the month; popular among the guys on the mailing lists but now I’ve fallen victim to her async charm and really think we have a future together.

When you and I started having fun on the couch a year ago, I thought maybe our relationship would just keep going.  But then node and I spent some time on that couch and – oh, what she can do with JSON makes my toes curl.  To be perfectly honest, you just can’t compete with her.  She’s all that you used to be – young, sexy, and fast.  I’m sure some of your other boyfriends will probably argue with me about it but I’ve been smitten.  While they’re fighting with you over namespacing, she and I will be branching in non-blocking ways and spawning like crazy.

I’m not saying we’ll never see each other again – heck, you’re even serving this blog.  But I’m moving on and I hope you will too.  If you want to see what node.js and I are up to, stop by some time.  Maybe we can even help you keep an eye on all those fatal exceptions of yours.

Sincerely,
Shawn

Someone should create a service like that!

Some time ago Shawn and I were lamenting about what a pain network and service monitoring can be. There are some very good open source applications out there for doing this sort of thing. We’ve both used several versions of Nagios, and it works really well. If you need to run your own monitoring or write your own custom plugins (which we’ve done in the past), that’s a good option. If what you want is to monitor a bunch of services easily without having to put up and maintain another server just for that, a service that does it for you is more attractive.

There are a number of services out there that do pings, HTTP checks and a variety of other checks with notification. Some of them even start out at low cost or free, but if you have more than a handful of hosts and ports to monitor, they get pricey fast, or they don’t let you check very often, or they have some other catch that makes them just not do what you want. Some of them you need a special graduate degree from MIT to understand the pricing.

So we were trying to figure out how to get monitoring done reliably and cost effectively for a set of services we were responsible for at the time, and saying to each other “Someone should create a service that is easy, just does what you need it to do reliably, and doesn’t cost a lot.” Someone, as it turns out, was us.

More recently Shawn and I were once again chatting about the kinds of things geeks talk about, and one of those things was Node.js. I had been working on a few projects just as a proof of concept. It was clear that Node.js has some real strengths for writing scalable asynchronous services. In the course of the conversation, it occurred to us that we could create a service that would scale to many thousands of checks with very low incremental cost. If someone wanted to check thirty or fifty hosts every minute, the cost would be very similar to checking three sites every fifteen minutes. NodePing was born.

The name NodePing stuck with us, not because it uses Node.js (although it does, and we’re proud of that fact), but because “node” refers generically to something on the network. Of course, it’s much wider than that, and the most common checks don’t turn out to be pings. We think the name NodePing conveys “checks on things on the ‘net” well, even beyond pinging nodes. Our goal is to let you check what you want, when you want, for not much money.

As we wrap up our initial testing (with thanks to our beta testers for some great feedback) and move towards taking on customers in real quantity, I recognize that getting here has been quite a process from that first conversation about how lousy the options were for monitoring. I wish this service had been available when I was responsible for a range of web and email services years ago. It would have made life easier, for a great value at the price. We hope you see it that way too.

What do you want from a monitoring service? We are creating the service we wished we’d have had. What would you add? Is there something you’ve been frustrated about monitoring services, and just wish someone would fix already? Let us know!