Monitoring Cached Websites

2023/10/26 by Leave a comment

Websites that use CDN or caching services like Cloudflare or Amazon Cloudfront can be a little tricky to monitor. You need to make sure all the regular website stuff is working along with additional monitoring for the backend web server and the caching service itself.

NodePing has you covered on both ends: the regular website monitoring and some powerful features specifically for monitoring CDN websites.

The Basics

When someone wants to connect to your website, there are 4 things that have to be working correctly:

  1. DNS
  2. Routing
  3. SSL
  4. Web Server response

DNS

If your DNS servers aren’t available, your visitor’s browser won’t be able to translate the FQDN in the URL to a routable IP address so monitoring each of your nameservers is vital to website availability.

Create a DNS check for each nameserver and be sure the query for your FQDN is being answered.

Routing

Now that the browser has the IP address of a web server, it needs to be able to reach out across the interwebs and request content. Incorrect routing and packet loss can make your website unreachable.

Use our PING and MTR checks to ensure that routing is working and that there is no packet loss.

We often get questions from site owners when the monitoring says their site is failing, but they can get to it fine from their device. Upstream connectivity issues are often the reason, and having the PING or MTR check in place and running from different geographical regions can help identify those troubles quickly.

SSL

Visitors expect websites to use industry best-practices for security, including transport encryption using TLS/SSL. You don’t want them to see that embarrassing “Unsafe” warning when they hit your website because your SSL certs are expired or incorrectly configured.

Create an SSL check to warn you before the cert expires.

Web Server Response

If everything mentioned above is firing on all cylinders, the web server will respond to the visitor’s request and reply with the expected content.

Using NodePing HTTP Content checks, you can verify that the web server is returning the expected HTTP response code and content.

With the basics of website monitoring nailed down, there’s a couple of additional challenges that caching services create that need special monitoring.

Monitor the Back-end Server

If your back-end server isn’t functioning, your CDN or caching service will continue to respond normally, at least for a while. But you’ll want to know right away if that back-end server is offline. You won’t be able to monitor that back-end server using the regular FQDN in the URL because it points to the caching service, not your back-end server.

NodePing can monitor a back-end server using an IP in the URL. The IP address can be either an IPv4 or an IPv6 address.

Example: https://192.168.1.1/index.html

Example: https://%5Bfe80::ec4:7aff:fe06:c186]/index.html

Note: When using IPv6, use square brackets around the IPv6 address.

To get the SSL to respond properly, use an HTTP Advanced check and send a special request header of “Hostname” set to the FQDN of the website.

Example: “Hostname” “example.com”

Cache-busting

To make sure the caching service or CDN is able to communicate properly with your back-end server, you need to send an HTTP request to the service with a URL it hasn’t cached. That will force the service to talk with your back-end server to get fresh content. To do that, it has to be a different URL each time it’s monitored. This is called cache-busting.

NodePing has a cool cache-busting feature on the HTTP Advanced check that will slightly change the URL each time it monitors so that it always causes the caching service to talk with your back-end server.

To use cache-busting, modify the URL query string. Add a non-essential element with the value of “{{now}}”. NodePing will replace that value with a millisecond timestamp each time the check is run.

Example URL: https://nodeping.com/?cachebusting={{now}}

When the URL is run, it will look like: https://nodeping.com/?cachebusting=1697232757035

Each time it is run, the value will be different: https://nodeping.com/?cachebusting=1697232816021

Since each URL is unique each time, there will be no cached entry and the caching service will hit your back-end server on each check run. If the service isn’t able to reach your back-end server, it should return a 522 error or something similar, which will make the NodePing check fail and alert you to the issue.

Website Monitoring

Using a CDN or caching service with your website can speed things up but it can also make things break in ways that basic website monitoring may miss. NodePing’s features allow you to ensure that your back-end web server is up and running and that your caching service is operating as expected.

If you don’t yet have a NodePing account, please avail yourself to our 15-day, free trial. You’ll see why those who know, use NodePing.

Filed under DNS, HTTP, Monitoring, MTR, PING, SSL, website monitoring Tagged with , , ,

SMTP Monitoring with NodePing

2023/08/22 by Leave a comment

In the world of email communication, Simple Mail Transfer Protocol (SMTP) plays a crucial role in ensuring the seamless delivery of messages. However, like any other technology, SMTP is not immune to issues that can disrupt email flow and affect business operations. To maintain a healthy and reliable email infrastructure, it is essential to monitor SMTP servers continuously. In this blog post, we will explore how NodePing can be used to monitor for SMTP functionality, packet loss, blacklisting, deferred queues, and MX/SPF records.

NodePing is a versatile and powerful server monitoring service that allows businesses to monitor their infrastructure’s performance and uptime. With its extensive range of monitoring checks, NodePing provides an excellent solution for monitoring SMTP servers and ensuring they are operating optimally.

Monitoring SMTP Functionality

Verifying the functionality of your incoming SMTP server is crucial to ensure that it can receive emails without any hiccups. NodePing’s SMTP check allows you to periodically test your SMTP server by trying to send a test email to a designated email address. If the test email is accepted, it indicates that your SMTP server is functioning correctly. In case of failures, like timeouts or server errors, NodePing will promptly alert you, enabling you to troubleshoot and rectify the issues.

Monitoring Packet Loss

Packet loss can severely impact the performance of your SMTP server and lead to email delivery delays or failures. NodePing’s ICMP PING check is a valuable tool to monitor packet loss and routing issues to to your SMTP server. By regularly performing ping tests, you can assess packet loss trends and determine whether network-related issues are affecting your email delivery. If a failure is seen, NodePing automated diagnostics will send you MTR results so you can quickly troubleshoot where the issue originates. Addressing packet loss problems promptly will lead to a smoother email experience for your users.

Monitoring SMTP Blacklisting

Blacklisting can be detrimental to email delivery, as it prevents messages from reaching their intended recipients. NodePing’s RBL check allows you to monitor your SMTP server’s IP addresses against popular DNS-based blacklists (RBLs) such as Spamhaus and Barracuda. By configuring blacklisting checks at regular intervals, you can quickly identify if your server’s IP addresses have been blacklisted, enabling you to take immediate action to resolve the issue and maintain a good sender reputation.

Monitoring Deferred Queues

When your SMTP server is unable to deliver emails to the recipient’s mailbox immediately, it sits in the deferred queue. If emails in your deferred queues are piling up, you likely have a delivery issue. NodePing’s PUSH check can watch your deferred queues and send you notifications if they rise above what you’re comfortable with; allowing you to investigate and resolve the underlying problems before they escalate.

Monitoring MX Records

MX (Mail Exchange) DNS records play a crucial role in email delivery by specifying the mail servers responsible for receiving emails for a domain. NodePing’s DNS check allows you to monitor the MX records of your domain to ensure they are correctly configured and that your DNS servers are responding with those records properly. Regular checks of MX records help you keep incoming mail flowing.

Monitoring SPF Record

Your SPF record is actually a TXT DNS record that specifies which servers are allowed to send email from your domain. If that record is missing, compromised, or your DNS servers aren’t responding, sending email may be blocked or delayed. NodePing’s DNS check will make sure your SPF record is available and hasn’t been hacked. That will keep your outgoing mail flowing.

In conclusion, email monitoring is essential to ensure the reliability and efficiency of your message communications. NodePing provides a comprehensive suite of checks that empower you to monitor for blacklisting, SMTP functionality, packet loss, deferred queues, along with MX and SPF records. By leveraging NodePing’s monitoring capabilities, you can proactively identify and address issues affecting your SMTP server’s availability and performance, leading to better email deliverability and improved customer satisfaction.

Remember, a robust email infrastructure is the backbone of modern businesses, and investing in reliable monitoring tools like NodePing with automated diagnostics is a step towards a smoother and more efficient communication system.

Start monitoring your SMTP servers with NodePing today by signing up for our free, 15-day trial and stay one step ahead of any potential email delivery challenges!

Filed under DNS, Email, Monitoring, PING, SMTP, uptime Tagged with , , , , , , , , , ,

Finding the Best Server Providers

2023/08/01 by Leave a comment

Great services need great boxes to run on. How do we know if a server or VPS host is performant and reliable?

We use dozens of different hosts for NodePing and our standards for performance and reliability are really high. There are many SaaS out there that host only on AWS. Putting all your eggs in one basket is nice for billing but would make our service fragile and vendor-dependent. We spread our boxes around to make it resilient and better represent the Internet’s disperate architecture for monitoring.

We have to take new boxes out and put them through their paces; kick the tires and make sure they’re solid. This is how we test out a new provider before we use a dedicated server or VPS.

Blacklisted

As soon as we have our IP assignments from the provider, we check to make sure the IPs aren’t listed in any spam blacklists using NodePing RBL checks. Most of our hosts don’t send any actual email but our public probes do a lot of SMTP connections to ensure our customers’ mail servers are functioning properly. If the IPs are blacklisted, we’ll need a clean IP from the provider or cancel and look elsewhere.

We’ll leave this RBL check running once an hour to make sure it doesn’t get listed half way through our testing period.

Blacklisted IPs can be a good indicator of provider quality even if the server won’t be sending any email. A provider that can’t keep spammers out of their service is unlikely to be able to keep a reliable network.

Incoming Traffic

Solid networks can be hard to find. We test for inbound packet-loss and routing issues using NodePing PING checks. We’ll sometimes test from a few different geographical regions to ensure global routing is stable. Anything less than 100% uptime for 30 days is unacceptable for us. If the provider had announced planned maintenance well in advance, we’d use NodePing’s maintenance feature to ensure the uptime stats remained accurate despite planned outages. In our decade-plus experience, a network that sees even one episode of packet-loss or route failure is going to continue to see them and isn’t stable enough for our use.

We’ll do the same for IPv6 addresses as routing and packet-loss can be independent of the IPv4 stack. Some providers have a hard time keeping their IPv6 blocks broadcasted and we’ve seen IPv6 completely fail while IPv4 continued to function normally.

We enable automated diagnostics for all our PING checks so we can see where on the route the packet-loss or routing failure is happening. Getting immediate MTRs can show us the weak links in a network and if we see issues with some of the usual suspects, we will for sure dump it. Yes, I’m looking at you, Cogent!

Outbound Traffic

Sometimes a network issue seems to only impact outbound routing. We use the AGENT functionality to assign additional PING checks to originate from the server being tested towards some of the other servers it would be connecting to if it’s moved into production. The AGENT software will run NodePing checks just like the public probes but originating from our test host. It’s a great way to detect outbound packet-loss and routing issues from the server. Again, anything less than 100% uptime on this test and the service isn’t going to make muster.

System Load

The performance of a VPS can be greatly impacted by issues outside our control. Two of the most frequent system load issues we’ve seen on VPS are noisy neighbors and host server backups.

A good provider won’t oversell their VPS host servers and will suspend anyone who is abusing more than their fair share of resources. If we end up on a box with noisy neighbors, the system load on our VPS will likely spike, starving our processes from getting the CPU, memory, networking, or storage I/O they need to function properly.

We’ve also come across providers where we saw system load rise every Saturday around midnight (GMT) for 30 mins or so. Turned out their backup process was overwhelming the disks and causing load issues on all the VPS on the host.

These types of issues are simple to find using PUSH checks that monitor the system load. Since we aren’t using these boxes for anything yet, we have to set the thresholds pretty low to detect load issues caused by resource starvation. This is one test that we’ll give a bit of slack to a provider if it fails though. Noisy neighbors or hungry backups can happen to any provider and we’ll give them a chance to find and address the cause. If it keeps happening though, pull the plug on that provider. It’ll just be worse once you start using the machine and an ongoing headache trying to get their support to do anything about it.

If a server can keep humming along for 30 days without any of the checks above failing, there’s a pretty good chance that provider and network are going to be solid and reliable. I hope this look into our vetting process will help you with your provider search for those elusive reliable networks and servers.

If you don’t yet use NodePing, please sign up for our free, 15-day trial and see for yourself how our monitoring can increase your uptime.

Filed under AGENT, diagnostics, Monitoring, MTR, NodePing, PING, PUSH, RBL, SMTP, System Monitoring, uptime Tagged with , , , , ,

Diagnostic Tools

2017/02/02 by Leave a comment

“Why is my check failing?”

It isn’t always obvious what’s causing the failure when a check does ‘down’ and additional information about what our probes are experiencing can be helpful. For example, if your website is timing out, is it the web server, a DNS problem, or maybe packet loss on the network?

Our new diagnostic tools allow you to run several utilities on our probes and give visibility to what our probes are seeing to help you troubleshoot a failing service. These tools can be useful to narrow down where the failure is so you can get things fixed and services restored as quickly as possible.

Tools available:

  • Ping
  • Traceroute
  • MTR
  • Dig
  • Page Load (browser loading with page speed – HAR viewer)
  • Screenshot

More information about the tools and some troubleshooting advice can be found in our documentation.

You can find these tools on the “Diagnostic Tools” tab when you login to your NodePing account.  If you don’t yet have a NodePing account, you can create one and try out these tools with our 15-day, free uptime monitoring trial.

What other tools would be helpful on that page? Let us know in the comments.

Filed under Announcement, Browser testing, DNS, HTTP Parse, MTR, NodePing Updates, PING, Screenshot, Traceroute Tagged with , , , , , , , , , ,