Monitoring Cached Websites

2023/10/26 by Leave a comment

Websites that use CDN or caching services like Cloudflare or Amazon Cloudfront can be a little tricky to monitor. You need to make sure all the regular website stuff is working along with additional monitoring for the backend web server and the caching service itself.

NodePing has you covered on both ends: the regular website monitoring and some powerful features specifically for monitoring CDN websites.

The Basics

When someone wants to connect to your website, there are 4 things that have to be working correctly:

  1. DNS
  2. Routing
  3. SSL
  4. Web Server response

DNS

If your DNS servers aren’t available, your visitor’s browser won’t be able to translate the FQDN in the URL to a routable IP address so monitoring each of your nameservers is vital to website availability.

Create a DNS check for each nameserver and be sure the query for your FQDN is being answered.

Routing

Now that the browser has the IP address of a web server, it needs to be able to reach out across the interwebs and request content. Incorrect routing and packet loss can make your website unreachable.

Use our PING and MTR checks to ensure that routing is working and that there is no packet loss.

We often get questions from site owners when the monitoring says their site is failing, but they can get to it fine from their device. Upstream connectivity issues are often the reason, and having the PING or MTR check in place and running from different geographical regions can help identify those troubles quickly.

SSL

Visitors expect websites to use industry best-practices for security, including transport encryption using TLS/SSL. You don’t want them to see that embarrassing “Unsafe” warning when they hit your website because your SSL certs are expired or incorrectly configured.

Create an SSL check to warn you before the cert expires.

Web Server Response

If everything mentioned above is firing on all cylinders, the web server will respond to the visitor’s request and reply with the expected content.

Using NodePing HTTP Content checks, you can verify that the web server is returning the expected HTTP response code and content.

With the basics of website monitoring nailed down, there’s a couple of additional challenges that caching services create that need special monitoring.

Monitor the Back-end Server

If your back-end server isn’t functioning, your CDN or caching service will continue to respond normally, at least for a while. But you’ll want to know right away if that back-end server is offline. You won’t be able to monitor that back-end server using the regular FQDN in the URL because it points to the caching service, not your back-end server.

NodePing can monitor a back-end server using an IP in the URL. The IP address can be either an IPv4 or an IPv6 address.

Example: https://192.168.1.1/index.html

Example: https://%5Bfe80::ec4:7aff:fe06:c186]/index.html

Note: When using IPv6, use square brackets around the IPv6 address.

To get the SSL to respond properly, use an HTTP Advanced check and send a special request header of “Hostname” set to the FQDN of the website.

Example: “Hostname” “example.com”

Cache-busting

To make sure the caching service or CDN is able to communicate properly with your back-end server, you need to send an HTTP request to the service with a URL it hasn’t cached. That will force the service to talk with your back-end server to get fresh content. To do that, it has to be a different URL each time it’s monitored. This is called cache-busting.

NodePing has a cool cache-busting feature on the HTTP Advanced check that will slightly change the URL each time it monitors so that it always causes the caching service to talk with your back-end server.

To use cache-busting, modify the URL query string. Add a non-essential element with the value of “{{now}}”. NodePing will replace that value with a millisecond timestamp each time the check is run.

Example URL: https://nodeping.com/?cachebusting={{now}}

When the URL is run, it will look like: https://nodeping.com/?cachebusting=1697232757035

Each time it is run, the value will be different: https://nodeping.com/?cachebusting=1697232816021

Since each URL is unique each time, there will be no cached entry and the caching service will hit your back-end server on each check run. If the service isn’t able to reach your back-end server, it should return a 522 error or something similar, which will make the NodePing check fail and alert you to the issue.

Website Monitoring

Using a CDN or caching service with your website can speed things up but it can also make things break in ways that basic website monitoring may miss. NodePing’s features allow you to ensure that your back-end web server is up and running and that your caching service is operating as expected.

If you don’t yet have a NodePing account, please avail yourself to our 15-day, free trial. You’ll see why those who know, use NodePing.

Filed under DNS, HTTP, Monitoring, MTR, PING, SSL, website monitoring Tagged with , , ,

Beyond “Is It Up?” – Website Monitoring should be Comprehensive

2023/09/05 by Leave a comment

When it comes to monitoring websites, the question most often asked is, “Is the site up?” While this is certainly an essential aspect, the answer hardly paints the whole picture. True website monitoring involves a plethora of factors that can affect the user experience and performance. These factors can often be hidden, and your user’s experience of you site might not be the same as what you are seeing from your network. With NodePing’s suite of tools, you have the ability to dig deep and understand the vital aspects of your website’s functionality. Let’s explore these considerations.

DNS Monitoring

Domain Name System (DNS) is the backbone of internet navigation, converting human-friendly URLs into IP addresses. Monitoring DNS health is crucial as an unresponsive DNS can render your site unreachable. Problems with DNS can be hidden by caching, and we are often asked why we are notifying about a site that seems to be working from the owner’s perspective. The answer is often that the site owner’s browser or DNS caching is making the site appear to be working when in fact for people who haven’t been on the site recently it appears to be offline because of DNS problems. NodePing offers robust DNS checks to ensure that your DNS servers are resolving correctly.

Monitoring Status Codes

Like with DNS, just checking with a browser can also miss situations in which the web server is actually responding with an error because the modern browsers try to show the page if they can. We often get messages from customers who’s website appears to be working but is actually returning status codes that indicate errors on the site. Even if the site looks right in your browser at the moment, you need to know if it is returning a status code in the 500 range indicating the server is throwing an error. Many content management systems or frameworks also return a visible page with a 404 Not Found status. NodePing’s HTTP checks watch for status code problems with your site.

Similarly, it is important to know if your site is properly following redirects. On some checks, you want the monitoring to follow the redirect to ensure that is getting the final page, and that page is responding with a 200 status code. You may also want to test specific URL’s for the 302 response as well. NodePing’s HTTP Advanced check allows you to ensure that a URL is returning a specific redirect response code.

SSL Certificate Validation

SSL certificates encrypt data transferred between users and your servers. Monitoring and receiving warnings about certificate expirations help you maintain trust and protect sensitive user information. Many of our check types include SSL validation, and and there is also a specialized SSL check that warns you if a certificate has a problem, as well as notifying you that your certificate will expire in a certain number of days. With NodePing, stay ahead with timely reminders and validations.

Domain Registration Expiration and the WHOIS Check

Keep track of your domain registration status with NodePing’s WHOIS checks. Understanding the ownership and registration details ensures that you stay in control of your domain and can prevent unexpected downtime.

Monitoring Other Services on the Host

If your website relies on additional services like databases or caching servers, monitoring them alongside the main site is essential. Integrating these checks into your monitoring strategy ensures that all parts of your site are functioning seamlessly.

CDN & Proxy Consideration

Content Delivery Networks (CDNs) and proxies enhance site performance but can complicate monitoring. By monitoring the back-end site directly, NodePing allows you to quickly pinpoint whether the issue lies with the CDN, helping you react quickly to any problems.

Tying It Together

Every notification your receive from your monitoring system should be actionable. Otherwise it becomes noise, and you either waste time or start ignoring alerts and miss important events. It is important to monitor every aspect of your site, but you don’t necessarily want ten notifications when the site is down. NodePing allows you to set a check as being dependent on another check, so you won’t get a stack of notifications if the dependent checks fail together.

Automated Diagnostics

NodePing’s has both on demand and automated diagnostics tools that provide extra insights when your site is down, supplying valuable information to help troubleshoot and resolve issues more efficiently.

Conclusion

Monitoring a website involves much more than merely checking if it’s up. With tools like NodePing, you can dive into a multitude of factors that contribute to your site’s performance and reliability. By understanding and keeping tabs on DNS, redirects, SSL certificates, domain registration, host services, CDN considerations, and more, you ensure a smooth user experience and robust site functionality.

At NodePing, we’re committed to helping you monitor your website from all angles. Get in touch with us to learn how you can take your website monitoring to the next level. If you don’t have an account yet, give it a try with our 15-day, free trial.

Filed under diagnostics, DNS, HTTP, Using NodePing, WHOIS Tagged with , ,

SMTP Monitoring with NodePing

2023/08/22 by Leave a comment

In the world of email communication, Simple Mail Transfer Protocol (SMTP) plays a crucial role in ensuring the seamless delivery of messages. However, like any other technology, SMTP is not immune to issues that can disrupt email flow and affect business operations. To maintain a healthy and reliable email infrastructure, it is essential to monitor SMTP servers continuously. In this blog post, we will explore how NodePing can be used to monitor for SMTP functionality, packet loss, blacklisting, deferred queues, and MX/SPF records.

NodePing is a versatile and powerful server monitoring service that allows businesses to monitor their infrastructure’s performance and uptime. With its extensive range of monitoring checks, NodePing provides an excellent solution for monitoring SMTP servers and ensuring they are operating optimally.

Monitoring SMTP Functionality

Verifying the functionality of your incoming SMTP server is crucial to ensure that it can receive emails without any hiccups. NodePing’s SMTP check allows you to periodically test your SMTP server by trying to send a test email to a designated email address. If the test email is accepted, it indicates that your SMTP server is functioning correctly. In case of failures, like timeouts or server errors, NodePing will promptly alert you, enabling you to troubleshoot and rectify the issues.

Monitoring Packet Loss

Packet loss can severely impact the performance of your SMTP server and lead to email delivery delays or failures. NodePing’s ICMP PING check is a valuable tool to monitor packet loss and routing issues to to your SMTP server. By regularly performing ping tests, you can assess packet loss trends and determine whether network-related issues are affecting your email delivery. If a failure is seen, NodePing automated diagnostics will send you MTR results so you can quickly troubleshoot where the issue originates. Addressing packet loss problems promptly will lead to a smoother email experience for your users.

Monitoring SMTP Blacklisting

Blacklisting can be detrimental to email delivery, as it prevents messages from reaching their intended recipients. NodePing’s RBL check allows you to monitor your SMTP server’s IP addresses against popular DNS-based blacklists (RBLs) such as Spamhaus and Barracuda. By configuring blacklisting checks at regular intervals, you can quickly identify if your server’s IP addresses have been blacklisted, enabling you to take immediate action to resolve the issue and maintain a good sender reputation.

Monitoring Deferred Queues

When your SMTP server is unable to deliver emails to the recipient’s mailbox immediately, it sits in the deferred queue. If emails in your deferred queues are piling up, you likely have a delivery issue. NodePing’s PUSH check can watch your deferred queues and send you notifications if they rise above what you’re comfortable with; allowing you to investigate and resolve the underlying problems before they escalate.

Monitoring MX Records

MX (Mail Exchange) DNS records play a crucial role in email delivery by specifying the mail servers responsible for receiving emails for a domain. NodePing’s DNS check allows you to monitor the MX records of your domain to ensure they are correctly configured and that your DNS servers are responding with those records properly. Regular checks of MX records help you keep incoming mail flowing.

Monitoring SPF Record

Your SPF record is actually a TXT DNS record that specifies which servers are allowed to send email from your domain. If that record is missing, compromised, or your DNS servers aren’t responding, sending email may be blocked or delayed. NodePing’s DNS check will make sure your SPF record is available and hasn’t been hacked. That will keep your outgoing mail flowing.

In conclusion, email monitoring is essential to ensure the reliability and efficiency of your message communications. NodePing provides a comprehensive suite of checks that empower you to monitor for blacklisting, SMTP functionality, packet loss, deferred queues, along with MX and SPF records. By leveraging NodePing’s monitoring capabilities, you can proactively identify and address issues affecting your SMTP server’s availability and performance, leading to better email deliverability and improved customer satisfaction.

Remember, a robust email infrastructure is the backbone of modern businesses, and investing in reliable monitoring tools like NodePing with automated diagnostics is a step towards a smoother and more efficient communication system.

Start monitoring your SMTP servers with NodePing today by signing up for our free, 15-day trial and stay one step ahead of any potential email delivery challenges!

Filed under DNS, Email, Monitoring, PING, SMTP, uptime Tagged with , , , , , , , , , ,

DoH Monitoring and DoT Monitoring

2021/02/15 by Leave a comment

NodePing can verify that your DNS over HTTPS (DoH) and DNS over TLS (DoT) services are available and replying with the expected records.

DoH/DoT was created to help secure DNS queries and responses over untrusted networks and is supported by most browsers and operating systems.

Our DoH/DoT check will ensure that your DNS over HTTPS (RFC 8484) or DNS over TLS (RFC 7858) servers are working correctly and if they’re not, we’ll send you notifications.

In accordance with the RFCs, our uptime monitoring for DoH uses the common DNS wire format via HTTPS GET or POST over port 443. Our uptime monitoring for DoT also uses DNS wire format over via TLS over port 853. You can include EDNS(0) OPT records in the queries as well as verify the query response. You can add custom HTTP headers (DoH-only) and sign your requests with your own TLS client certificates for authentication.

The DoH/DoT check can be found in the standard check type drop down menu in your NodePing account. If you don’t have a NodePing account yet, please sign up for our 15-day, free trial.

Filed under Announcement, DNS Tagged with , , ,

DNSSEC Monitoring

2019/10/18 by Leave a comment

Your DNSSEC implementation is an important part of increasing authentication for your DNS data. NodePing’s DNS check can now verify DNSSEC authenticated data.

To monitor DNSSEC, NodePing uses public key cryptography to verify the digital signature in the data all the way back to the root servers. If there’s an issue anywhere in the chain, you’ll be the first to know with NodePing’s fast and accurate notifications.

Designed to protect DNS clients from using forged or modified DNS data, DNSSEC ensures the information in the DNS reply is identical to what the owner of the DNS zone has digitally signed and published on their authoritative nameservers.

Only enable DNSSEC verification in your NodePing checks if your nameservers have that functionality.

If you do not yet have a NodePing account, please sign up for our free, 15-day trial. We’re confident you’ll find our uptime monitoring to not only be the fastest and most accurate, but also a great value.

Filed under Announcement, DNS Tagged with ,

Auditing IP Resolution with DNS Checks

2019/02/19 by Leave a comment

How would you know if your DNS account had been compromised?  If tampered with, an attacker could point your web and email traffic to their own controlled servers, enabling them to intercept potentially confidential information from you or your customers without your knowledge.

Emergency Directive 19-01

Recently, the US Department of Homeland Security issued its first ever Emergency Directive with a list of actions to mitigate DNS account tampering, an issue they report is on the raise.

The first recommended actions in their directive is to verify DNS resolution.

Action One: Audit DNS Records … audit public DNS records on all authoritative and secondary DNS servers to verify they resolve to the intended location.

In this post, I’ll show you how to continually monitor your DNS resolution using NodePing DNS checks to ensure your important domain names are resolving to the expected IP addresses. If anyone tamplers with your DNS records, you’ll quickly receive actionable notifications from NodePing.

Some of the record types you may want to verify with DNS checks are:

  • SOA – Start of Authority record
  • NS – Nameservers and the IPs they resolve to
  • Website FQDN
  • Website FQDN with www prefix (example: http://www.nodeping.com)
  • Email MX records and the IPs they resolve to
  • IMAP, POP, and SMTP FQDNs
  • FTP service FQDNs
  • All the above services for both IPv4 and IPv6 addresses

Setting up DNS monitoring

To create a new DNS check, click on the “Add new check” button in your NodePing account dashboard.

  1. Select DNS from the Check type drop down.
  2. Give it a friendly label to identify this check in lists and notifications.  Something like “Website resolver”
  3. Set how often you want the check to run on the Check Frequency field.  We recommend 1 minute intervals.
  4. Leave the DNS server field blank.  This will ensure that our probes will use whatever nameservers are listed on the domain.  If a hijacker accesses your DNS account , they’ll likely change the IPs of the nameservers so testing your own nameservers won’t be helpful for this type of monitoring.
  5. Enter the type of query you want to perform, and address you want the check to look up. Usually this should be a fully qualified domain name. It should not include “http://” or “https://”. Example : ‘nodeping.com’ or ’email.nodeping.com’ or ‘www.nodeping.com’
  6. Enter the information the check should look for in the DNS resolution response to verify the query has not been tampered with. What you put in this field will depend on the query type. For example, for A records, this will be your IPv4 address. For other types, such as MX or NS records, this is likely to be a fully qualified domain name. For AAAA records, the full notation is required. Example: IPv6 address 2606:c700:4020:11::53:4a3b requires the ‘missing’ zero sections – 2606:c700:4020:11:0:0:53:4a3b – there should be 8 sections total.
  7. Set a time out. The default 5 seconds works fine for most situations.
  8. Set the Sensitivity. High is usually appropriate.
  9. Set the notifications for this check. More information about notifications.

If your services are offered on both IPv4 and IPv6, you’ll need to create a separate check for each with the appropriate query type set to ‘A’ for IPv4 and ‘AAAA’ for IPv6.

Setting up one DNS check for every critical server and service will give you the peace of mind that your DNS hasn’t been tampered with and your customers are interacting with you, not some DNS hijacker.

Got questions or need help setting up DNS resolution audit checks?  Contact us; we’re happy to help.

If you don’t yet have a NodePing account, please sign up for our free, 15-day trial and sleep well knowing we’re keeping an eye on your DNS resolution.

Filed under DNS, Monitoring, Using NodePing Tagged with ,

Diagnostic Tools

2017/02/02 by Leave a comment

“Why is my check failing?”

It isn’t always obvious what’s causing the failure when a check does ‘down’ and additional information about what our probes are experiencing can be helpful. For example, if your website is timing out, is it the web server, a DNS problem, or maybe packet loss on the network?

Our new diagnostic tools allow you to run several utilities on our probes and give visibility to what our probes are seeing to help you troubleshoot a failing service. These tools can be useful to narrow down where the failure is so you can get things fixed and services restored as quickly as possible.

Tools available:

  • Ping
  • Traceroute
  • MTR
  • Dig
  • Page Load (browser loading with page speed – HAR viewer)
  • Screenshot

More information about the tools and some troubleshooting advice can be found in our documentation.

You can find these tools on the “Diagnostic Tools” tab when you login to your NodePing account.  If you don’t yet have a NodePing account, you can create one and try out these tools with our 15-day, free uptime monitoring trial.

What other tools would be helpful on that page? Let us know in the comments.

Filed under Announcement, Browser testing, DNS, HTTP Parse, MTR, NodePing Updates, PING, Screenshot, Traceroute Tagged with , , , , , , , , , ,

rDNS: Monitoring the Flip Side of DNS

2014/02/24 by Leave a comment

DNS monitoring is an important part of keeping your services available. DNS is what allows your browser to turn the name ‘nodeping.com’ into the IP address 192.95.37.22. Without proper functioning DNS, your website, email, and other services would be unreachable.

One often neglected part of DNS monitoring is the rDNS, or reverse DNS, entries. As the name suggests, rDNS is the reverse of DNS. It maps an IP address to a hostname using a special PTR DNS record type. In essence, it associates an IP address to a specific hostname or domain.

PTR records are used by all kinds of utilities and services like the humble ‘ping’ and ‘traceroute’ as well as more complex FCrDNS-enabled services. A forward-confirmed reverse DNS (FCrDNS) verification uses the PTR record to associate a domain owner with an IP address. It’s not a rock-solid form of validating ownership but is usually considered enough to be used for whitelisting servers for SMTP services because spammers usually can’t fake an rDNS record when they forge domains.

If you’re sending email from a server, you should have a proper PTR record in place that includes the domain for the ‘from’ address. This will help ensure your email from that server will not get sent to the spam bucket. A PTR, or ‘pointer’, record is usually configured by whomever owns the IP address so you often have to put in a ticket with your colocation or service provider to set or change rDNS entries.

NodePing is one of very few server monitoring services that can monitor rDNS entries. To set up a rDNS check, select ‘DNS’ from the check type and ‘PTR’ from the record type dropdown. It’s important to note that your PTR record will be in what is commonly called ‘arpa’ format. An example of the ‘arpa’ format for the IP 192.95.37.22 is ‘22.37.95.192.in-addr.arpa’ – please note how the octets are in reverse order, not the numbers. Set the ‘arpa’ address for your IP address in the ‘Query’ text field. You’ll also want to set the ‘Expected Response:’ field to the hostname for that IP, example: ‘api.nodeping.com’.

For more information about rDNS/PTR monitoring or our DNS check capabilities in general, check out our DNS check documentation.

Filed under DNS, NodePing

DNS Monitoring for Both Sides

2011/11/11 by 4 Comments

DNS monitoring, like a coin, has two sides: “What does my DNS server say?” and “What does ‘public’ DNS say?”  With NodePing server monitoring, you can ask both questions.

Our DNS check allows you to send a query of a specific type to your DNS server (or a public DNS server) and test the response against a string you define.  For example, you can verify that your website domain resolves to your web server’s static IP address and have NodePing send you an email or SMS alert when either the server or the response fails.

DNS queries can be made for the following types and the response verified:

  • A
  • CNAME
  • MX
  • NS
  • PTR
  • SOA
  • TXT

You can find more info on the DNS checks and our other check types in our documentation.

If you don’t have a NodePing account yet, try out our new DNS monitoring checks for free with a 15-day trial.

Filed under DNS, Monitoring, NodePing, NodePing Updates, Using NodePing Tagged with ,