Monitoring Web Applications in Dynamically Growing Environments

The reliable operation of applications, and consequently – customer satisfaction – is the key element in operating a business efficiently. Monitoring processes that can quickly detect a failure, find hidden dependencies and even predict a failure are essential tools in such dynamically growing environments.


You will learn from this article:

  • What is application monitoring
  • What tools are used to monitor environments

A Quick Introduction to Application Monitoring

The idea of monitoring can contain many different functionalities:

  • sending notifications when a failure occurs 
  • visualizing data collected from various sources 
  • data mining 
  • establishing relationships between certain facts and their consequences 
  • predicting failures 

Below, we will discuss the most important application monitoring tools, such as IcingaGrafanaKibanaPrometheus, and AWS CloudWatch.

Overview of application monitoring tools 

Almost every monitoring system is based on metrics and the thresholds at which alerts are triggered and notifications are sent.

Icinga

Icinga is a tool that actively monitors the components of a given system. In the classic approach, where applications, databases and other components of a web application are hosted on servers, they can be monitored by an NRPE agent. The Icinga server sends them requests for specific sensors and metrics that check, for example, CPU load, available disk memory, length of queues, page loading time, or the number of database queries.

Depending on Icinga’s settings, you can define the tasks to be performed if a critical situation occurs, which can range from notifying the relevant people to performing an automated action. 

Grafana

Grafana is a much more visually appealing tool, though inferior in terms of notifications. 

It enables data visualization in the form of charts, indicators, and diagrams showing the key performance metrics of an application.

The data sources for Grafana include the databases ElasticSearchInfluxDB, SQL, or AWS CloudWatch

A well-constructed dashboard allows you to determine the current and historical status of the entire environment.

Graphs can be populated with relevant comments, such as running a newer version of the application, allowing you to easily find the relationship between the performance of the environment and the version of the application.

Grafana is also suitable for visualizing the metrics checked routinely, allowing a skillful administrator to predict failures – such as running out of memory or disk space – before they occur. 

It also allows you to define simple notification rules, although in a much poorer quality than Icinga.

Kibana 

Kibana is a dataset mining tool that enables the aggregation of logs from various sources. 

In addition to functions such as browsing and efficient search of logs, it allows you to create views with graphs; it also features a timeline module, allowing you to analyze trends or data over time.

In addition to creating graphs and assembling dashboards, you can construct a timeline query that looks for anomalies between data, for example, week-to-week.

The database for Kibana is usually the ElasticSearch index, which can also be used by the aforementioned Grafana. The two tools combined offer a great set for viewing and visualizing the collected metrics.

Prometheus 

Prometheus is a very good tool dedicated to modern environments: auto-scaled, dynamic, containerized, and with non-deterministic component lifetimes. 

Integration with a directory of services issued by service discovery (e.g. consul, eureka) can be effected in a relatively simple way. With Prometheus, it is also possible to collect data using the http(s) protocol with various exporters.

Prometheus integrates with Grafana, using it as a data visualization engine, and with alert manager as a module for aggregating alerts and sending notifications. 

CloudWatch

If our application is running in the AWS cloud, the CloudWatch service provided by Amazon seems like a natural monitoring solution to use. 

CloudWatch collects basic metrics from running services automatically and without any configuration, but in addition, we can define our own configurations. 

The AWS web panel provides us with integrated functions to view them, create charts, indicators, dashboards, although – if needed – the same data can be accessed via API. 

Alerts can also be configured and integrated with SNS notifications or Lambda functions. We can easily program events that integrate with other AWS services, such as scaling an ASG group based on a CloudWatch alert. 

Another feature of CloudWatch is the storage of application logs along with the option to view them using SQL queries (Athena) or Insights (GUI interface).

It is worth remembering that CloudWatch can serve as a visualization base for Grafana.

What should be monitored? 

Every project should monitor business-critical metrics, such as website loading speed or the handling time for customer requests.

The quick detection of defects in development or acceptance environments helps to avoid situations in which potentially ineffective code is applied in production environments. When designing monitoring systems, one should remember to imitate the normal behavior of Internet users as closely as possible – query applications using public addresses, verify DNS name recognition, and check the validity of certificates. 

Features available within specific applications:

  • Icinga – notifications, sadly without charts or log analysis.
  • Grafana – charts and notifications (limited scope).
  • Kibana – charts and log analysis, but no notifications.
  • Prometheus – charts (additionally you can link Grafana) and notifications (via alert manager module).
  • CloudWatch – the complete set from graphs to log analysis and notifications.

Summary

Well-organized monitoring allows you to not only react to failures but also to effectively predict and prevent them. If a failure does occur, we should be able to determine the cause of the failure using monitoring and define the actions to prevent the failure from occurring.

It is definitely worth taking the time to set up the comprehensive monitoring of environments not only for production but for all stages of software development.

Finally, the most important thing is the satisfaction of the application users, and without measuring its performance, it is hard to guarantee the quality of service. 

If you would like to learn more about monitoring services, feel free to take a look at our managed services offering.

Our Experts
/ Knowledge Shared

12.10.2021

A Single Source of Truth / A Key Way To Eliminate Technical Debt

E-Commerce

Repetition, inaccuracy and inconsistencies are issues every business wants to avoid. They’re also typical signs of poorly chosen – or poorly used – technologies. When used right, good technology can eliminate unnecessary repetition, ensure greater consistency and generally improve results over all. At the core of this is often the concept of “A...

05.10.2021

Personalization in E-Commerce / The Basics

E-Commerce

When talking face to face, personalization often happens at unconscious level. Interacts are smooth, adapted on the fly to suit the customer’s needs. Shop assistants can guide people to where they need to be and, when it comes to homing in on exactly what someone wants, it’s the fundamental secret to any good salesperson. But online? We must adapt....

28.09.2021

Black Friday / A Practical Checklist for E-Commerce Owners

E-Commerce

Everyone gets stressed on Black Friday but, between us, shoppers are the ones that have it easy 😉 Behind the scenes, entire IT, marketing and customer support teams need to prepare a solid strategy. And that’s not even mentioning warehouse, logistics or the subsequent challenge of order fulfilment… Black Friday (and Cyber Monday) are some of the...

Expert Knowledge
For Your Business

As you can see, we've gained a lot of knowledge over the years - and we love to share! Let's talk about how we can help you.

Contact us

<dialogue.opened>