Monitoring Web Applications in Dynamically Growing Environments

The reliable operation of applications, and consequently – customer satisfaction – is the key element in operating a business efficiently. Monitoring processes that can quickly detect a failure, find hidden dependencies and even predict a failure are essential tools in such dynamically growing environments.


You will learn from this article:

  • What is application monitoring
  • What tools are used to monitor environments

A Quick Introduction to Application Monitoring

The idea of monitoring can contain many different functionalities:

  • sending notifications when a failure occurs 
  • visualizing data collected from various sources 
  • data mining 
  • establishing relationships between certain facts and their consequences 
  • predicting failures 

Below, we will discuss the most important application monitoring tools, such as IcingaGrafanaKibanaPrometheus, and AWS CloudWatch.

Overview of application monitoring tools 

Almost every monitoring system is based on metrics and the thresholds at which alerts are triggered and notifications are sent.

Icinga

Icinga is a tool that actively monitors the components of a given system. In the classic approach, where applications, databases and other components of a web application are hosted on servers, they can be monitored by an NRPE agent. The Icinga server sends them requests for specific sensors and metrics that check, for example, CPU load, available disk memory, length of queues, page loading time, or the number of database queries.

Depending on Icinga’s settings, you can define the tasks to be performed if a critical situation occurs, which can range from notifying the relevant people to performing an automated action. 

Grafana

Grafana is a much more visually appealing tool, though inferior in terms of notifications. 

It enables data visualization in the form of charts, indicators, and diagrams showing the key performance metrics of an application.

The data sources for Grafana include the databases ElasticSearchInfluxDB, SQL, or AWS CloudWatch

A well-constructed dashboard allows you to determine the current and historical status of the entire environment.

Graphs can be populated with relevant comments, such as running a newer version of the application, allowing you to easily find the relationship between the performance of the environment and the version of the application.

Grafana is also suitable for visualizing the metrics checked routinely, allowing a skillful administrator to predict failures – such as running out of memory or disk space – before they occur. 

It also allows you to define simple notification rules, although in a much poorer quality than Icinga.

Kibana 

Kibana is a dataset mining tool that enables the aggregation of logs from various sources. 

In addition to functions such as browsing and efficient search of logs, it allows you to create views with graphs; it also features a timeline module, allowing you to analyze trends or data over time.

In addition to creating graphs and assembling dashboards, you can construct a timeline query that looks for anomalies between data, for example, week-to-week.

The database for Kibana is usually the ElasticSearch index, which can also be used by the aforementioned Grafana. The two tools combined offer a great set for viewing and visualizing the collected metrics.

Prometheus 

Prometheus is a very good tool dedicated to modern environments: auto-scaled, dynamic, containerized, and with non-deterministic component lifetimes. 

Integration with a directory of services issued by service discovery (e.g. consul, eureka) can be effected in a relatively simple way. With Prometheus, it is also possible to collect data using the http(s) protocol with various exporters.

Prometheus integrates with Grafana, using it as a data visualization engine, and with alert manager as a module for aggregating alerts and sending notifications. 

CloudWatch

If our application is running in the AWS cloud, the CloudWatch service provided by Amazon seems like a natural monitoring solution to use. 

CloudWatch collects basic metrics from running services automatically and without any configuration, but in addition, we can define our own configurations. 

The AWS web panel provides us with integrated functions to view them, create charts, indicators, dashboards, although – if needed – the same data can be accessed via API. 

Alerts can also be configured and integrated with SNS notifications or Lambda functions. We can easily program events that integrate with other AWS services, such as scaling an ASG group based on a CloudWatch alert. 

Another feature of CloudWatch is the storage of application logs along with the option to view them using SQL queries (Athena) or Insights (GUI interface).

It is worth remembering that CloudWatch can serve as a visualization base for Grafana.

What should be monitored? 

Every project should monitor business-critical metrics, such as website loading speed or the handling time for customer requests.

The quick detection of defects in development or acceptance environments helps to avoid situations in which potentially ineffective code is applied in production environments. When designing monitoring systems, one should remember to imitate the normal behavior of Internet users as closely as possible – query applications using public addresses, verify DNS name recognition, and check the validity of certificates. 

Features available within specific applications:

  • Icinga – notifications, sadly without charts or log analysis.
  • Grafana – charts and notifications (limited scope).
  • Kibana – charts and log analysis, but no notifications.
  • Prometheus – charts (additionally you can link Grafana) and notifications (via alert manager module).
  • CloudWatch – the complete set from graphs to log analysis and notifications.

Summary

Well-organized monitoring allows you to not only react to failures but also to effectively predict and prevent them. If a failure does occur, we should be able to determine the cause of the failure using monitoring and define the actions to prevent the failure from occurring.

It is definitely worth taking the time to set up the comprehensive monitoring of environments not only for production but for all stages of software development.

Finally, the most important thing is the satisfaction of the application users, and without measuring its performance, it is hard to guarantee the quality of service. 

If you would like to learn more about monitoring services, feel free to take a look at our managed services offering.

Our Experts
/ Knowledge Shared

E-Commerce Development in 2023 – Trends and Expert Tips
17.11.2022

E-Commerce Development in 2023 / Trends and Expert Tips

E-Commerce

Due to the anticipated crisis, 2023 will be a tough year, which will undoubtedly affect the operations of many companies. After the difficult two-year pandemic, which disorganized or even crushed many businesses around the world, we will probably face such challenges as rampant inflation, energy problems and disrupted supply chains. The economic and...

Inventory Management and Forecasting for supermarkets
10.11.2022

How to Better Forecast Demand in a Supermarket. Meet Inventory Optimization System.

Data Analytics & Machine Learning

Modern inventory optimization system automatically and accurately predicts supermarket demand. Find out...

07.11.2022

MACH Architecture vs Digital Experience Platform (DXP) / Why MACH Technology Dominates

General

We have already presented our observations on the imperfections of the “monolithic” Digital Experience Platforms (DXPs) and the MACH Architecture’s business and technological advantage over DXPs. Below we have outlined the most important pros and cons of both DXPs and the MACH architecture to help you answer the question about what to choose, and to...

Expert Knowledge
For Your Business

As you can see, we've gained a lot of knowledge over the years - and we love to share! Let's talk about how we can help you.

Contact us

<dialogue.opened>