What are the pros and cons of Azure native monitoring solution?

2021-07-05|By Incognito Redacted|Cloud

Azure

is a Microsoft cloud computing platform that provides everything a business might need to run a part, or all of its computing operations virtually - including servers, storage, databases, networking, analytics and more. Microsoft Azure competes with similar services from Amazon (AWS) and Google (GCP).


The Azure cloud platform has more than 200 products and cloud services, designed to help you bring new solutions to life, solve today's challenges and create the future. Having so many cloud based products and services, Microsoft had to design and implement common observability solution. That is why Azure Monitor

has been introduced.

Azure monitor overview

The basic overview of Azure Monitor.

The basic overview of Azure Monitor.

Azure Monitor is able to receive information from all resources within Azure and on-premise environments. This data helps you to understand how your applications are performing, proactively identify issues affecting them, and the resources they depend on. For example, you can:

  • Application - get level-defined data on performance and functionality of your own code.
  • Operating System - grab information related to yor Linux or Windows hosts, like syslogs or system application logs.
  • Azure Resources - collect data from all kind of Azure resources like Load Balancers, Web Applications, Azure Kubernetes, etc.
  • Azure Subscription - get data that relates to operation and management on the subscription level. It also includes data about Azure health and availability.
  • Azure Tenant - gather intel related to organization-level services, like Azure Active Directory.
  • Custom Sources - create custom monitoring scenarios, and extend monitoring to resources that don't expose telemetry through other sources using the Data Collector API

There are two main types of the observable data here:
  • Metrics - measure of a certain resource parameters over a given period of time. For instance, CPU utilization, disk IOPS, number of connections, requeset latency etc. Metrics are stored in a time-series database and are suitable for alerting and issue detection.
  • Logs - time-stamped information about the resource changes. In other word logs can be generated by the resources in a form of Event or by your application as an application log.

Pros of native monitoring solution

Detailed metrics and standardized logs

You can collect various types of metrics, to increase visual clarity of your computing environment operations, and use them to monitor current utilization, analyze root issue causes, cut costs... This feature is integrated into all Azure-related resources, and can be used with your on-premise infrastructure, also offering functionality of collecting Logs or Events.

Customizable and rich visualization tools

Azure monitor offers well-designed and easy to customize tools, that allows you to present the collected data on detailed charts and dashboards. For the metric data, you can set up charts, and group them into dedicated, functional dashboards:

Azure Monitor dashboard overview.

Azure Monitor dashboard overview.

You can create multiple dashboards for different purposes - for example, for different audience and its requirements, you can create a Workbooks

or a Power BI.


Logs has their own tool, Log Analytics. Data collected by Azure Monitor Logs is stored in one or more Log Analytics workspaces. The workspace defines the geographic location of the data, proper access permissions for users, and configuration settings such as the pricing tier or data retention.

Azure Monitor dashboard overview.

Azure Monitor dashboard overview.

Log queries are written in Kusto Query Language (KQL), the same query language that is used by the Azure Data Explorer. You can write log queries in Log Analytics to interactively analyze their results, use them in alert rules to be proactively notified of issues, or include their results in workbooks or dashboards.

Integration with internal Azure services

Azure monitor allows you to execute actions, based on the conditions calculated using the metrics. You can use alerts to implement notifications for defined action group, and trigger them when your metrics exceed the values. Notifications can be sent by many channels, for example:

  • E-mail - standard e-mail, to the users or groups.
  • SMS - direct SMS to the defined phone numbers.
  • Azure app Push Notification - when you use the dedicated Azure application on your Apple or Android-based phone
  • Voice call - you can even receive a direct phone call!

Azure Monitor also allows to define actions:

Azure Monitor, defining an action.

Azure Monitor, defining an action.

With actions, you can call different services, such as:

  • Azure Function - calls an existing HTTP trigger endpoint for the function
  • ITSM - allows you to connect Azure to a supported IT Service Management (ITSM) product or service.
  • Logic App - calls an existing HTTP trigger endpoint for the Logic App
  • Secure Webhook - secured(authenticated) webhook that can contain sensitive data
  • Webhook - unsecured simple HTTP trigger
  • Automation Runbook - definition of automated actions implemented within an Azure infrastructure

Ability to export monitoring data to external monitoring systems

For various reasons, you might need to use external tools for your monitoring solutions, and send the data to other tools in your environment too. The most effective method to stream monitoring information to external tools is to use Azure Event Hubs

. It enables you to easily integrate with external SIEM and monitoring tools. For example, the list of tools with Azure Monitor integration includes the following:


ToolIs hosted on Azure?Description
IBM QRadarNoThe Microsoft Azure DSM and Microsoft Azure Event Hub Protocol are available for download from the IBM support website. You can learn more about the integration with Azure at QRadar DSM configuration.
SplunkNoMicrosoft Azure Add-On for Splunk is an open source project available in Splunkbase. If you cannot install an add-on in your Splunk instance, if for example you're using a proxy or running on Splunk Cloud, you can forward these events to the Splunk HTTP Event Collector using Azure Function For Splunk, which is triggered by new messages in the event hub.
SumoLogicNoInstructions for setting up SumoLogic to consume data from an event hub are available at Collect Logs for the Azure Audit App from Event Hub.
ArcSightNoThe ArcSight Azure Event Hub smart connector is available as part of the ArcSight smart connector collection.
Syslog serverNoIf you want to stream Azure Monitor data directly to a syslog server, you can use a solution based on an Azure function.
LogRhythmNoInstructions to set up LogRhythm to collect logs from an event hub are available here.
Logz.ioYesFor more information, see Getting started with monitoring and logging using Logz.io for Java apps running on Azure

Other partners may also be available. Complete list of all Azure Monitor integrations can be found here

.

Cons of native monitoring solution

High costs

Considering Azure monitor as your main observability stack, you need to take potential costs into consideration. It is highly recommended to define which data are you going to collect, and how long do you want to keep it. Lack of proper planning and management can significantly increase the expenses of data transmission and storage. Azure tries to meet the expectations of its customers, providing information on best practices[[13]] for the optimal use of monitoring tools.

A lot od manual configuration

Complexity of the monitoring configuration depends on the size of your Azure infrastructure and components you are using. There is no standard guideline for all kind of resources and some of them, especially these used less commonly, require manual configuration. Not everything is available via the Azure portal and Terraform providers. It is very often that you need to use the PowerShell or Azure CLI directly, to achieve expected results.

Difficult implementation

Building the proper and effective monitoring solution is not a trivial task. You need to have detailed knowledge and experience with the monitoring aspects to understand which data do you need, how you are going to use it and how you can get it from Azure. You need to know, how to properly configure observability within your system. Azure is unable to provide a simple guideline suiting any requirements.

References

LinkedInLinkedInLinkedIn
Incognito Redacted photo

About the author

Incognito Redacted