The Azure cloud platform has more than 200 products and cloud services, designed to help you bring new solutions to life, solve today's challenges and create the future. Having so many cloud based products and services, Microsoft had to design and implement common observability solution. That is why Azure Monitor
Azure monitor overview
The basic overview of Azure Monitor.
Azure Monitor is able to receive information from all resources within Azure and on-premise environments. This data helps you to understand how your applications are performing, proactively identify issues affecting them, and the resources they depend on. For example, you can:
- Application - get level-defined data on performance and functionality of your own code.
- Operating System - grab information related to yor Linux or Windows hosts, like syslogs or system application logs.
- Azure Resources - collect data from all kind of Azure resources like Load Balancers, Web Applications, Azure Kubernetes, etc.
- Azure Subscription - get data that relates to operation and management on the subscription level. It also includes data about Azure health and availability.
- Azure Tenant - gather intel related to organization-level services, like Azure Active Directory.
- Custom Sources - create custom monitoring scenarios, and extend monitoring to resources that don't expose telemetry through other sources using the Data Collector API
There are two main types of the observable data here:
- Metrics - measure of a certain resource parameters over a given period of time. For instance, CPU utilization, disk IOPS, number of connections, requeset latency etc. Metrics are stored in a time-series database and are suitable for alerting and issue detection.
- Logs - time-stamped information about the resource changes. In other word logs can be generated by the resources in a form of Event or by your application as an application log.
Pros of native monitoring solution
Detailed metrics and standardized logs
You can collect various types of metrics, to increase visual clarity of your computing environment operations, and use them to monitor current utilization, analyze root issue causes, cut costs... This feature is integrated into all Azure-related resources, and can be used with your on-premise infrastructure, also offering functionality of collecting Logs or Events.
Customizable and rich visualization tools
Azure monitor offers well-designed and easy to customize tools, that allows you to present the collected data on detailed charts and dashboards. For the metric data, you can set up charts, and group them into dedicated, functional dashboards:
Azure Monitor dashboard overview.
You can create multiple dashboards for different purposes - for example, for different audience and its requirements, you can create a Workbooks
Logs has their own tool, Log Analytics. Data collected by Azure Monitor Logs is stored in one or more Log Analytics workspaces. The workspace defines the geographic location of the data, proper access permissions for users, and configuration settings such as the pricing tier or data retention.
Azure Monitor dashboard overview.
Log queries are written in Kusto Query Language (KQL), the same query language that is used by the Azure Data Explorer. You can write log queries in Log Analytics to interactively analyze their results, use them in alert rules to be proactively notified of issues, or include their results in workbooks or dashboards.
Integration with internal Azure services
Azure monitor allows you to execute actions, based on the conditions calculated using the metrics. You can use alerts to implement notifications for defined action group, and trigger them when your metrics exceed the values. Notifications can be sent by many channels, for example:
- E-mail - standard e-mail, to the users or groups.
- SMS - direct SMS to the defined phone numbers.
- Azure app Push Notification - when you use the dedicated Azure application on your Apple or Android-based phone
- Voice call - you can even receive a direct phone call!
Azure Monitor also allows to define actions:
Azure Monitor, defining an action.
With actions, you can call different services, such as:
- Azure Function
- Logic App
- Secure Webhook
- Automation Runbook
Ability to export monitoring data to external monitoring systems
For various reasons, you might need to use external tools for your monitoring solutions, and send the data to other tools in your environment too. The most effective method to stream monitoring information to external tools is to use Azure Event Hubs
|Tool||Is hosted on Azure?||Description|
|IBM QRadar||No||The Microsoft Azure DSM and Microsoft Azure Event Hub Protocol are available for download from the IBM support website. You can learn more about the integration with Azure at QRadar DSM configuration.|
|Splunk||No||Microsoft Azure Add-On for Splunk is an open source project available in Splunkbase. If you cannot install an add-on in your Splunk instance, if for example you're using a proxy or running on Splunk Cloud, you can forward these events to the Splunk HTTP Event Collector using Azure Function For Splunk, which is triggered by new messages in the event hub.|
|SumoLogic||No||Instructions for setting up SumoLogic to consume data from an event hub are available at Collect Logs for the Azure Audit App from Event Hub.|
|ArcSight||No||The ArcSight Azure Event Hub smart connector is available as part of the ArcSight smart connector collection.|
|Syslog server||No||If you want to stream Azure Monitor data directly to a syslog server, you can use a solution based on an Azure function.|
|LogRhythm||No||Instructions to set up LogRhythm to collect logs from an event hub are available here.|
|Logz.io||Yes||For more information, see Getting started with monitoring and logging using Logz.io for Java apps running on Azure|
Other partners may also be available. Complete list of all Azure Monitor integrations can be found here
Cons of native monitoring solution
Considering Azure monitor as your main observability stack, you need to take potential costs into consideration. It is highly recommended to define which data are you going to collect, and how long do you want to keep it. Lack of proper planning and management can significantly increase the expenses of data transmission and storage. Azure tries to meet the expectations of its customers, providing information on best practices[] for the optimal use of monitoring tools.
A lot od manual configuration
Complexity of the monitoring configuration depends on the size of your Azure infrastructure and components you are using. There is no standard guideline for all kind of resources and some of them, especially these used less commonly, require manual configuration. Not everything is available via the Azure portal and Terraform providers. It is very often that you need to use the PowerShell or Azure CLI directly, to achieve expected results.
Building the proper and effective monitoring solution is not a trivial task. You need to have detailed knowledge and experience with the monitoring aspects to understand which data do you need, how you are going to use it and how you can get it from Azure. You need to know, how to properly configure observability within your system. Azure is unable to provide a simple guideline suiting any requirements.
- What is Azure?. (n. d.). https://azure.microsoft.com/en-us/overview/what-is-azure/ (accessed July 5, 2021).
- Azure Monitor. (n. d.). https://docs.microsoft.com/en-us/azure/azure-monitor/overview (accessed July 5, 2021).
- Azure Workbooks. (n. d.). https://docs.microsoft.com/en-us/azure/azure-monitor/visualize/workbooks-overview (accessed July 5, 2021).
- Microsoft PowerBI. (n. d.). https://powerbi.microsoft.com/ (accessed July 5, 2021).
- Azure Functions. (n. d.). https://docs.microsoft.com/en-us/azure/azure-functions/functions-get-started (accessed July 5, 2021).
- Azure ITSM Connector. (n. d.). https://docs.microsoft.com/en-us/azure/azure-monitor/alerts/itsmc-overview (accessed July 5, 2021).
- Azure Logic Apps. (n. d.). https://docs.microsoft.com/en-us/azure/logic-apps (accessed July 5, 2021).
- Azure Secure Webhook Alert. (n. d.). https://docs.microsoft.com/en-us/azure/azure-monitor/alerts/action-groups#secure-webhook (accessed July 5, 2021).
- Azure Simple Webhook. (n. d.). https://docs.microsoft.com/en-us/azure/azure-monitor/alerts/action-groups#webhook (accessed July 5, 2021).
- Azure Automation Runbook. (n. d.). https://docs.microsoft.com/en-us/azure/automation/automation-runbook-execution (accessed July 5, 2021).
- Azure Event Hubs. (n. d.). https://docs.microsoft.com/en-gb/azure/event-hubs/ (accessed July 5, 2021).
- Azure Monitor Integration Partners. (n. d.). https://docs.microsoft.com/en-gb/azure/azure-monitor/partners (accessed July 5, 2021).