Publish Application Logs to Azure Monitor Logs

This guide describes how to use the Graal Development Kit for Micronaut (GDK) to create an application that publishes logs to Azure Monitor Logs. Typically, application logs are written to stdout and to files, but the Micronaut^® Logging module supports multiple logging frameworks such as Logback, and logging to various other appenders, to email, a database, or other destinations.

Prerequisites #

• JDK 17 or higher. See Setting up Your Desktop.
• An Azure account. See Setting up Your Cloud Accounts.
• The Azure CLI. Follow the Azure documentation for installing or updating the latest version of the Azure CLI.
• An Azure user with sufficient permissions to configure Azure Monitor Logs.
• The GDK CLI. See Setting up Your Desktop. (Optional.)

Follow the steps below to create the application from scratch. However, you can also download the completed example:

A note regarding your development environment

Consider using Visual Studio Code, which provides native support for developing applications with the Graal Development Kit extension.

If you use IntelliJ IDEA, enable annotation processing.

Windows platform: The GDK guides are compatible with Gradle only. Maven support is coming soon.

1. Create the Application #

Create an application using the GDK Launcher.

1. Open the GDK Launcher in advanced mode.

2. Create a new project using the following selections.
   • Project Type: Application (Default)
   • Project Name: azure-logging-demo
   • Base Package: com.example (Default)
   • Clouds: Azure
   • Build Tool: Gradle (Groovy) or Maven
   • Language: Java (Default)
   • Test Framework: JUnit (Default)
   • Java Version: 17 (Default)
   • Micronaut Version: (Default)
   • Cloud Services: Logging
   • Features: GraalVM Native Image (Default)
   • Sample Code: Yes (Default)
3. Click Generate Project, then click Download Zip. The GDK Launcher creates an application with the default package com.example in a directory named azure-logging-demo. The application ZIP file will be downloaded to your default downloads directory. Unzip it, open it in your code editor, and proceed to the next steps.

Alternatively, use the GDK CLI as follows:

gdk create-app com.example.azure-logging-demo \
    --clouds=azure \
    --services=logging \
    --features=graalvm \
    --build=gradle \
    --jdk=17 \
    --lang=java

gdk create-app com.example.azure-logging-demo \
    --clouds=azure \
    --services=logging \
    --features=graalvm \
    --build=maven \
    --jdk=17 \
    --lang=java

The GDK Launcher creates a multi-module project with two subprojects: azure for Microsoft Azure, and lib. You develop the application logic in the azure subproject. If your application is to be deployed to multiple cloud providers, use the lib subproject to create classes that can be shared between the providers. This enables you to separate the code that is different between cloud providers, while keeping most of the implementation in the common lib subproject.

The Micronaut Logging service that you selected at the project generation step bundles Logback, Jackson Databind, Azure Logging, and other necessary dependencies. The Logback appender publishes logs to Azure Monitor Logs.

1.1. Controller Class #

The example code includes a controller in a file named azure/src/main/java/com/example/LogController.java, which enables you to send POST requests to publish a message to a log:

package com.example;

import io.micronaut.http.annotation.Body;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Post;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Controller
class LogController {

    private static final Logger LOG = LoggerFactory.getLogger(LogController.class);

    @Post("/log")
    void log(@Body String message) {
        LOG.info(message);
    }
}

2. Create Azure Cloud Resources #

You will create a resource group, a Log Analytics Workspace and a table in the workspace to hold log event data, a Data Collection Endpoint, and a Data Collection Rule.

2.1. Create a Resource Group #

We recommend that you create a new resource group for this guide, but you can use an existing resource group instead.

Run the az group create command to create a resource group named gdkguides in the eastus region:

az group create --location eastus --name gdkguides

If you prefer using the region geographically closer to you, run az account list-locations to list all available regions.

2.2. Add the monitor-control-service CLI Extension #

Run the az extension add command to add the monitor-control-service CLI Extension:

az extension add --name monitor-control-service

2.3. Add the log-analytics CLI Extension #

Run the az extension add command to add the log-analytics CLI Extension:

az extension add --name log-analytics

2.4. Create a Log Analytics Workspace #

Run the az monitor log-analytics workspace create command to create a Log Analytics Workspace:

az monitor log-analytics workspace create \
   --name gdkworkspace \
   --resource-group gdkguides

The response should look like this:

{
  "createdDate": "2024-08-19T20:11:18.4002398Z",
  "customerId": "c222d080-5b14-43e7-b648-71c2b358dc74",
  ...
  "name": "gdkworkspace",
  "provisioningState": "Creating",
   ...
}

Save the value of the customerId attribute. This is your workspace GUID, and it will be needed later.

2.5. Create a Data Collection Endpoint #

Run the az monitor data-collection endpoint create command to create a Data Collection Endpoint:

az monitor data-collection endpoint create \
   --data-collection-endpoint-name gdkCollectionEndpoint \
   --public-network-access Enabled \
   --resource-group gdkguides

The response should look like this:

{
  ...
  "logsIngestion": {
    "endpoint": "https://gdkcollectionendpoint-xxxx.eastus-1.ingest.monitor.azure.com"
  },
  ...
  "name": "gdkCollectionEndpoint",
  ...
  "type": "Microsoft.Insights/dataCollectionEndpoints"
}

Save the value of the logsIngestion.endpoint attribute. This is the logs ingestion endpoint URL, and it will be needed later.

2.6. Create a Table in Your Log Analytics Workspace #

Run the az monitor log-analytics workspace table create command to create a table in your Log Analytics workspace to hold the log records:

az monitor log-analytics workspace table create \
   --name GDKTable_CL \
   --workspace-name gdkworkspace \
   --columns EventTimestamp=long Source=string Subject=string Data=string TimeGenerated=datetime \
   --resource-group gdkguides

2.7. Create a Data Collection Rule #

Create a file named dcr.json with this content:

{
  "location": "<region>",
  "properties": {
    "streamDeclarations": {
      "Custom-GDKTable": {
        "columns": [
          {
            "name": "TimeGenerated",
            "type": "datetime"
          },
          {
            "name": "EventTimestamp",
            "type": "long"
          },
          {
            "name": "Source",
            "type": "string"
          },
          {
            "name": "Subject",
            "type": "string"
          },
          {
            "name": "Data",
            "type": "string"
          }
        ]
      }
    },
    "destinations": {
      "logAnalytics": [
        {
          "workspaceResourceId": "/subscriptions/<subscription-id>/resourceGroups/gdkguides/providers/microsoft.operationalinsights/workspaces/gdkworkspace",
          "name": "gdkLogDestination"
        }
      ]
    },
    "dataFlows": [
      {
        "streams": [
          "Custom-GDKTable"
        ],
        "destinations": [
          "gdkLogDestination"
        ],
        "transformKql": "source | extend TimeGenerated = now()",
        "outputStream": "Custom-GDKTable_CL"
      }
    ]
  }
}

Replace <region> with the name of the region you are using, for example, “eastus”, and replace <subscription-id> with your Azure Subscription ID.

Run the az monitor data-collection rule create command to create a data collection rule:

az monitor data-collection rule create \
   --name gdkCollectionRule \
   --location <region> \
   --endpoint-id /subscriptions/<subscription-id>/resourceGroups/gdkguides/providers/Microsoft.Insights/dataCollectionEndpoints/gdkCollectionEndpoint \
   --rule-file "path/to/dcr.json" \
   --resource-group gdkguides

Replace <region> with the name of the region you are using, for example, “eastus”, replace <subscription-id> with your Azure Subscription ID, and replace “path/to/dcr.json” with the file location of the file you created.

The response should look like this:

{
  "dataCollectionEndpointId": "/subscriptions/fe053...",
  "dataFlows": [
    {
      "destinations": [
        "gdkLogDestination"
      ],
      "outputStream": "Custom-GDKTable_CL",
      "streams": [
        "Custom-GDKTable"
      ],
      "transformKql": "source | extend TimeGenerated = now()"
    }
  ],
  ...
  "immutableId": "dcr-e7ebfceb7df24631b64d7ae880eb8ada",
  "location": "eastus",
  "name": "gdkCollectionRule",
  ...
  "type": "Microsoft.Insights/dataCollectionRules"
}

Save the value of the immutableId attribute. This is the rule ID, and it will be needed later.

2.8. Authorize Sending Logs #

Authorize sending logs by assigning the Monitoring Metrics Publisher role to yourself:

az role assignment create \
   --role "Monitoring Metrics Publisher" \
   --assignee <email> \
   --scope "/subscriptions/<subscription-id>/resourceGroups/gdkguides/providers/Microsoft.Insights/dataCollectionRules/gdkCollectionRule"

replacing <email> with the email address associated with your account, and <subscription-id> with your Azure Subscription ID.

Note that it can take a few minutes for the role grant to propagate.

3. Configure Appender and Application Configuration #

3.1. Logback Appender #

The GDK Launcher generated a file named azure/src/main/resources/logback.xml containing the configuration for an appender that publishes log events to Azure Monitor Logs:

<configuration debug='false'>

    <!--
    You can un-comment the STDOUT appender and <appender-ref ref='STDOUT'/> in
    the cloud appender to log to STDOUT as the 'emergency' appender.
    -->

    <!--
    <appender name='STDOUT' class='ch.qos.logback.core.ConsoleAppender'>
        <encoder>
            <pattern>%cyan(%d{HH:mm:ss.SSS}) %gray([%thread]) %highlight(%-5level) %magenta(%logger{36}) - %msg%n</pattern>
        </encoder>
    </appender>
    -->

    <appender name='AZURE' class='io.micronaut.azure.logging.AzureAppender'>
        <!-- <appender-ref ref='STDOUT'/> -->
        <encoder class='ch.qos.logback.core.encoder.LayoutWrappingEncoder'>
            <layout class='ch.qos.logback.contrib.json.classic.JsonLayout'>
                <jsonFormatter class='io.micronaut.azure.logging.AzureJsonFormatter'/>
            </layout>
        </encoder>
    </appender>

    <root level='INFO'>
        <appender-ref ref='AZURE'/>
    </root>

</configuration>

Note: You can un-comment the STDOUT appender as the ‘emergency’ appender. See the file for details.

3.2. Set Application Configuration Properties #

Update the Azure application.properties as follows:

azure/src/main/resources/application.properties

# <1>
azure.logging.data-collection-endpoint=<endpoint>
# <2>
azure.logging.rule-id=<ruleid>
# <3>
azure.logging.stream-name=Custom-GDKTable

1 Replace <endpoint> with the logs ingestion endpoint URL that you saved earlier

2 Replace <ruleid> with the rule ID that you saved earlier

3 Set the value of the azure.logging.stream-name property with the name of the workspace table you created, “Custom-GDKTable”

Having configured the appender and the application configuration, you can proceed to run the application, publishing the logs.

4. Run the Application, Publish and View Logs #

1. Run the application using the following command (it will start the application on port 8080):

   • Gradle
   • Maven

   ./gradlew :azure:run

   ./mvnw install -pl lib -am
   ./mvnw mn:run -pl azure

2. Send some curl requests to test logging:

    curl -id '{"message":"your message here"}' \
         -H "Content-Type: application/json" \
         -X POST http://localhost:8080/log
   

3. Run the az monitor log-analytics query command to retrieve log events that were pushed while running your application:

    az monitor log-analytics query \
       --workspace <workspace-guid> \
       --analytics-query "GDKTable_CL | where TimeGenerated > ago(1h)"
   

Replace <workspace-guid> with the workspace GUID that you saved earlier.

Note that it can take a few minutes for log entries be available.

5. Clean up Cloud Resources #

Once you are done with this guide, you can delete the Azure resources created to avoid incurring unnecessary charges.

Delete the resource group and all of its resources with:

az group delete --name gdkguides

Alternatively, run these commands to delete resources individually:

az monitor data-collection rule delete --name gdkCollectionRule --resource-group gdkguides

az monitor log-analytics workspace table delete --name GDKTable_CL --workspace-name gdkworkspace --resource-group gdkguides

az monitor data-collection endpoint delete --name gdkCollectionEndpoint --resource-group gdkguides

az monitor log-analytics workspace delete --workspace-name gdkworkspace --resource-group gdkguides --force

az group delete --name gdkguides

Summary #

This guide demonstrated how to use the GDK to create an application that publishes logs to Azure Monitor Logs. Then, using Azure Monitor Logs, you viewed the logs produced by the application. Finally, you saw how to build a native executable for this application with GraalVM Native Image, and ran it to test publishing logs to Azure Monitor Logs.

Related Documentation #

• Micronaut Azure Logging
• Azure Monitor Logs
• GraalVM Native Image