The Pillars of Observability
After completing the Game Store application, the last week was all about scripting the first few modules of the upcoming .NET Cloud Developer Bootcamp, which essentially means creating a detailed Word document for each lesson, describing exactly how that lesson should go.
I don't know how many content creators do this, since it's a long (and sometimes tedious) process, but I find it essential to make sure each concept and technique is introduced at exactly the right time and to significantly reduce the chance of errors during recording.
As usual, the first few modules are the hardest to prepare, since those are the ones where I'm forced to put myself in the shoes of an absolute beginner so that I can get across the most fundamental concepts in the most didactic way possible.
Before going back to scripting, let me tell you how the Game Store application deals with something every cloud-ready application must enable: Observability.
The pillars of observability
The 3 pillars of observability are what you use to monitor and understand what's happening in your system, from knowing if the app is alive to understanding its overall performance.
The 3 pillars are:
- Logs. Logs capture events and messages about what’s happening in your code, from error messages to successful actions. They help you trace issues and understand specific actions at a particular moment.
- Metrics. The numbers that give you an overview of your system's health. Things like CPU usage, memory consumption, or the number of requests per second. They help you spot problems early before they become critical.
- Traces. These are like a roadmap that follows a single request or transaction as it travels through different parts of your system. They help you see where slowdowns or failures happen in complex, distributed systems.
So, what you want to do is collect logs, metrics and traces for all the services involved in your system, from the frontend, to the API gateway, and across all your microservices. It's the only way to understand what's going on when things don't go as expected, especially in cloud-based distributed apps.
With all that data at hand, you want to publish it to a place where you can visualize and analyze them. There are several options for this, including the popular Prometheus, but in our case, we'll go for the native Azure service that's designed for this: Application Insights.
But first, how do we collect all that data in our services?
Using OpenTelemetry
OpenTelemetry is like a toolkit for your .NET services that helps you easily collect and send observability data (logs, metrics, and traces) to the platform or tool of your choice.
And, if you are using .NET Aspire, you are already using OpenTelemetry. Just open up your Extensions.cs file in your ServiceDefaults project and you'll find this:
AddServiceDefaults is the method all your Aspire-enabled apps should be calling, and that one, in turn, calls the ConfigureOpenTelemetry method, which will eventually call this:
There are several ways you can further customize what data your services should collect, but the AddOpenTelemetry().UseAzureMonitor() call is all you need to get started.
In the Game Store application, all the microservices reference ServiceDefaults as a NuGet package and make this call from Program.cs:
The next piece to figure out is how to get that Application Insights resource deployed and ready for our apps to connect to it.
Configuring Application Insights
Application Insights is nothing more than one more Azure resource, and since I already got another 7 Azure services configured via Bicep, all I did was prepare this new Bicep file based on the docs:
Notice that, just like we did with all other Azure resources, the App Insights connection string is conveniently placed in a KeyVault secret with a format easy to understand by any .NET app.
And, to plug that into your Aspire application model, all you do is this:
I will further improve this so that you can just use a managed identity here as opposed to a connection string, and I also heard .NET Aspire 9 has massive improvements in its Azure C# API support (so we don't have to switch to Bicep), but for now, this works great.
With that in place, and after redeploying the microservices with my latest ServiceDefaults NuGet package, it's time to see all that telemetry in action.
Analyzing the telemetry
After playing with the app for a few minutes in the cloud, we can already see the metrics pop up in the Azure Portal:
We can see traces that give us an end-to-end view of important processes like a purchase order:
And we can dive into a filterable sea of logs that keep track of every single exception and warning emitted by our services:
Notice that I'm querying here all the logs for a specific operation_Id. That ID lets you correlate everything that happened across the 7 microservices + API Gateway + Frontend, to complete the processing of an order. Super useful!
Wrapping up
A few of you have been asking me for the release date of this bootcamp. Although I don't have a concrete date yet, the plan is to have the first part ready by sometime around Thanksgiving.
I'm sorry it's taking so long but I really didn't want to start recording anything until I knew exactly how the finished application would work, in the cloud, with CI/CD pipelines and full observability.
The best way to know when the release date and any other bootcamp news are announced is to join the waitlist.
The scripts I'm working on for the first part of the bootcamp should be done by this time next week. After that, I'll spend a few days working on the slide decks and then we'll start recording. So we are getting there!
Now, back to work!
Julio
Whenever you’re ready, there are 4 ways I can help you:
- .NET Cloud Developer Bootcamp: Everything you need to go from zero to building real-world .NET cloud applications, step by step.
- All-Access Pass: A complete catalog of premium courses, with continuous access to new training and updates.
- Patreon Community: Join for exclusive discounts on all my in-depth courses and access my Discord server for community support and discussions.
- Promote yourself to 17,000+ subscribers by sponsoring this newsletter.