
<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240420), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240420)

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

</Sponsorships>

If you're building or maintaining distributed .NET applications, understanding how they behave is key to ensuring reliability and performance.

Distributed systems offer flexibility but introduce complexity, making troubleshooting a headache.
Understanding how requests flow through your system is crucial for debugging and performance optimization.

OpenTelemetry is an open-source observability framework that makes this possible.

In this article, we'll dive into what OpenTelemetry is, how to use it in your .NET projects, and the powerful insights it provides.

## OpenTelemetry Introduction

[OpenTelemetry](https://opentelemetry.io/) (OTel) is a vendor-neutral, open-source standard for instrumenting applications to generate telemetry data.
OpenTelemetry contains APIs, SDKs, tools, and integrations for creating and managing this telemetry data (traces, metrics, and logs).

Telemetry data includes:

- **Traces**: Represent the flow of requests through distributed systems, showing timings and relationships between services.
- **Metrics**: Numerical measurements of system behavior over time (e.g., request counts, error rates, memory usage).
- **Logs**: Textual records of events with rich contextual information. Structured logs.

<figure>
  <Image
    src="blogs/mnw_086/otel.png"
    width={1500}
    height={996}
    alt="OpenTelemetry Reference Architecture."
  />
  <figcaption>
    Source: [https://opentelemetry.io/docs/](https://opentelemetry.io/docs/)
  </figcaption>
</figure>

OpenTelemetry provides a unified way to collect this data, making it easier to understand the behavior and health of complex distributed applications.

We can export the telemetry data we are collecting to a service capable of processing it and providing us with an interface to analyze it.

We're going to configure OpenTelemetry to export traces directly to [Jaeger](https://www.jaegertracing.io/).

## Adding OpenTelemetry to .NET Applications

OpenTelemetry provides libraries and SDKs to add code (instrumentation) into your .NET applications.
These instrumentations automatically capture the traces, metrics, and logs we are interested in.

We're going to install the following NuGet packages:

```powershell
# Automatic tracing, metrics
Install-Package OpenTelemetry.Extensions.Hosting

# Telemetry data exporter
Install-Package OpenTelemetry.Exporter.OpenTelemetryProtocol

# Instrumentation packages
Install-Package OpenTelemetry.Instrumentation.Http
Install-Package OpenTelemetry.Instrumentation.AspNetCore
Install-Package OpenTelemetry.Instrumentation.EntityFrameworkCore
Install-Package OpenTelemetry.Instrumentation.StackExchangeRedis
Install-Package Npgsql.OpenTelemetry
```

Once we have these NuGet packages installed, it's time to configure some services.

```csharp
services
    .AddOpenTelemetry()
    .ConfigureResource(resource => resource.AddService(serviceName))
    .WithTracing(tracing =>
    {
        tracing
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation()
            .AddEntityFrameworkCoreInstrumentation()
            .AddRedisInstrumentation()
            .AddNpgsql();

        tracing.AddOtlpExporter();
    });
```

- `AddAspNetCoreInstrumentation` - This enables ASP.NET Core instrumentation.
- `AddHttpClientInstrumentation` - This enables `HttpClient` instrumentation for outgoing requests.
- `AddEntityFrameworkCoreInstrumentation` - This enables EF Core instrumentation.
- `AddRedisInstrumentation` - This enables Redis instrumentation.
- `AddNpgsql` - This enables PostgreSQL instrumentation.

With all of these instrumentations configured, our application will start collecting a lot of valuable traces at runtime.

We also need to configure an environment variable for the exporter added with `AddOtlpExporter` to work correctly.
We can set `OTEL_EXPORTER_OTLP_ENDPOINT` through application settings.
The address specified here will point to a local Jaeger instance.

```
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
```

## Running Jaeger Locally

[Jaeger](https://www.jaegertracing.io/) is an open source, distributed tracing platform.
Jaeger maps the flow of requests and data as they travel through a distributed system.
These requests could be calling out to multiple services, and Jaeger knows how to piece all of this information together.

Here's how to run Jaeger inside a Docker container:

```
docker run -d -p 4317:4317 -p 16686:16686 jaegertracing/all-in-one:latest
```

We're using the `jaegertracing/all-in-one:latest` image, and exposing the `4317` to accept telemetry data.
The Jaeger user interface will be exposed on the `16686` port.

## Distributed Tracing

After installing the OpenTelemetry libraries and configuring tracing in our applications, we can send some requests to generate telemetry data.
We can then access Jaeger to start analyzing our distributed traces.

**Registering a new user**

Here's an example of registering a new user with the system.
We're accessing the API gateway (`Evently.Gateway`) service, which proxies the request to the `Evently.Api` service.
And you can see that the `Evently.Api` service makes a few HTTP requests before persisting a new record in the database.

<Image
  src="blogs/mnw_086/trace_1.png"
  width={1920}
  height={650}
  alt="Distributed trace."
/>

**Publishing a message with MassTransit**

Here's another distributed trace where we publish the `UserRegisteredIntegrationEvent` over a message bus.
You can see that it's being consumed by two different services that write some data to the database.

<Image
  src="blogs/mnw_086/trace_2.png"
  width={1920}
  height={596}
  alt="Distributed trace."
/>

**Examining additional trace information**

Distributed traces can include some useful contextual information.
Here's an example trace representing a database command.
This comes from the PostgreSQL instrumentation, and we can see the SQL query that we are executing.

<Image
  src="blogs/mnw_086/trace_3.png"
  width={1920}
  height={500}
  alt="Distributed trace."
/>

**Complex distributed traces**

Here's a more complex distributed trace, which includes:

- Three .NET applications
- PostgreSQL database
- Redis distributed cache

We're sending a request to get the customer's cart.
The request will first hit the API gateway, which proxies it to the `Evently.Ticketing.Api` service that owns the data.
However, the `Evently.Ticketing.Api` service needs to reach out to the `Evently.Api` service to get the authorization information.
And all of this leads to the distributed trace you can see below.

<Image
  src="blogs/mnw_086/trace_4.png"
  width={1920}
  height={850}
  alt="Distributed trace."
/>

## Summary

Understanding modern applications, especially distributed ones, can be a real mind-bender.
OpenTelemetry is like having X-ray vision into your system.

While adding OpenTelemetry takes some upfront work, consider it an investment.
That investment pays off big time when problems pop up.
Instead of frantic guesswork, you have precise data to zero in on issues fast.

Is OpenTelemetry a magic bullet for all your problems? Nope.

But it's an excellent tool to add to your troubleshooting arsenal, especially as your .NET applications grow and get more complex.

If you're curious where the distributed traces come from, it's from the application we're building in my [**Modular Monolith course**](/modular-monolith-architecture).

That's all for today. Stay awesome, and I'll see you next week.

---



<Sponsorships>

[**Xamarin Days Are Numbered**](https://www.telerik.com/blogs/xamarin-days-numbered-ready-migrate-net-maui?utm_medium=cpm&utm_source=milanjovanovich&utm_campaign=end-of-xamarin-support) -
Are You Ready to Migrate to .NET MAUI?
Microsoft's decision to halt support for Xamarin on May 1, 2024, is rooted in the evolution of technology.
What's next?
[**Find out here.**](https://www.telerik.com/blogs/xamarin-days-numbered-ready-migrate-net-maui?utm_medium=cpm&utm_source=milanjovanovich&utm_campaign=end-of-xamarin-support)

Become a Postman master with [**Postman Intergalactic sessions.**](https://www.postman.com/events/intergalactic)
Test your APIs with ease and collaborate on API development with workspaces.
[**Get Postman for FREE!**](https://www.postman.com/downloads)

</Sponsorships>

Sometimes, especially in high-traffic scenarios, you absolutely need to ensure that only one process can modify a piece of data at a time.

Imagine you're building the ticket sales system for a wildly popular concert.
Customers are eagerly grabbing tickets, and the last few could sell out simultaneously.
If you're not careful, multiple customers might think they've secured the final seat, leading to overbooking and disappointment!

Entity Framework Core is a fantastic tool, but it doesn't have a direct mechanism for pessimistic locking.
[Optimistic locking](solving-race-conditions-with-ef-core-optimistic-locking) (using versions) can work, but in high-contention scenarios, it can lead to a lot of retries.

So, how can we solve this problem with EF Core?

## The Scenario in More Detail

Here's a simplified code snippet to illustrate our ticketing challenge:

```csharp
public async Task Handle(CreateOrderCommand request)
{
    await using DbTransaction transaction = await unitOfWork
        .BeginTransactionAsync();

    Customer customer = await customerRepository.GetAsync(request.CustomerId);

    Order order = Order.Create(customer);
    Cart cart = await cartService.GetAsync(customer.Id);

    foreach (CartItem cartItem in cart.Items)
    {
        // Uh-oh... what if two requests hit this at the same time?
        TicketType ticketType = await ticketTypeRepository.GetAsync(
            cartItem.TicketTypeId);

        ticketType.UpdateQuantity(cartItem.Quantity);

        order.AddItem(ticketType, cartItem.Quantity, cartItem.Price);
    }

    orderRepository.Insert(order);

    await unitOfWork.SaveChangesAsync();

    await transaction.CommitAsync();

    await cartService.ClearAsync(customer.Id);
}
```

The example above is contrived, but it should be enough to explain the problem.
During checkout, we verify the `AvailableQuantity` for each ticket.

What will happen if we get concurrent requests trying to purchase the same ticket?

The worst-case scenario is we end up "overselling" the tickets.
Concurrent requests could see available tickets for sale and complete the checkout.

So, how do we solve this?

## Raw SQL to the Rescue!

Since EF Core doesn't offer pessimistic locking directly, we'll dip into a bit of good old-fashioned SQL.
We will replace the `GetAsync` call to fetch the ticket with `GetWithLockAsync`.

Thankfully, EF Core makes this easy with [raw SQL queries](ef-core-raw-sql-queries):

```csharp {10}
public async Task<TicketType> GetWithLockAsync(Guid id)
{
    return await context
        .TicketTypes
        .FromSql(
            $@"
            SELECT id, event_id, name, price, currency, quantity
            FROM ticketing.ticket_types
            WHERE id = {id}
            FOR UPDATE NOWAIT") // PostgreSQL: Lock or fail immediately
        .SingleAsync();
}
```

Understanding the magic:

- `FOR UPDATE NOWAIT`: This is the heart of pessimistic locking in PostgreSQL.
  It tells the database "Grab this row, lock it for me, and if it's already locked, raise an error right now."
- **Error Handling**: We'd wrap our `GetWithLockAsync` call in a `try-catch` block to gracefully handle locking failures, either retrying or notifying the user.

Since there isn't a built-in way in EF Core to add query hints, we have to write raw SQL queries.
We can use the PostgreSQL `SELECT FOR UPDATE` statement to acquire a row-level lock on the selected rows.
Any competing transactions will be blocked until the current transaction releases the lock.
This is a very simple way to implement pessimistic locking.

## Flavors of Locking and When to Use Them

To prevent the operation from waiting for other transactions to release any locked rows, you can combine `FOR UPDATE` with:

- `NO WAIT` - Reports an error if the row can't be locked instead of waiting.
- `SKIP LOCKED` - Skips any selected rows that cannot be locked.

Skipping locked rows comes with a caveat - you will get inconsistent results from the database.
However, this can be useful to avoid lock contention when multiple consumers access a queue-like table.
Implementing the [Outbox pattern](outbox-pattern-for-reliable-microservices-messaging) is a great example of this.

**SQL Server**: You'd use the `WITH (UPDLOCK, READPAST)` query hint for a similar effect.

## Pessimistic Locking vs. Serializable Transactions

**Serializable** transactions offer the highest level of data consistency.
They guarantee that all transactions are executed as if they happened in a strict, sequential order, even if they occur simultaneously.
This eliminates the possibility of anomalies like dirty reads (seeing uncommitted data) or non-repeatable reads (data changing between reads).

Here's how it works:

- When a transaction starts under the Serializable isolation level, the database locks all the data the transaction might access.
- These locks are held until the entire transaction is committed or rolled back.
- Any other transaction attempting to access the locked data will be blocked until the first transaction releases its locks.

While Serializable transactions provide the ultimate isolation, they come with a significant cost:

- **Performance Overhead**: Locking a large chunk of data can severely impact performance,
  especially in high-concurrency scenarios.
- **Deadlocks**: With so much locking happening, there's a higher risk of deadlocks.
  These occur when two or more transactions are waiting for locks held by each other, creating a stalemate.

Pessimistic locking with `SELECT FOR UPDATE` offers a more targeted approach to data isolation.
You explicitly lock the specific rows you need to modify.
Other transactions attempting to access the locked rows are blocked until the lock is released.

By locking only the necessary data, pessimistic locking avoids the performance overhead associated with locking everything.
Since you're locking fewer resources, the chances of deadlocks are lower.

## When to Use Each Approach

The best approach depends on your specific needs:

- **Serializable Transactions**: Ideal for scenarios involving highly sensitive data where even the slightest inconsistency is unacceptable.
  Examples include financial transactions and medical record updates.
- **Pessimistic Locking**: A great choice for most use cases, especially in high-traffic applications.
  It provides strong consistency while maintaining good performance and reducing deadlock risks.

## Takeaway

I hope this exploration of pessimistic locking has been helpful.
It's a powerful tool to have in your arsenal if you have scenarios where absolute data consistency is paramount.

Both **Serializable transactions** and pessimistic locking with `SELECT FOR UPDATE` are excellent options for ensuring data consistency.
Consider the level of isolation required, potential performance impact, and the likelihood of deadlocks when making your choice.

That's all for today. Stay awesome, and I'll see you next week.

---



<Sponsorships>

Introducing [**Shesha**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240406), a brand new, open-source, low-code framework for .NET developers.
Create business applications faster and with >80% less code! [**Learn more here.**](https://shesha.io?utm_source=milan&utm_medium=email&utm_campaign=20240406)

[IronXL for .NET](https://ironsoftware.com/csharp/excel/?utm_source=milan&utm_medium=email&utm_campaign=milan):
Streamline Excel management with the intuitive IronXL NET Excel library.
No Microsoft Office or Excel Interop needed.
Edit files across all .NET Core and Framework versions.
[Try it free for 30 days](https://ironsoftware.com/csharp/excel/?utm_source=milan&utm_medium=email&utm_campaign=milan).

</Sponsorships>

[Claims-based authorization](https://learn.microsoft.com/en-us/aspnet/core/security/authorization/claims) mechanisms are central to modern authorization in ASP.NET Core.
However, the access tokens issued by your Identity Provider (IDP) might not always perfectly align with your application's internal authorization needs.

External IDPs like [Microsoft Entra ID](https://www.microsoft.com/en-us/security/business/identity-access/microsoft-entra-id) (previously Azure AD) or
[Auth0](https://auth0.com) might have their own schema for claims or might not directly issue all the claims your application needs for its authorization logic.

The solution? Claims transformation.

Claims transformation allows you to modify the claims before the application uses them for authorization.

In today's issue, we will:

- Explore the concept of claims transformation in ASP.NET Core
- Explore the `IClaimsTransformation` interface with practical examples
- Address considerations for security and RBAC (Role-Based Access Control)

## How Does Claims Transformation Work?

They say a picture is worth a thousand words.
In software engineering, we have something called [UML](https://en.wikipedia.org/wiki/Unified_Modeling_Language) diagrams that we can use to paint a picture.

Here's a [sequence diagram](https://en.wikipedia.org/wiki/Sequence_diagram) showing the claims transformation flow:

1. The user authenticates with the Identity Provider
2. The user calls the backend API and provides an access token
3. The backend API performs claims transformation and authorization
4. If the user is correctly authorized, the backend API returns a response

<Image
  src="blogs/mnw_084/claims_transformation_sequence_diagram.png"
  width={1614}
  height={1357}
  alt="Claims transformation sequence diagram."
/>

Let's see how to implement this in ASP.NET Core.

## Simple Claims Transformation

Claims can be created from any user or identity data issued by a trusted identity provider.
A claim is a name-value pair that represents the subject's identity, not what the subject can do.

The core of [claims transformation](https://learn.microsoft.com/en-us/aspnet/core/security/authentication/claims#extend-or-add-custom-claims-using-iclaimstransformation)
in ASP.NET Core is the [`IClaimsTransformation`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.authentication.iclaimstransformation) interface.

It exposes a single method to transform claims:

```csharp
public interface IClaimsTransformation
{
    Task<ClaimsPrincipal> TransformAsync(ClaimsPrincipal principal);
}
```

Here's a simple example of using `IClaimsTransformation` to add a custom claim:

```csharp
internal static class CustomClaims
{
    internal const string CardType = "card_type";
}

internal sealed class CustomClaimsTransformation : IClaimsTransformation
{
    public Task<ClaimsPrincipal> TransformAsync(ClaimsPrincipal principal)
    {
        if (principal.HasClaim(claim => claim.Type == CustomClaims.CardType))
        {
            return Task.FromResult(principal);
        }

        ClaimsIdentity claimsIdentity = new ClaimsIdentity();

        claimsIdentity.AddClaim(new Claim(CustomClaims.CardType, "platinum"));

        principal.AddIdentity(claimsIdentity);

        return Task.FromResult(principal);
    }
}
```

The `CustomClaimsTransformation` class should be registered as a service:

```csharp
builder.Services
    .AddTransient<IClaimsTransformation, CustomClaimsTransformation>();
```

Finally, you can define a custom authorization policy that uses this claim:

```csharp
builder.Services.AddAuthorization(options =>
{
    options.AddPolicy(
        "HasPlatinumCard",
        builder => builder
            .RequireAuthenticatedUser()
            .RequireClaim(CustomClaims.CardType, "platinum"));
});
```

There are a few caveats with using `IClaimsTransformation` you should be aware of:

- **Might execute multiple times**: The `TransformAsync` method might get called multiple times.
  Claims transformation should be idempotent to avoid adding the same claim multiple times to the `ClaimsPrincipal`.
- **Potential performance impact**: Since it's executed on authentication requests, be mindful of your transformation logic's performance,
  especially if it involves external calls (database, APIs). Consider caching where appropriate.

## Implementing RBAC With Claims Transformation

[Role-Based Access Control (RBAC)](https://auth0.com/docs/manage-users/access-control/rbac) is an authorization model where permissions are assigned to roles,
and users are granted roles.
Claims transformation helps implement RBAC smoothly.
By adding role claims and potentially permission claims, authorization logic throughout your application can be simplified.
Another benefit is that you can keep the access token smaller and free of any role or permission claims.

Let's consider a scenario where your application manages resources at a granular level,
but your identity provider only provides coarse-grained roles like `Registered` or `Member`.
You could use claims transformation to map the `Member` role to specific fine-grained permissions like `SubmitOrder` and `PurchaseTicket`.

Here's a more complex `CustomClaimsTransformation` implementation.
We send a database query using `GetUserPermissionsQuery` and get the `PermissionsResponse` back.
The `PermissionsResponse` contains the user's permissions, which are added as custom claims.

```csharp
internal sealed class CustomClaimsTransformation(
    IServiceProvider serviceProvider)
    : IClaimsTransformation
{
    public async Task<ClaimsPrincipal> TransformAsync(
        ClaimsPrincipal principal)
    {
        if (principal.HasClaim(c => c.Type == CustomClaims.Sub ||
                                    c.Type == CustomClaims.Permission))
        {
            return principal;
        }

        using IServiceScope scope = serviceProvider.CreateScope();

        ISender sender = scope.ServiceProvider.GetRequiredService<ISender>();

        string identityId = principal.GetIdentityId();

        Result<PermissionsResponse> result = await sender.Send(
            new GetUserPermissionsQuery(identityId));

        if (result.IsFailure)
        {
            throw new ClaimsAuthorizationException(
                nameof(GetUserPermissionsQuery), result.Error);
        }

        var claimsIdentity = new ClaimsIdentity();

        claimsIdentity.AddClaim(
            new Claim(CustomClaims.Sub, result.Value.UserId.ToString()));

        foreach (string permission in result.Value.Permissions)
        {
            claimsIdentity.AddClaim(
                new Claim(CustomClaims.Permission, permission));
        }

        principal.AddIdentity(claimsIdentity);

        return principal;
    }
}
```

Now that the `ClaimsPrincipal` contains the permissions as custom claims, you can do some interesting things.
For example, you can implement a permission-based `AuthorizationHandler`:

```csharp {8}
internal sealed class PermissionAuthorizationHandler
    : AuthorizationHandler<PermissionRequirement>
{
    protected override Task HandleRequirementAsync(
        AuthorizationHandlerContext context,
        PermissionRequirement requirement)
    {
        HashSet<string> permissions = context.User.GetPermissions();

        if (permissions.Contains(requirement.Permission))
        {
            context.Succeed(requirement);
        }

        return Task.CompletedTask;
    }
}
```

## Takeaway

Claims transformation is an elegant way to bridge the gap between claims provided by identity providers and the needs of your ASP.NET Core application.
The `IClaimsTransformation` interface enables you to customize the claims of the current `ClaimsPrincipal`.
Whether you need to add roles, map external groups to internal permissions, or extract additional information from a user profile,
claims transformation offers the flexibility to do so.

However, it's important to use claims transformation with a few key considerations in mind:

- Claims transformations are executed on each request.
- The `IClaimsTransformation` should be idempotent. It should not add existing claims to the `ClaimsPrincipal` if executed multiple times.
- Design your transformations efficiently, and consider caching the results if you're fetching external data to enrich your claims.

If you want to see a complete implementation of RBAC in ASP.NET Core, check out this [Authentication & Authorization playlist](https://www.youtube.com/playlist?list=PLYpjLpq5ZDGtJOHUbv7KHuxtYLk1nJPw5).

Hope this was helpful.

See you next week.

---


Latest On YouTube

Let's connect

The .NET Weekly

Latest On YouTube

Let's connect