
<Sponsorships>

[Telerik UI for Blazor](https://www.telerik.com/blogs/why-modernize-blazor-winning-strategy-next-application?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-mnm-content-exposure) -
a winning strategy for your next application.
With .NET 8 just around the corner, there's a lot of talk about Blazor becoming a "drop-in" replacement for MVC or Razor Pages.
If you are exploring how to modernize existing applications, moving from MVC, Razor Pages or Web Forms,
[take a look at what Blazor brings to the table.](https://www.telerik.com/blogs/why-modernize-blazor-winning-strategy-next-application?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-mnm-content-exposure)

[IcePanel](https://u.icepanel.io/1eec0e6f) is a collaborative C4 model modelling & diagramming tool that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.
[Check it out here.](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

Validation is an essential cross-cutting concern that you need to solve in your application.
You want to ensure the request is valid before you consider processing it.

Another important question you need to answer is how you approach different types of validation.
For example, I consider input and business validation differently, and each deserves a specific solution.

I want to show you an elegant solution for validation using [MediatR](https://github.com/jbogard/MediatR)
and [FluentValidation.](https://docs.fluentvalidation.net/en/latest/index.html)

If you aren't using [CQRS](https://learn.microsoft.com/en-us/azure/architecture/patterns/cqrs) with MediatR, don't worry.
Everything I explain about validation can easily be adapted to other paradigms.

Here's what I'm going to talk about in this week's newsletter:

- Standard validation approach
- Input vs business validation
- Separating validation logic
- Generic `ValidationBehavior`

Let's dive in.

## The Standard Command Validation Approach

The standard way of implementing validation is right before processing the command.
The validation is tightly coupled to the command handler, which could be problematic.

I find this approach difficult to maintain as the complexity of the validation increases.
Each change to the validation logic also touches the handler, and the handler itself can grow out of control.

It also makes it harder to differentiate between input and _business_ validation.

Here's an example `ShipOrderCommandHandler` that checks if the `ShippingAddress.Country` is one of the supported countries:

```csharp
internal sealed class ShipOrderCommandHandler
    : IRequestHandler<ShipOrderCommand>
{
    private readonly IOrderRepository _orderRepository;
    private readonly IShippingService _shippingService;
    private readonly ShipmentSettings _shipmentSettings;

    public async Task Handle(
        ShipOrderCommand command,
        CancellationToken cancellationToken)
    {
        if (!_shipmentSettings
                .SupportedCountries
                .Contains(command.ShippingAddress.Country))
        {
            throw new ArgumentException(nameof(ShipOrderCommand.Address));
        }

        var order = _orderRepository.Get(command.OrderId);

        _shippingService.ShipTo(
            command.ShippingAddress,
            command.ShippingMethod);
    }
}
```

What if we can separate command validation and command handling?

## Input Validation and Business Validation

I mentioned input and _business_ validation in the previous section.

Here's how I consider them to be different:

- **Input validation** - We only validate that the command is _processable_.
  These are simple validations, such as checking for `null` values, empty strings, etc.
- **Business validation** - We validate the command to satisfy the business rules.
  This includes checking the system state for required preconditions before processing the command.

Another way to compare them is cheap vs. expensive.
Input validation is usually cheap to execute and can be done in memory.
While business validation involves reading state and is slower.

So, input validation sits at the entry point of the use case before handling the request.
After it completes, we have a _valid_ command.
And this is a rule I always follow - an invalid command should never reach the handler.

## Input Validation With FluentValidation

[FluentValidation](https://docs.fluentvalidation.net/en/latest/index.html) is an excellent validation library for .NET,
which uses a fluent interface and lambda expressions for building strongly typed validation rules.

Here's the `ShipOrderCommand` that we want to validate:

```csharp
public sealed record ShipOrderCommand : IRequest
{
    public Guid OrderId { get; set; }

    public string ShippingMethod { get; set; }

    public Address ShippingAddress { get; set; }
}
```

To implement a validator with [FluentValidation,](https://github.com/FluentValidation/FluentValidation)
you create a class that inherits from the `AbstractValidator<T>` base class.
Then, you can add the validation rules from the constructor using `RuleFor`:

```csharp
public sealed class ShipOrderCommandValidator
    : AbstractValidator<ShipOrderCommand>
{
    public ShipOrderCommandValidator(ShipmentSettings settings)
    {
        RuleFor(command => command.OrderId)
            .NotEmpty()
            .WithMessage("The order identifier can't be empty.");

        RuleFor(command => command.ShippingMethod)
            .NotEmpty()
            .WithMessage("The shipping method can't be empty.");

        RuleFor(command => command.ShippingAddress)
            .NotNull()
            .WithMessage("The shipping address can't be empty.");

        RuleFor(command => command.ShippingAddress.Country)
            .Must(country => settings.SupportedCountries.Contains(country))
            .WithMessage("The shipping country isn't supported.");
    }
}
```

The naming convention I like to use is the name of the command and append _Validator_.
You can also enforce this by writing [architecture tests.](enforcing-software-architecture-with-architecture-tests)

To automatically register all validators from an assembly, you need to call the `AddValidatorsFromAssembly` method:

```csharp
services.AddValidatorsFromAssembly(ApplicationAssembly.Assembly);
```

## Running Validation From the Use Case

To run the `ShipOrderCommandValidator`, you can use the `IValidator<T>` service and inject it from the constructor.

The validator exposes a few methods you can call, like `Validate`, `ValidateAsync`, or `ValidateAndThrow`.

The `Validate` method returns a `ValidationResult` object which contains two properties:

- `IsValid` - a boolean flag saying whether the validation succeeded
- `Errors` - a collection of `ValidationFailure` objects containing any validation failures

Alternatively, calling the `ValidateAndThrow` method throws a `ValidationException` if validation fails.

```csharp
internal sealed class ShipOrderCommandHandler
    : IRequestHandler<ShipOrderCommand>
{
    private readonly IOrderRepository _orderRepository;
    private readonly IShippingService _shippingService;
    private readonly IValidator<ShipOrderCommand> _validator;

    public async Task Handle(
        ShipOrderCommand command,
        CancellationToken cancellationToken)
    {
        _validator.ValidateAndThrow(command);

        var order = _orderRepository.Get(command.OrderId);

        _shippingService.ShipTo(
            command.ShippingAddress,
            command.ShippingMethod);
    }
}
```

This approach forces you to define an explicit dependency on `IValidator` in every command handler.

What if we can implement this cross-cutting concern in a more generic way?

## MediatR Validation Pipeline

Here's a complete implementation of a `ValidationBehavior` using FluentValidation and MediatR's `IPipelineBehavior`.

The `ValidationBehavior` acts as a middleware for the request pipeline and performs validation.
If the validation fails, it will throw a custom `ValidationException` with a collection of `ValidationError` objects.

I also want to highlight the use of `ValidateAsync`, which allows you to define asynchronous validation rules.
You must call the `ValidateAsync` method if you have asynchronous rules.
Otherwise, the validator will throw an exception.

```csharp
public sealed class ValidationBehavior<TRequest, TResponse>
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : ICommandBase
{
    private readonly IEnumerable<IValidator<TRequest>> _validators;

    public ValidationBehavior(IEnumerable<IValidator<TRequest>> validators)
    {
        _validators = validators;
    }

    public async Task<TResponse> Handle(
        TRequest request,
        RequestHandlerDelegate<TResponse> next,
        CancellationToken cancellationToken)
    {
        var context = new ValidationContext<TRequest>(request);

        var validationFailures = await Task.WhenAll(
            _validators.Select(validator => validator.ValidateAsync(context)));

        var errors = validationFailures
            .Where(validationResult => !validationResult.IsValid)
            .SelectMany(validationResult => validationResult.Errors)
            .Select(validationFailure => new ValidationError(
                validationFailure.PropertyName,
                validationFailure.ErrorMessage))
            .ToList();

        if (errors.Any())
        {
            throw new Exceptions.ValidationException(errors);
        }

        var response = await next();

        return response;
    }
}
```

Don't forget to register the `ValidationBehavior` with MediatR by calling `AddOpenBehavior`:

```csharp
services.AddMediatR(config =>
{
    config.RegisterServicesFromAssemblyContaining<ApplicationAssembly>();

    config.AddOpenBehavior(typeof(ValidationBehavior<,>));
});
```

## Handling Validation Exceptions

Here's a custom `ValidationExceptionHandlingMiddleware` middleware that only handles the custom `ValidationException`.
It converts the exception to a `ProblemDetails` response and includes any validation errors.

You can easily expand this to be a generic global exception handler.

```csharp
public sealed class ValidationExceptionHandlingMiddleware
{
    private readonly RequestDelegate _next;

    public ValidationExceptionHandlingMiddleware(RequestDelegate next)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await _next(context);
        }
        catch (Exceptions.ValidationException exception)
        {
            var problemDetails = new ProblemDetails
            {
                Status = StatusCodes.Status400BadRequest,
                Type = "ValidationFailure",
                Title = "Validation error",
                Detail = "One or more validation errors has occurred"
            };

            if (exception.Errors is not null)
            {
                problemDetails.Extensions["errors"] = exception.Errors;
            }

            context.Response.StatusCode = StatusCodes.Status400BadRequest;

            await context.Response.WriteAsJsonAsync(problemDetails);
        }
    }
}
```

You also need to include the middleware in the request pipeline by calling `UseMiddleware`:

```csharp
app.UseMiddleware<ExceptionHandlingMiddleware>();
```

## Takeaway

This implementation of `ValidationBehavior` is something I use in real projects, and it works incredibly well.
If I don't want to throw an exception, I can update the `ValidationBehavior` to return a result object instead.

How do you apply this if you're not using MediatR?

I'm using an `IPipelineBehavior`, which allows me to implement a _middleware_ wrapping each request.

So, all you need is a way to implement middleware and place your validation inside.
And I like having options, so here are [three ways to create middleware in ASP.NET Core.](3-ways-to-create-middleware-in-asp-net-core)

Hope this was valuable.

Stay awesome!

---



<Sponsorships>

Do you build complex software systems?
See how [**NServiceBus**](https://go.particular.net/milanjovanovic) makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling.
[**Get started for FREE.**](https://go.particular.net/milanjovanovic)

Want to learn [**Blazor?**](https://www.jetbrains.com/dotnet/guide/tutorials/blazor-essentials/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetguide)
Check out this [**Blazor Essentials tutorial,**](https://www.jetbrains.com/dotnet/guide/tutorials/blazor-essentials/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetguide)
where Rachel Appel, Developer Advocate at JetBrains, explores working with its components, forms, data, JavaScript interoperability, and more.

</Sponsorships>

You start building a beautiful monolith system.
Maybe a modular monolith.

The system grows over time, and requirements are ever-changing. Slowly, cracks begin to appear in the system.

This could be for organizational reasons and distributing the work across a team.
Or it could be because of scaling issues and performance bottlenecks.

You begin evaluating the possible solutions, and the benefits and tradeoffs of each one.
At last, you come to a decision.

It's time to migrate parts of the system to individual (micro)services.

So, how do we approach this migration from monolith to microservices?

That is the topic of this week's newsletter.

Let's dive in!

## Decoupling Using Bounded Contexts

The first step in moving from a monolith to microservices is identifying the bounded contexts.
Because they represent cohesive parts of the domain that are candidates for extraction.

One solution is to identify [bounded contexts](https://martinfowler.com/bliki/BoundedContext.html)
using the domain-driven design strategic modeling.

Bounded contexts define the explicit boundaries between modules and separate the responsibilities.
This is one of the biggest challenges when migrating to microservices.
[Identifying good boundaries](https://learn.microsoft.com/en-us/azure/architecture/microservices/model/domain-analysis)
between modules ensures microservices are narrowly focused on one problem domain.

Defining boundaries is also easier in a monolith because you aren't working with a distributed system.
[Refactoring bad boundaries](https://cloud.google.com/architecture/microservices-architecture-refactoring-monoliths)
is less risky, and you have more freedom to "get it right".

<Image
  src="blogs/mnw_056/bounded_contexts.png"
  width={978}
  height={597}
  alt="Bounded contexts."
/>

And the size of the bounded context shouldn't worry you.
Instead, focus on [service boundaries.](https://go.particular.net/right-sized-services)

The next problem you need to solve is coupling.
Coupling is manifested in two ways:

- Database dependencies
- Communication between modules

You can solve these problems from the start by building a modular monolith.
But I'll also explain the guiding principles you can use to solve coupling.

## How a Modular Monolith Solves Coupling

A [modular monolith](modular-monolith-communication-patterns#what-is-a-modular-monolith) is a catchy name for a monolith system built from a few
bounded contexts (modules) and following a set of principles to control coupling.
Each module contains a cohesive set of functionalities and is isolated from other modules in the system.
The isolation refers to database dependencies and inter-module communication.

<Image
  src="blogs/mnw_056/modular_monolith.png"
  width={978}
  height={597}
  alt="Modular monolith."
/>

You can think of a module as a distinct application within the system.
A module has its own domain, entities, use cases, database tables.
The modules are deployed together as a single executable application.
But they are otherwise independent.

You can apply different architectural approaches to each module, like [Clean Architecture.](/pragmatic-clean-architecture)

So I mentioned that you need to reduce the coupling between modules.

Here are two principles to solve database coupling:

- Modules can't share tables in the database
- Modules can't directly query the database tables of other modules

Sharing database tables leads to a high degree of coupling, and this is exactly what you are trying to avoid.
You can isolate the data for each module on a logical level using schemas or physically with different databases.

A module should expose a public API that other modules can call.
This public API is the entry point into the module.
And this is the only way for modules to communicate.

Communication between modules can be [synchronous](modular-monolith-communication-patterns#synchronous-communication-with-method-calls)
using method calls, or [asynchronous](modular-monolith-communication-patterns#asynchronous-communication-with-messaging)
using a message bus.

My preferred approach is asynchronous communication using messaging.
It's loosely coupled and makes the transition to microservices easier.

## Adding a Message Broker To The System

To implement asynchronous communication between modules, you can introduce a message broker.
But you don't need to introduce a full-blown message broker from the start.

You can implement messaging between modules using an abstraction like [MassTransit](https://masstransit.io) while abstracting away the transport mechanism.

MassTransit has an in-memory transport that works well inside a single process.
It's very fast.
But it isn't durable, and you can lose messages if the bus is stopped.

You only need to configure a different transport mechanism when introducing a real message broker.
But you don't need to change your messaging code.

<Image
  src="blogs/mnw_056/modular_monolith_queue.png"
  width={978}
  height={597}
  alt="Modular monolith with a queue."
/>

What is the purpose of messaging inside a modular monolith?

Designing your system like this makes the modules loosely coupled and independent.
The price you pay in increased complexity at the start is justified as the project matures.

## Extracting Modules to Microservices

We decided to move from a monolith system to microservices.
Since we built our system in a modular way, the migration comes down to extracting a module into a new process.

You should introduce a [reverse proxy](implementing-an-api-gateway-for-microservices-with-yarp) in front of your services to route incoming traffic.
This will hide the implementation details of the microservices system from client applications.

The new microservice needs to connect to the message bus, but we don't need to change anything in our code.
Using messaging for communication between modules simplifies the migration process.
This might remind you of [event-driven architecture.](https://go.particular.net/break-that-big-ball-of-mud)

If you implement inter-module communication using method calls, you must replace that implementation with HTTP calls over the network.
Because you're now building a distributed system, and the previous implementation using method calls will not work.
You also need to consider authentication, fault tolerance...

<Image
  src="blogs/mnw_056/extracting_modules.png"
  width={978}
  height={597}
  alt="Microservices with extracting modules."
/>

Extracting modules from the monolith system leads to replacing all the functionalities of the old system with new microservices.
This process of migrating to microservices follows the [strangler fig pattern.](https://learn.microsoft.com/en-us/azure/architecture/patterns/strangler-fig)

## Closing Thoughts

The biggest blocker for moving from a monolith to microservices is coupling.
Coupling is a change preventer.
So, this is the first thing you need to tackle.

You need to solve coupling at the database level and between components in the code.
Building the system in a modular way can prevent these problems from the start.

Which is why a Modular monolith is an excellent approach.

You can identify bounded contexts in the system and use them as the boundaries in the monolith.
And getting the boundaries right is easier in a monolith.

Migrating to microservices comes down to extracting the modules into individual services.

Of course, you still need to think about security and fault tolerance because you now have a distributed system.

Talking about architecture in abstract terms can be difficult to grasp, but it's important when discussing conceptual solutions.

I'll show you a practical Modular monolith implementation soon to complete the circle.

Until then, I hope this was valuable.

See you next week!

---



<Sponsorships>

[**Progress**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-blazor-hybrid-ebook)
prepared a free eBook for you - [**Blazor Hybrid and Web in One Solution.**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-blazor-hybrid-ebook)
Dive into Blazor cross-platform apps, potential roadblocks, testing with Razor components, and more.
[**Get the Blazor eBook here.**](https://www.telerik.com/campaigns/blazor/ebook-blazor-hybrid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-blazor-hybrid-ebook)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a collaborative C4 model modelling & diagramming tool that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.
[**Check it out here.**](https://u.icepanel.io/1eec0e6f)

</Sponsorships>

The ability to conditionally turn features on or off in your application without redeploying the code is a powerful tool.

It lets you quickly iterate on new features and frequently integrate your changes with the main branch.

You can use **feature flags** to achieve this.

**Feature flags** are a software development technique that allows you to wrap application features in a conditional statement.
You can then toggle the feature on or off in runtime to control which features are enabled.

We have a lot to cover in this week's newsletter:

- Feature flag fundamentals in .NET
- Feature filters and phased rollouts
- Trunk-based development
- A/B testing

Let's dive in!

## Feature Flags In .NET

[Feature flags](https://github.com/microsoft/FeatureManagement-Dotnet) provide a way for .NET and ASP.NET Core applications to turn features on or off dynamically.

To get started, you need to install the `Microsoft.FeatureManagement` library in your project:

```powershell
Install-Package Microsoft.FeatureManagement
```

This library will allow you to develop and expose application functionality based on features.
It's useful when you have special requirements when a new feature should be enabled and under what conditions.

The next step is to register the required services with dependency injection by calling `AddFeatureManagement`:

```csharp
builder.Services.AddFeatureManagement();
```

And you are ready to create your first feature flag.
Feature flags are built on top of the .NET configuration system.
Any .NET configuration provider can act as the backbone for exposing feature flags.

Let's create a feature flag called `ClipArticleContent` in our `appsettings.json` file:

```json
"FeatureManagement": {
  "ClipArticleContent": false
}
```

By convention, feature flags have to be defined in the `FeatureManagement` configuration section.
But you can change this by providing a different configuration section when calling `AddFeatureManagement`.

Microsoft recommends exposing feature flags using enums and then consuming them with the `nameof` operator.
For example, you would write `nameof(FeatureFlags.ClipArticleContent)`.

However, I prefer defining feature flags as constants in a static class because it simplifies the usage.

```csharp
// Using enums
public enum FeatureFlags
{
    ClipArticleContent = 1
}

// Using constants
public static class FeatureFlags
{
    public const string ClipArticleContent = "ClipArticleContent";
}
```

To check the feature flag state, you can use the `IFeatureManager` service.
In this example, if the `ClipArticleContent` feature flag is turned on, we will return only the first thirty characters of the article's content.

```csharp {8}
app.MapGet("articles/{id}", async (
    Guid id,
    IGetArticle query,
    IFeatureManager featureManager) =>
{
    var article = query.Execute(id);

    if (await featureManager.IsEnabledAsync(FeatureFlags.ClipArticleContent))
    {
        article.Content = article.Content.Substring(0, 50);
    }

    return Results.Ok(article);
});
```

You can also apply feature flags on a controller or endpoint level using the `FeatureGate` attribute:

```csharp
[FeatureGate(FeatureFlags.EnableArticlesApi)]
public class ArticlesController : Controller
{
   // ...
}
```

This covers the fundamentals of using feature flags, and now, let's tackle more advanced topics.

## Feature Filters And Phased Rollouts

The feature flags I showed you in the previous section were like a simple on-off switch.
Although practical, you might want more flexibility from your feature flags.

The `Microsoft.FeatureManagement` package comes with a few built-in feature filters that allow you to create dynamic rules for enabling feature flags.

The available feature filters are
[`Microsoft.Percentage`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.percentagefilter?view=azure-dotnet),
[`Microsoft.TimeWindow`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.timewindowfilter?view=azure-dotnet)
and [`Microsoft.Targeting`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.targetingfilter?view=azure-dotnet).

Here's an example of defining a `ShowArticlePreview` feature flag that uses a percentage filter:

```json {3,6}
"FeatureFlags": {
  "ClipArticleContent": false,
  "ShowArticlePreview": {
    "EnabledFor": [
      {
        "Name": "Percentage",
        "Parameters": {
          "Value": 50
        }
      }
    ]
  }
}
```

This means the feature flag will be randomly turned on 50% of the time.
The downside is the same user might see different behavior on subsequent requests.
A more realistic scenario is to have the feature flag state be cached for the duration of the user's session.

To use the `PercentageFilter`, you need to enable it by calling `AddFeatureFilter`:

```csharp
builder.Services.AddFeatureManagement().AddFeatureFilter<PercentageFilter>();
```

Another interesting feature filter is the `TargetingFilter`, which allows you to target specific users.
Targeting is used in phased rollouts, where you want to introduce a new feature to your users gradually.
You start by enabling the feature for a small percentage of users and slowly increase the rollout percentage while monitoring how the system responds.

## Trunk-based Development and Feature Flags

[Trunk-based development](https://trunkbaseddevelopment.com/) is a Git branching strategy where all developers work in short-lived branches or directly in the trunk, the main codebase.
The _"trunk"_ is the main branch of your repository.
If you're using Git, it will be either the `main` or `master` branch.
Trunk-based development avoids the "merge hell" problem caused by long-lived branches.

So, how do feature flags fit into trunk-based development?

The only way to ensure the trunk is always releasable is to hide incomplete features behind feature flags.
You continue pushing changes to the trunk as you work on the feature while the feature flag remains turned off on the main branch.
When the feature is complete, you turn on the feature flag and release it to production.

<Image
  src="blogs/mnw_055/trunk_based_development.png"
  width={900}
  height={468}
  alt="Trunk based development."
/>

## How I Used Feature Flags for A/B Testing On My Website

A/B testing (split testing) is an experiment where two or more variants of a page (or feature) are shown randomly to users.
Statistical analysis is performed in the background to determine which variation performs better for a given conversion goal.

Here's an example A/B test I performed on my website:

<Image
  src="blogs/mnw_055/split_test.png"
  width={1400}
  height={1700}
  alt="Split test with two variants."
/>

The hypothesis was that removing the image and focusing on the benefits would make more people want to subscribe.
I measure this using the conversion rate, which is the number of people visiting the page divided by the number of people subscribing.

I'm using a platform called [Posthog](https://posthog.com) to run experiments, which automatically calculates the results.

You can see that the _test_ variant has a significantly higher conversion rate, so it becomes the winner of the A/B test.

<Image
  src="blogs/mnw_055/experiment_results.png"
  width={907}
  height={749}
  alt="Split test with two variants experiment results."
/>

## Takeaway

The ability to dynamically turn features on or off without deploying the code is like a superpower.
Feature flags give you this ability with very little work.

You can work with feature flags in .NET by installing the `Microsoft.FeatureManagement` library.
Feature flags build on top of the .NET configuration system, and you can check the feature flag state using the `IFeatureManager` service.

Another use case for feature flags is A/B testing.
I run weekly experiments on my website, testing changes that will improve my conversion rate.
Feature flags help me decide which version of the website to show to the user.
And then, I can measure results based on the user's actions.

I also made a video about [**feature flagging in .NET,**](https://youtu.be/QVEUgIC7Wpo) and you can watch it [**here**](https://youtu.be/QVEUgIC7Wpo) if you want to learn more.

Hope this was valuable.

Stay awesome!

---



<Sponsorships>

[**JetBrains .NET Day Online '23**](https://lp.jetbrains.com/dotnet-day-2023/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetday23)
is a free virtual event that will take place on September 26.
Ten amazing community speakers will share what they're passionate about in the .NET world.
Topics include C#, F#, Blazor, Avalonia, EF Core, xUnit, and more.
[**Register to save the date.**](https://lp.jetbrains.com/dotnet-day-2023/?utm_source=milan&utm_medium=newsletter&utm_campaign=dotnetday23)

Do you build complex software systems?
See how [**NServiceBus**](https://go.particular.net/milanjovanovic) makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling.
[**Get started for FREE.**](https://go.particular.net/milanjovanovic)

</Sponsorships>

How often do you think about concurrency conflicts when writing code?

You write the code for a new feature, confirm that it works, and call it a day.

But one week later, you find out you introduced a nasty bug because you didn't think about concurrency.

The most common issue is race conditions with two competing threads executing the same function.
If you don't consider this during development, you introduce the risk of leaving the system in a corrupted state.

In this week's newsletter, I'll challenge you to spot the race condition in a method for reserving a booking.
The business requirement is you can't have two overlapping reservations for the same dates.

And then, I'll show you how to solve this race condition using EF Core optimistic concurrency.

Let's dive in!

## What's Wrong With This Code?

There's a race condition hiding somewhere in this code snippet.

Can you see it?

```csharp {9,18}
public Result<Guid> Handle(
    ReserveBooking command,
    AppDbContext dbContext)
{
    var user = dbContext.Users.GetById(command.UserId);
    var apartment = dbContext.Apartments.GetById(command.ApartmentId);
    var (startDate, endDate) = command;

    if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate))
    {
        return Result.Failure<Guid>(BookingErrors.Overlap);
    }

    var booking = Booking.Reserve(apartment, user, startDate, endDate);

    dbContext.Add(booking);

    dbContext.SaveChanges();

    return booking.Id;
}
```

The call to `IsOverlapping` is an optimistic check to see if there's an existing booking for the specified dates.

```csharp
if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate)) { }
```

If it returns `true`, we're trying to double-book the apartment.
So we return a failure, and the method completes.

But if it returns `false`, we reserve a booking and call `SaveChanges` to persist the changes in the database.

And there lies the problem.

There's a chance for a concurrent request to pass the `IsOverlapping` check and attempt to reserve the booking.
Without any concurrency control, both requests will succeed, and we will end up with an inconsistent state in the database.

So how can we solve this?

## Optimistic Concurrency With EF Core

The pessimistic concurrency approach acquires a lock for the data before modifying it.
It's slower and causes competing transactions to be blocked until the lock is released.
EF Core doesn't support this approach out of the box.

You can also solve this problem using optimistic concurrency with EF Core.
It doesn't take any locks, but any data modifications will fail to save if the data has changed since it was queried.

To implement optimistic concurrency in EF Core, you need to configure a property as a _concurrency token_.
It's loaded and tracked with the entity.
When you call `SaveChanges`, EF Core will compare the value of the concurrency token to the value in the database.

Let's assume we're using SQL Server, which has a native [`rowversion`](https://learn.microsoft.com/en-us/sql/t-sql/data-types/rowversion-transact-sql?view=sql-server-ver16) column.
The `rowversion` automatically changes when the row is updated, so it's a great option for a concurrency token.

To configure a `byte[]` property as a concurrency token you can decorate it with the `Timestamp` attribute.
It will be mapped to a `rowversion` column in SQL Server.

```csharp {5}
public class Apartment
{
    public Guid Id { get; set; }

    [Timestamp]
    public byte[] Version { get; set; }
}
```

I prefer a different approach because attributes pollute the entity.

You can do the same with the Fluent API.
I will even use a shadow property to hide the concurrency token from the entity class.

```csharp {4,5}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<Apartment>()
        .Property<byte[]>("Version")
        .IsRowVersion();
}
```

The exact configuration will differ based on the database you are using, so check the documentation.

## How Optimistic Concurrency Works In Practice

So here's what changes when we configure the concurrency token.

When loading the `Apartment` entity, EF will also load the concurrency token.

```sql
SELECT a.Id, a.Version
FROM Apartments a
WHERE a.Id = @p0
```

And when we call `SaveChanges`, the update statement will compare the concurrency token value with the one in the database:

```sql
UPDATE Apartments a
SET a.LastBookedOnUtc = @p0
WHERE a.Id = @p1 AND a.Version = @p2;
```

If the `rowversion` in the database changes, the number of updated rows will be `0`.

EF Core expects to update `1` row, so it will throw a `DbUpdateConcurrencyException`, which you need to handle.

## Handling Concurrency Exceptions

Now that you know how to use optimistic concurrency with EF Core, you can fix the previous code snippet.

If two concurrent requests pass the `IsOverlapping` check, only one can complete the `SaveChanges` call.
The other concurrent request will run into a `Version` mismatch in the database and throw a `DbUpdateConcurrencyException`.

In case of a concurrency conflict, we need to add a `try-catch` statement to catch the `DbUpdateConcurrencyException`.
How you handle the actual exception depends on your business requirements.
And sometimes, [race conditions](https://go.particular.net/milanjovanovic/raceconditions) might not even exist.

```csharp {24}
public Result<Guid> Handle(
    ReserveBooking command,
    AppDbContext dbContext)
{
    var user = dbContext.Users.GetById(command.UserId);
    var apartment = dbContext.Apartments.GetById(command.ApartmentId);
    var (startDate, endDate) = command;

    if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate))
    {
        return Result.Failure<Guid>(BookingErrors.Overlap);
    }

    try
    {
        var booking = Booking.Reserve(apartment, user, startDate, endDate);

        dbContext.Add(booking);

        dbContext.SaveChanges();

        return booking.Id;
    }
    catch (DbUpdateConcurrencyException)
    {
        return Result.Failure<Guid>(BookingErrors.Overlap);
    }
}
```

## When Should You Use Optimistic Concurrency?

Optimistic concurrency considers the best scenario is also the most probable one.
It assumes conflicts between transactions will be infrequent and doesn't acquire locks on the data.
This means your system can scale better because there is no blocking slowing down performance.

However, you must still expect concurrency conflicts and implement custom logic to handle them.

Optimistic concurrency is a good choice if your application doesn't expect many conflicts.

Another reason to use optimistic concurrency is when you can't hold an open connection to the database for the length of the transaction.
This is required for pessimistic locking.

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

[**How to Get Started with Telerik Blazor Data Grid:**](https://www.telerik.com/blogs/how-to-get-started-telerik-blazor-data-grid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-content-exposure&utm_content=get-started-telerik-blazor-data-grid)
If there's one thing you're likely to find yourself doing in your Blazor web applications, it's working with data.
The simplest (and most common) way to do this is often via a table (or grid).
[**Learn how the Telerik UI for Blazor DataGrid**](https://www.telerik.com/blogs/how-to-get-started-telerik-blazor-data-grid?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-awareness-hdibt-content-exposure&utm_content=get-started-telerik-blazor-data-grid)
can make working with data in your Blazor web applications much easier.

[**IcePanel**](https://u.icepanel.io/1eec0e6f) is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

</Sponsorships>

Modern software applications rarely work in isolation.
On the contrary, a typical application will talk to several external systems like databases, messaging systems, cache providers, and many 3rd party services.
And it's up to you to ensure everything functions correctly.

Hopefully, I don't have to convince you about the value of writing tests.

You should be writing tests.
Period.

However, I do want to discuss the _value_ of **integration testing**.

**Unit tests** are helpful to test business logic in isolation, without any external services.
They are easy to write and provide almost instant feedback.

But you can't be fully confident in your application without **integration tests**.

So, in this week's newsletter, I'll show you how to use **Docker** for integration testing.

Here's what we will use to write **integration tests**:

- **Testcontainers**
- Docker
- [**xUnit**](creating-data-driven-tests-with-xunit)

Let's dive in!

## What is Testcontainers?

[Testcontainers](https://dotnet.testcontainers.org/) is a library for writing tests with throwaway Docker containers.

Why should you use it?

Integration testing is considered "difficult" because you have to maintain testing infrastructure.
Before running tests, you need to make sure the database is up and running.
You also have to seed any data required for the tests.
If you have tests running in parallel on the same database, they could interfere with each other.

A possible solution could be using in-memory variations of the required services.
But this isn't much different from using mocks.
In-memory services might not have all the features of the production service.

Testcontainers solves this by using Docker to spin up real services for integration testing.

Here's an example of creating a **SQL Server** container:

```csharp
MsSqlContainer dbContainer = new MsSqlBuilder()
    .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
    .WithPassword("Strong_password_123!")
    .Build();
```

You can then use the `MsSqlContainer` instance to get a connection string for the database running inside the container.

Do you see how this is valuable for writing integration tests?

No more need for mocks or fake in-memory databases.
Instead, you can use the real deal.

I won't do a deep dive into this library here, so refer to the documentation for more information.

## Implementing a Custom WebApplicationFactory

**ASP.NET Core** provides an in-memory test server that we can use to spin up an application instance for running tests.
The `Microsoft.AspNetCore.Mvc.Testing` package provides the `WebApplicationFactory` class that we will use as the base for our implementation.

`WebApplicationFactory<TEntryPoint>` is used to create a `TestServer` for the integration tests.

The custom `IntegrationTestWebAppFactory` will do a few things:

- Create and configure a `MsSqlContainer` instance
- Call `ConfigureTestServices` to set up EF Core with the container database
- Start and stop the container instance with `IAsyncLifetime`

`MsSqlContainer` has a `GetConnectionString` method to grab the connection string for the current container.
Note that this can change between tests, as each test class will create a separate container instance.
Test cases inside the same test class will use the same container instance.
So keep that in mind if you need to do a cleanup between tests.

Another thing to keep in mind is **database migrations**.
You will have to run them manually before every test to create the required database structure.

Starting the container instance is done asynchronously using `IAsyncLifetime`.
The container is started inside `StartAsync` before any of the tests run.
And it's stopped inside `StopAsync`.

Here's the complete code for `IntegrationTestWebAppFactory`:

```csharp
public class IntegrationTestWebAppFactory
    : WebApplicationFactory<Program>,
      IAsyncLifetime
{
    private readonly MsSqlContainer _dbContainer = new MsSqlBuilder()
        .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
        .WithPassword("Strong_password_123!")
        .Build();

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureTestServices(services =>
        {
            var descriptorType =
                typeof(DbContextOptions<ApplicationDbContext>);

            var descriptor = services
                .SingleOrDefault(s => s.ServiceType == descriptorType);

            if (descriptor is not null)
            {
                services.Remove(descriptor);
            }

            services.AddDbContext<ApplicationDbContext>(options =>
                options.UseSqlServer(_dbContainer.GetConnectionString()));
        });
    }

    public Task InitializeAsync()
    {
        return _dbContainer.StartAsync();
    }

    public new Task DisposeAsync()
    {
        return _dbContainer.StopAsync();
    }
}
```

## Creating The Base Test Class

The base test class will implement a class fixture interface `IClassFixture`.
It indicates the class contains tests and provides shared object instances across the test cases inside.
This is a good place to instantiate any services that are required for most tests.

For example, I'm creating an `IServiceScope` for resolving scoped services inside the tests.

- `ISender` for sending commands and queries
- `ApplicationDbContext` for database setup or verifying results

```csharp
public abstract class BaseIntegrationTest
    : IClassFixture<IntegrationTestWebAppFactory>,
      IDisposable
{
    private readonly IServiceScope _scope;
    protected readonly ISender Sender;
    protected readonly ApplicationDbContext DbContext;

    protected BaseIntegrationTest(IntegrationTestWebAppFactory factory)
    {
        _scope = factory.Services.CreateScope();

        Sender = _scope.ServiceProvider.GetRequiredService<ISender>();

        DbContext = _scope.ServiceProvider
            .GetRequiredService<ApplicationDbContext>();
    }

    public void Dispose()
    {
        _scope?.Dispose();
        DbContext?.Dispose();
    }
}
```

With all the infrastructure in place, we're finally ready to write the tests.

## Putting It All Together - Writing Integration Tests

Here's a `ProductTests` class with an integration test inside.

I use the _Arrange-Act-Assert_ pattern to structure tests:

- _Arrange_ - create the `CreateProduct.Command` instance
- _Act_ - send the command using `ISender` and store the result
- _Assert_ - use the result from the _Act_ step to verify the database state

The value of writing integration tests like this is that you can use the complete **MediatR** request pipeline.
If you have any `IPipelineBehavior` wrapping the request, it will also be executed.

The same applies if you write your business logic inside service classes.
Instead of resolving the `ISender`, you would resolve the specific services you want to test.

Most importantly, this test uses a real database instance running inside a **Docker container**.

```csharp
public class ProductTests : BaseIntegrationTest
{
    public ProductTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task Create_ShouldCreateProduct()
    {
        // Arrange
        var command = new CreateProduct.Command
        {
            Name = "AMD Ryzen 7 7700X",
            Category = "CPU",
            Price = 223.99m
        };

        // Act
        var productId = await Sender.Send(command);

        // Assert
        var product = DbContext
            .Products
            .FirstOrDefault(p => p.Id == productId);

        Assert.NotNull(product);
    }
}
```

## Running Integration Tests In CI/CD Pipelines

You can also run integration tests with **Testcontainers** inside **CI/CD pipelines**.
The only requirement is that it supports Docker.

**GitHub Actions** does support Docker.
If you are hosting your project there, integration tests will work out of the box.

You can learn more about [**building a CI/CD pipeline with GitHub Actions here.**](how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet)

And if you want a plug-in solution, here's a GitHub Actions workflow you can use:

```yaml
name: Run Tests 🚀

on:
  workflow_dispatch:
  push:
    branches:
      - main

jobs:
  run-tests:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: '7.0.x'

      - name: Restore
        run: dotnet restore ./Products.Api.sln

      - name: Build
        run: dotnet build ./Products.Api.sln --no-restore

      - name: Test
        run: dotnet test ./Products.Api.sln --no-build
```

## Takeaway

**Testcontainers** is an excellent solution for writing **integration tests** with Docker.
You can spin up and configure any **Docker** image and use it from your application.
This is far better than using mocks or in-memory variations because they lack many features.

If you have a CI/CD pipeline that supports Docker, Testcontainers will work out of the box.

Only a few integration tests will drastically improve your confidence in the system.

You can grab the [**source code for this newsletter**](https://github.com/m-jovanovic/testcontainers-sample) on my GitHub.\
It's completely free, so what are you waiting for?

And if you prefer video, here's a quick tutorial on [**integration testing with Testcontainers.**](https://youtu.be/tj5ZCtvgXKY)

Hope this was valuable.

Stay awesome!

---



<Sponsorships>

This week's issue is sponsored by [**NServiceBus.**](https://go.particular.net/milanjovanovic)
Do you build complex software systems?
See how NServiceBus makes it easier to design, build, and manage software systems that use message queues to achieve loose coupling.
[**Get started for FREE.**](https://go.particular.net/milanjovanovic)

And [**ReSharper,**](https://www.jetbrains.com/resharper/?utm_source=milan&utm_medium=newsletter&utm_campaign=resharper)
which is a productivity extension by JetBrains that makes Visual Studio a much better IDE.
Check out the [**ReSharper Essentials**](https://www.jetbrains.com/dotnet/guide/tutorials/resharper-essentials/?utm_source=milan&utm_medium=newsletter&utm_campaign=resharper)
video series to learn how to write better code faster by using on-the-fly code analysis,
automated solution-wide refactorings, and instant fixes to eliminate errors and code smells.

</Sponsorships>

More than 63%+ of organizations said in a [Dzone survey](https://dzone.com/articles/new-research-shows-63-percent-of-enterprises-are-a)
that they are adopting **Microservices** for some or all of their applications.

As more businesses adopt the use of Microservice architectures, we as developers have to become more skilled with Microservices communication.

Working with **distributed systems** is both fun and challenging at the same time.
One of those challenges is designing **effective communication** between services.

More centralization or less centralization?
More coupling or less coupling?
More control or less control?

These are only a few questions you need to answer.

In this week's newsletter, we will:

- Break down **Orchestration vs. Choreography**
- Understand key differences and **tradeoffs** between them
- Create a framework for deciding which approach to use

Let's dive in!

## What Are Microservices?

**Microservices** are a **software architecture** style where an application is built from small, autonomous services.
Each microservice serves a distinct purpose and can be independently deployed.

You probably already know this, so this is a quick refresher.

<Image src="blogs/mnw_052/microservices_hell.png" width={900} height={468} />

Here are the key tenets of Microservices:

- Independent development
- Deployment independence
- Technological freedom
- Scalability
- Resilience

A significant challenge is designing effective inter-service communication within this distributed environment.

Inside a Monolith system, communication happens through direct method calls.
This is a straightforward approach that works well when all components live within a single process.
However, this doesn't work with microservices.

## Orchestration - Command-driven Communication

**Orchestration** is a centralized approach to Microservices communication.
One of the services takes on the role of the **orchestrator** and coordinates the communication between services.

Orchestration uses **command-driven** communication.
The command communicates the intent of the action.
The sender wants something to happen, and the recipient doesn't need to know who sent the command.

An example of orchestration can be a [**Saga implemented with RabbitMQ.**](implementing-the-saga-pattern-with-rebus-and-rabbitmq)

<Image src="blogs/mnw_052/orchestration.png" width={1021} height={600} />

It has some nice **benefits**:

- Simple
- Centralized
- Ease of troubleshooting
- Monitoring is straightforward

**Orchestration** is typically **simpler** to implement and maintain than choreography.
Because there is a central coordinator, you can manage and monitor service interactions.
This, in turn, improves troubleshooting, as you know where to look when something goes wrong.

The **drawbacks** of orchestration are:

- Tight coupling
- Single point of failure
- Difficulty adding, removing, or replacing microservices

## Choreography - Event-driven Communication

**Choreography** is a **decentralized** communication approach.
Choreography uses **event-driven** communication - as opposed to orchestration, which uses commands.

An **event** is something that has happened in the past and is a fact.
The sender does not know who will handle the event or what will happen after processing it.

I talked about events in-depth in the newsletter about [**publishing domain events.**](how-to-use-domain-events-to-build-loosely-coupled-systems)

<Image src="blogs/mnw_052/choreography.png" width={1021} height={600} />

The most important **benefits** of choreography are:

- Loose coupling
- Ease of maintenance
- Decentralized control
- Asynchronous communication

**Choreography** allows microservices to be **loosely coupled**, which means they can operate independently and asynchronously.
This makes the system more scalable and resilient.
A failure of one microservice won't necessarily affect microservices.

Of course, there are **downsides** to choreography:

- Complexity
- Monitoring is difficult
- Difficulty troubleshooting

It's more complex to implement and maintain than orchestration.

Effective monitoring is the biggest challenge from my experience.

## The Crossroads: Which One Should You Choose?

So, how do you know which one to pick for your system?

I always start with the requirements of the system I'm building.
And then, I look at the pros and cons of orchestration vs. choreography.
Here's a small framework to help you decide.

**Orchestration** excels when:

- You need to wait for the completion of intermediate steps (such as credit card payment confirmation)
- You need to make a conditional choice of subsequent steps
- The process must be carried out atomically (entirely or not at all)
- The process needs to be centralized in one place for monitoring

This also means you will need a central database managed by the orchestrator to handle all workflow-related state.

**Choreography** works best when:

- The process can rely on the input message without needing additional context
- Steps clearly follow one another
- Progress is made in one direction

You can benefit from increased flexibility (such as modifying individual steps in isolation)
Unfortunately, choreography can make it difficult to trace, debug, or monitor the processes triggered by an event.
And the larger the event stream is, the more challenging it becomes.

So, think carefully before opting for orchestration or choreography.

Both approaches bring their advantages but also downsides.

## Takeaway

**Orchestration** defines a sequence of steps that each microservice must follow.
This is great for identifying and addressing complex service interdependencies.
Another benefit is that business logic can be managed and monitored in one place.

**Choreography**, on the other hand, is a decentralized technique for microservices communication.
Each service can operate independently while still being part of the larger architecture.

To decide which approach to use, you should observe your system and identify what you stand to gain or lose.
Everything is a tradeoff.

There's also an **alternative approach** I want to mention.

You could go for a **hybrid approach** that integrates orchestration and choreography.

In a **hybrid approach**, you decide which communication technique to use for a specific workflow.
Some workflows can benefit from orchestration, and others can benefit more from choreography.

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

[**Treblle**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan) is a lightweight SDK that helps engineering and product teams build,
ship & maintain REST-based APIs faster.
Simple integration for all popular languages & frameworks, including .NET 6.
[**Check it out here.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)

[**ABP Commercial**](https://commercial.abp.io/?utm_source=milanw&utm_medium=w51) is a complete web development platform built on ABP Framework, perfect for enterprise-grade ASP.NET Core based web applications.
Pre-built application modules, advanced startup templates, rapid application development tooling, professional UI themes and premium support.
[**Get instant access here.**](https://commercial.abp.io/?utm_source=milanw&utm_medium=w51)

</Sponsorships>

**Rate limiting** is about restricting the number of requests to your application.
It's usually applied within a specific time window or based on other criteria.

It's helpful for a few reasons:

- Improves security
- Guards against DDoS attacks
- Prevents overloading of application servers
- Reduces costs by preventing unnecessary resource consumption

**.NET 7** shipped with a **built-in rate limiter**, but you need to know how to implement it correctly.
Or you could grind your system to a halt - and we don't want that.

In this week's newsletter, I'll teach you:

- How to rate limit users by **IP address**
- How to rate limit users by their **identity**
- How to apply **rate limiting** on the **reverse proxy**

So let's dive in!

## Built-In Rate Limiting In .NET 7

Starting with .NET 7, we have access to built-in **rate limiting middleware** in the `Microsoft.AspNetCore.RateLimiting` namespace.
The API is straightforward, and you can create a rate limit policy with a few lines of code.

We can use one of the four **rate limiting algorithms**:

- Fixed window
- Sliding window
- Token bucket
- Concurrency

Here's how to define a **rate limit policy** by calling the `AddTokenBucketLimiter` method:

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    rateLimiterOptions.AddTokenBucketLimiter("token", options =>
    {
        options.TokenLimit = 1000;
        options.ReplenishmentPeriod = TimeSpan.FromHours(1);
        options.TokensPerPeriod = 700;
        options.AutoReplenishment = true;
    });
});
```

Now you can reference the `token` rate limit policy on your endpoint or controller.

You also have to add the `RateLimitingMiddleware` to the request pipeline:

```csharp
app.UseRateLimiter();
```

You can learn more about [**rate limiting in .NET 7 here,**](how-to-use-rate-limiting-in-aspnet-core) so I won't go deeper into the fundamentals.

## Rate Limiting Users By IP Address

The approach I just showed you has a **problem** - the **rate limit policy** is global and **applies to all users**.

Most of the time, you don't want to do this.
Rate limiting should be granular and apply to **individual users**.

Luckily, you can achieve this by creating a `RateLimitPartition`.

The `RateLimitPartition` has two components:

- Partition key
- Rate limiter policy

Here's how to define a rate limiter with a fixed window policy, and the **partition key** is the user's **IP address**.

```csharp {5}
builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy("fixed-by-ip", httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.Connection.RemoteIpAddress?.ToString(),
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
```

Rate limiting by **IP address** can be a good layer of security for **unauthenticated users**.
You don't know who is accessing your system and can't apply more granular rate limiting.
This can help protect your system from malicious users trying to perform a DDoS attack.

You can also [**create chained limiters**](https://learn.microsoft.com/en-us/aspnet/core/performance/rate-limit?view=aspnetcore-7.0#create-chained-limiters) using the `CreateChained` API.
It allows you to pass in multiple `PartitionedRateLimiter`, which are combined into one `PartitionedRateLimiter`.
The chained limiter runs all the input limiters in sequence (one by one).

If your application is running behind a **reverse proxy**, you need to make sure not to rate limit the proxy IP address.
Reverse proxies usually **forward** the original IP address with the `X-Forwarded-For` header.
So you can use it as the **partition key**:

```csharp {5}
builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy("fixed-by-ip", httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            httpContext.Request.Headers["X-Forwarded-For"].ToString(),
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
```

## Rate Limiting Users By Identity

If you require users to **authenticate** with your API, you can determine who the current is.
Then you can use the user's **identity** as the **partition key** for a `RateLimitPartition`.

Here's how you would create such a rate limit policy:

```csharp {5}
builder.Services.AddRateLimiter(options =>
{
    options.AddPolicy("fixed-by-user", httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.User.Identity?.Name?.ToString(),
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
```

I'm using the `User.Identity` value on the `HttpContext` to get the current user's `Name` claim.
This usually corresponds to the `sub` claim inside a JWT - which is the user identifier.

## Applying Rate Limting On The Reverse Proxy

In a robust implementation, you want to **rate limit** on the **reverse proxy** level before the request hits your API.
And if you have a distributed system, this is a requirement.
Otherwise, your system wouldn't function correctly.

There are many reverse proxy implementations to choose from.

**YARP** is a reverse proxy with excellent .NET integration.
Not surprising since it was written in C#.
You can learn more about [**building an API Gateway with YARP here.**](implementing-an-api-gateway-for-microservices-with-yarp)

To implement rate limiting on the reverse proxy with **YARP** you need to:

- Define a rate limit policy (covered in previous examples)
- Configure the `RateLimiterPolicy` for the route in YARP settings

```json
"products-route": {
  "ClusterId": "products-cluster",
  "RateLimiterPolicy": "sixty-per-minute-fixed",
  "Match": {
    "Path": "/products/{**catch-all}"
  },
  "Transforms": [
    { "PathPattern": "{**catch-all}" }
  ]
}
```

The built-in rate limiter middleware uses an **in-memory** store to track the number of requests.
If you want to run your reverse proxy in a high-availability setup, you will need to use a **distributed cache**.
A nice option to look into is using a [**Redis backplane for rate limiting.**](https://github.com/cristipufu/aspnetcore-redis-rate-limiting)

## Closing Thoughts

With the `PartitionedRateLimiter` you can easily create granular rate limit policies.

The two common approaches are:

- Rate limiting by **IP address**
- Rate limiting by the **user identifier**

I was really excited to see the .NET team ship rate limiting.
But, the current implementation has its shortcomings.
The main issue is that it only works **in memory**.
For a **distributed** solution, you need to implement something yourself or use an external library.

You can use the **YARP** reverse proxy to build robust and scalable distributed systems.
And it only takes a few lines of code to add **rate limiting** on the reverse proxy level.
I'm using it extensively in my systems.

Thanks for reading.

And stay awesome!

---



<Sponsorships>

[**JetBrains Rider**](https://www.jetbrains.com/rider/?utm_source=milan&utm_medium=newsletter&utm_campaign=rider)
is a fast and powerful cross-platform .NET IDE that helps you develop ASP.NET Core and
.NET desktop apps, services, libraries, Unity and Unreal Engine games, Xamarin and MAUI apps, and more.
Watch the [**Rider Essentials**](https://www.jetbrains.com/dotnet/guide/tutorials/rider-essentials/?utm_source=milan&utm_medium=newsletter&utm_campaign=rider)
video series with tips and tricks to get more out of the IDE.

[**Workshop: How to Migrate to .NET MAUI? →**](https://www.telerik.com/campaigns/maui-workshop-migration?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-workshop-mnm-na-awareness)
Master the basic concepts behind .NET MAUI and learn about the possibilities of sharing code between web and native mobile and desktop applications.
[**Join the workshop here.**](https://www.telerik.com/campaigns/maui-workshop-migration?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=maui-workshop-mnm-na-awareness)

</Sponsorships>

**Dapper** is a lightweight **object-relational mapper** in .NET.
It's popular because it's easy to use and fast at the same time.

Dapper extends the `IDbConnection` interface with methods for sending SQL queries to the database.

But, because of the nature of SQL, mapping the result into an object model can be tricky.

So in this week's newsletter, I'll teach you how to map:

- Simple queries
- One-to-one relationships
- One-to-many relationships
- Many-to-many relationships

Let's dive in!

## Simple Mapping

Let's first see how to do a **simple mapping** using Dapper.

Writing a query with Dapper has three parts:

- Creating an `IDbConnection` instance
- Writing the SQL query
- Calling a method that Dapper exposes

We will write a SQL query to load a set of `LineItem` objects for a specific `Order`.

```csharp
public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }
}
```

Here's the SQL query returning the result we need:

```sql
SELECT Id AS LineItemId, OrderId, Price, Currency, Quantity
FROM LineItems
WHERE OrderId = @OrderId
```

I'm parameterizing the `Order` identifier using the `@OrderId` syntax.
This is a Dapper convention.
It's important that you use **parameterized queries** to **avoid SQL injection attacks**.

The mapping is straightforward in this case because we are only returning one type from the database.

We call the `QueryAsync` method and specify `LineItem` as the return type.
Make sure to pass in the arguments for this method, the SQL query, and the `OrderId` parameter.
I prefer creating anonymous objects for Dapper parameters.

```csharp
using var connection = new SqlConnection();

var lineItems = await connection.QueryAsync<LineItem>(
    sql,
    new { OrderId = orderId });
```

That's everything you need for a simple mapping.

## Dapper One To One Relationship Mapping

What if the object we want to return from the SQL query contains a **nested object**?

Here's an updated `LineItem` type that also contains a `Product` inside.

```csharp
public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }
}

public class Product
{
    public long ProductId { get; init; }

    public string Name { get; init; }
}
```

Now you need to return two types in the same query.

Here's the updated SQL query with a join on the `Products` table:

```sql
SELECT li.Id AS LineItemId, li.OrderId, li.Price, li.Currency, li.Quantity,
       p.Id AS ProductId, p.Name
FROM LineItems li
JOIN Products p ON p.Id = li.ProductId
WHERE li.OrderId = @OrderId
```

This query is more complicated because we need to use Dapper's **multi-mapping** feature.

In the `QueryAsync` method, we specify both `LineItem` and `Product` as return types and `LineItem` as the final return type for the method.

We must also tell Dapper how to map the `LineItem` and `Product` from the result set into a single `LineItem` object.

And we need to specify the `splitOn` argument, which tells Dapper where one object ends and the next begins.

```csharp
using var connection = new SqlConnection();

var lineItems = await connection.QueryAsync<LineItem, Product, LineItem>(
    sql,
    (lineItem, product) =>
    {
        lineItem.Product = product;

        return lineItem;
    },
    new { OrderId = orderId },
    splitOn: "ProductId");
```

We write more code to make this work, but it should be easy to wrap your head around it.

## Dapper One To Many Relationship Mapping

Another frequent situation is mapping a **one-to-many relationship** from SQL into an object model.

Because you are joining two tables, the result set will contain duplicate data on the "one" side of the relationship.

For this example, let's use an `Order` with a list of `LineItem` objects.

```csharp
public class Order
{
    public long OrderId { get; init; }

    public List<LineItem> LineItems { get; init; } = new();
}

public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }
}
```

Here's the SQL query returning the data we need from the database:

```sql
SELECT o.Id AS OrderId,
       li.Id AS LineItemId, li.OrderId, li.Price, li.Currency, li.Quantity
FROM Orders o
JOIN LineItems li ON li.OrderId = o.Id
WHERE o.Id = @OrderId
```

We're going to get back duplicate `Order` data because of the `JOIN`.
But we only want to return one `Order` with all the line items.

The Dapper mapping function only gives us the `Order` and `LineItem` for the current row in the result set.

One way to solve this is to use a `Dictionary` to store the `Order` and reuse it inside the mapping.

- Store the `Order` in the dictionary if it's not there
- If it is there, add the `LineItem` to the existing `Order` instance

```csharp
using var connection = new SqlConnection();

var ordersDictionary = new Dictionary<long, Order>();

await connection.QueryAsync<Order, LineItem, Order>(
    sql,
    (order, lineItem) =>
    {
        if (ordersDictionary.TryGetValue(order.OrderId, out var existingOrder))
        {
            order = existingOrder;
        }
        else
        {
            ordersDictionary.Add(order.OrderId, order);
        }

        order.LineItems.Add(lineItem);

        return order;
    },
    new { OrderId = orderId },
    splitOn: "LineItemId");

var mappedOrder = ordersDictionary[orderId];
```

A **many-to-many relationship** would use the same idea, except you'll need two dictionaries for each side of the relationship.

## In Summary

**Dapper** is a fantastic library for writing fast database queries using SQL.

Because of how SQL works, mapping into an object model is sometimes complicated.

There are four common scenarios:

- Simple mapping - a flat structure mapped directly from SQL to an object
- One-to-one mapping - provide a mapping function to connect two objects
- One-to-many mapping - manage a dictionary for the "one" side of the relationship
- Many-to-many mapping - same as above, except you need a dictionary for both sides of the relationship

Now you have a cheat sheet for mapping relationships with Dapper.

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

[**Workshop: How to migrate a legacy app to Blazor:**](https://www.telerik.com/campaigns/blazor-workshop-migration?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-workshop-mnm-na-awareness)
Join Ed Charbeneau for a live application-building workshop, where you'll migrate an existing .NET Web Forms application to Blazor in real time.
Learn how to avoid potential roadblocks and master the basic concepts behind .NET 7 and Blazor.
[**Join the workshop here.**](https://www.telerik.com/campaigns/blazor-workshop-migration?utm_medium=cpm&utm_source=milanjovanovic&utm_campaign=blazor-workshop-mnm-na-awareness)

[**Treblle**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan) is a lightweight SDK that helps engineering and product teams build, ship & maintain REST-based APIs faster.
Simple integration for all popular languages & frameworks, including .NET 6.
[**Check it out here.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)

</Sponsorships>

**Modular monoliths** are becoming more popular in the software engineering community.

The allure of **Microservices** is becoming less compelling.
We also have seasoned veterans of our industry saying you should reconsider:

> You shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.

_&mdash; [Martin Fowler](https://martinfowler.com/bliki/MonolithFirst.html)_

Modular monoliths give you the **logical architecture** of Microservices without the operational complexity.
You can safely determine the boundaries between modules.
And refactoring is straightforward and less risky.
They can also be easily migrated into Microservices if you decide to do so.

I've built and maintained several large **Modular monolith** systems in the past years.

In this week's newsletter, I want to focus on the **communication patterns** in the Modular monolith architecture.

But first, let me explain what is a **Modular monolith**.

## What Is a Modular Monolith?

Here's one definition of what a **Modular monolith** is:

> A Modular Monolith is a software design approach in which a monolith is designed with an emphasis on interchangeable (and potentially reusable) modules.

The problem with most monolith systems is that they become **tightly coupled** over time.
Components are deeply intertwined.
Making a change in one component impacts many others.
Introducing new features is difficult and error-prone.

Modular monoliths aim to solve these problems.

<Image
  src="blogs/mnw_049/modular_monolith_diagram.png"
  width={3876}
  height={1880}
/>

A Modular monolith consists of many **loosely coupled** modules.
Modules represent cohesive sets of functionalities.
Modules are also **independent** of each other.

Here are a few examples:

- Payments module
- Shipping module
- Booking module
- Reviews module

If this concept reminds you of Microservices, that's because it should.

Microservices represent self-contained services encapsulating a set of functionalities of the larger system, much like modules in a Modular monolith.

For a Modular monolith to be loosely coupled, you need to solve how modules will communicate.

Modules cannot reference each other directly except through their public APIs.

There are two widely used communication patterns.
Both have pros and cons and a set of tradeoffs that you need to understand.

## Synchronous Communication With Method Calls

The first and easiest communication pattern is simple **method calls** between modules.
Method calls are **synchronous** and very fast because they're in memory.

Module A calls a method declared on the **public API** of Module B and waits until it receives a result.

Each module exposes a **public API**, which can be an `interface` in .NET.

The module will implement this interface internally and hide any implementation details.
You can use the `internal` keyword to make the implementation inaccessible outside the module.

Modules depend on the interfaces at compile-time.
At runtime, **dependency injection** will provide the respective implementation.

<Image
  src="blogs/mnw_049/modular_monolith_sync_communication.png"
  width={3220}
  height={1192}
/>

The benefits of this approach are:

- Speed of in-memory calls
- Easy to implement
- No indirection

But, the drawback of this approach is **strong coupling**.

**Synchronous communication** means that the modules will be tightly coupled.
If one of the modules is unavailable, it will affect any dependent modules.
You can introduce a retry mechanism, but this only goes so far.

## Asynchronous Communication With Messaging

The second communication pattern is asynchronous **messaging** between modules.

Module A sends a message to the message broker in a fire-and-forget fashion.
Module B subscribes to relevant messages and handles them accordingly.

Modules don't need to know about each other, but they do need to know about the message contracts.

**Message contracts** are the **public API** of a module in this scenario.

<Image
  src="blogs/mnw_049/modular_monolith_async_communication.png"
  width={5384}
  height={1569}
/>

The benefits of this approach are:

- High availability
- Loose coupling

**Asynchronous communication** gives us loose coupling because modules communicate using messages.
Module B doesn't need to be available for Module A to send a message.

The obvious drawback of this approach is **increased complexity**.

We're introducing a **message broker** to the system.
This is another infrastructure component we have to manage.
It's also a single point of failure.
If the message broker fails, so does communication between the modules.

You can prevent message loss by storing messages in an [**Outbox**](outbox-pattern-for-reliable-microservices-messaging) before publishing them.
We can always send the messages again from the database in case of a message broker failure.

## Takeaway

**Synchronous communication** between modules is easy to implement, and it's performant. But it comes at the cost of tight coupling between modules.

**Asynchronous communication** using a message broker is loosely coupled. But it's more complex to implement.

So which **communication pattern** should you be using?

It depends.

Asynchronous communication can help you build **loosely coupled** and **independent** modules.
Migrating a **Modular monolith** using messaging into a **distributed system** is much easier.

You extract a module into its own deployment unit.
And the communication between modules remains the same.
Because you are using messaging, you don't need to reimplement anything.

**Synchronous method calls** are an excellent choice to increase development velocity and reduce operational complexity.

I'll let the software architect in you decide.

Thanks for reading.

And stay awesome!

---



<Sponsorships>

[**Software Design Simplified .NET Edition**](https://softwaredesignsimplified.com/?utm_source=newsletter&utm_medium=email&utm_campaign=dotnet-newsletter-29-07-2023)
is a short and practical book for junior and mid-level engineers.
You will learn encapsulation, cohesion, polymorphism, and more in one afternoon.
[**Check it out here.**](https://softwaredesignsimplified.com/?utm_source=newsletter&utm_medium=email&utm_campaign=dotnet-newsletter-29-07-2023)

[**IcePanel**](https://u.icepanel.io/1eec0e6f) realigns your engineering & product teammates on the technical decisions made across the business.
[**Explain your complex software systems**](https://u.icepanel.io/1eec0e6f) with an interactive map that teammates can drill down to the level of detail they need.

</Sponsorships>

I've been using **Clean Architecture** for 6+ years on large-scale applications serving thousands of customers and millions of requests.
Today I want to talk about why it's a great approach for structuring your applications.

I'm aware that Clean Architecture isn't a silver bullet, so I will discuss what types of systems can benefit from this architecture.

Clean architecture isn't revolutionary.

But it's **prescriptive** about how you should structure the code.

It's an evolution of layered architecture, focusing on the core domain and the direction of dependencies.
All dependencies should point inwards, applying **dependency inversion**.

Here are some of the promises of Clean Architecture:

- Maintainability
- Testability
- Loose coupling
- Separation of concerns

It's **independent** of UI, databases, or external services - but you also need to be **pragmatic** (more on this later).

Let's dive in!

## What Is Clean Architecture?

Clean Architecture was created by [**Robert C. Martin**](https://en.wikipedia.org/wiki/Robert_C._Martin), aka Uncle Bob.

Clean Architecture is an approach to organizing a software system to **separate the concerns** of the various components.
Making the system easier to understand and maintain.

You can think about Clean Architecture as a domain-centric approach to organizing dependencies.

There are similar architectures that follow the same domain-centric idea.
You may know them as the Hexagonal and Onion architectures.
They are more or less interchangeable with each other.
And they all place the core domain at the center of the architecture.

This is my high-level interpretation of **Clean Architecture**:

<Image src="blogs/mnw_048/clean_architecture.png" width={3268} height={2006} />

There are four layers inside:

- Domain
- Application
- Infrastructure
- Presentation

Let's see what should live inside each layer.

## Clean Architecture Layers

Here's a breakdown of what should live inside each **layer** of the **Clean Architecture**.

**Domain**

- Contains the core business rules & logic
- It should be independent of other layers in the system
- Should be persistence ignorant - the persistence mechanism shouldn't influence your domain model
- **Examples:** Entities, Value Objects, Domain services, Domain events, Enums, Repository interfaces

**Application**

- Contains the application use cases
- Contains application-specific business rules
- Orchestrates the domain entities to perform business operations
- It should be independent of external concerns (but it doesn't have to be)
- **Examples:** Application services, Commands, Queries, External service interfaces, Exceptions

**Infrastructure**

- Contains anything related to external concerns
- Implements interfaces defined in the layers below
- **Examples:** PostgreSQL, Keycloak, AWS S3, RabbitMQ, Kafka, SendGrid

**Presentation**

- Represents the entry point to the system
- Accepts data from the outside and passes it to the use cases
- Acts as the composition root for dependency injection
- **Examples:** ASP.NET Core, gRPC

[**Here's an example**](clean-architecture-folder-structure) of how to structure the Clean Architecture on a solution level.
You can also group related components together by feature.
It leads to better cohesion and is an excellent option if your project is more complex.

I also made a few videos covering the Clean Architecture project setup:

- [**Clean Architecture walkthrough**](https://youtu.be/tLk4pZZtiDY) (100k+ views)
- [**Clean Architecture project setup from scratch**](https://youtu.be/fe4iuaoxGbA) (40k+ views)

## Where Should You Use Clean Architecture?

**Clean Architecture** is very versatile and applies to various domains and systems.

But, you should play to its strengths and use it only when there's a tangible benefit.

I use the **Clean Architecture** when I want to:

- Apply Domain-Driven Design
- Solve complex business logic
- Build highly testable projects
- Enforce design policies via the architecture

If the above is true for your project, then Clean Architecture is an excellent option.

You should also consider the [**benefits of Clean Architecture.**](clean-architecture-and-the-benefits-of-structured-software-design)

## The Case For Being Pragmatic

I try to be **pragmatic** when using **Clean Architecture**.

Applying what I like and having the freedom of "breaking" Clean Architecture if it will simplify things.

Can this be called Clean Architecture, then?
No, not in the purest sense.

But, I still get **most** of the **benefits** of Clean Architecture.

Here's an example, when I'm applying CQRS in Clean Architecture to implement the use cases.

On the command (write) side, it's valuable to be independent of external concerns, so I will use repositories behind an interface.
I can control the repository contract, and unit testing is straightforward.

But, on the query (read) side, I want to return the response as fast as possible.
Creating an abstraction only adds indirection and reduces performance.

A better approach is to use the EF or Dapper in the handler and query the database.
It's simple, fast, and you can use all the features offered by the ORM.
You don't need a lot of complexity or abstractions on the query side.

I should call this approach [**Pragmatic Clean Architecture**.](/pragmatic-clean-architecture)

## Closing Thoughts

**Clean Architecture** gives you a **standard** for organizing your solution.

You don't have to reinvent the wheel every time at the start of the project.

But, the layered structure and architectural constraints can increase the **complexity** of smaller projects.\
So make sure your project is complex enough to apply Clean Architecture.

Another **caveat** of Clean Architecture is the danger of **over-engineering**.

Don't follow the principles religiously without considering the specific project requirements.\
The overhead of maintaining so many layers and abstractions may not be justified.

Be **pragmatic** and try to make the best decision possible.

Sometimes that means straying from the paradigm.

Hope this was helpful.

I'll see you next week!

---



<Sponsorships>

Today's issue is sponsored by [**Pluralsight.**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/)
Pluralsight's [**2023 State of Cloud report**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/) comes with some surprising findings.
Did you know 70% of organizations have more than half of their infrastructure in the cloud?
Or that 44% of organizations adopt the latest cloud products as soon as they're available?
Find out where you stand in the industry - and [**how to get ahead.**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/)

And by [**ABP Commercial.**](https://commercial.abp.io/?utm_source=milanw&utm_medium=w47&utm_campaign=newsletter-sponsorship)
ABP Commercial is a complete [**web development platform built on ABP Framework**](https://commercial.abp.io/?utm_source=milanw&utm_medium=w47&utm_campaign=newsletter-sponsorship),
perfect for enterprise-grade ASP.NET Core based web applications.
Pre-built application modules, advanced startup templates, rapid application development tooling, professional UI themes and premium support.

</Sponsorships>

In software engineering, "coupling" means how much different parts of a software system depend on each other.
If they are **tightly coupled**, changes to one part can affect many others.
But if they are **loosely coupled**, changes to one part won't cause big problems in the rest of the system.

**Domain events** are a **Domain-Driven Design (DDD)** tactical pattern that we can use to build **loosely coupled** systems.

You can raise a **domain event** from the domain, which represents a fact that has occurred.
And other components in the system can subscribe to this event and handle it accordingly.

Here's what you will learn in this week's newsletter:

- What are [domain events](#what-are-domain-events)
- How they're [different from integration events](#domain-events-versus-integration-events)
- How to [implement](#implementing-domain-events) & [raise domain events](#raising-domain-events)
- How to [publish domain events](#how-to-publish-domain-events-with-ef-core) with EF Core
- How to [handle domain events](#how-to-handle-domain-events) with MediatR

We have a lot to cover, so let's dive in!

## What Are Domain Events?

An **event** is something that has happened in the past.

It is a fact.

Unchangeable.

A **domain event** is something that happened in the domain, and other parts of the domain should be aware of it.

**Domain events** allow you to express side effects explicitly, and provide a better separation of concerns in the domain.
They're an ideal way to trigger side effects across multiple aggregates inside the domain.

It's your responsibility to ensure that publishing a **domain event** is transactional.
You'll see why this is easier said than done.

<Image src="blogs/mnw_047/domain_events.png" width={2704} height={1515} />

## Domain Events Versus Integration Events

You may have heard of **integration events**, and you're now wondering what's the difference between them and **domain events**.

Semantically, they're the same thing: a representation of something that occurred in the past.

However, their **intent is different** and this is important to understand.

Domain events:

- Published and consumed within a single domain
- Sent using an in-memory message bus
- Can be processed synchronously or asynchronously

Integration events:

- Consumed by other subsystems (microservices, Bounded Contexts)
- Sent with a message broker over a queue
- Processed completely asynchronously

So if you're wondering what type of event you should publish, think about the intent and who should be handling the event.

**Domain events** can also be used to **generate integration events**, which leave the domain boundary.

## Implementing Domain Events

My preferred approach to implement **domain events** is creating an `IDomainEvent` abstraction and implementing MediatR `INotification`.

The benefit is you can use **MediatR's publish-subscribe** support to publish a notification to one or multiple handlers.

```csharp
using MediatR;

public interface IDomainEvent : INotification
{
}
```

Now you can implement a concrete domain event.

Here are a few **constraints** to consider when **designing domain events**:

- Immutability - domain events are facts, and should be immutable
- Fat vs Thin domain events - how much information do you need?
- Use past tense for event naming

```csharp
public class CourseCompletedDomainEvent : IDomainEvent
{
    public Guid CourseId { get; init; }
}
```

## Raising Domain Events

After you create your domain events, you want to raise them from the domain.

My approach is creating an `Entity` base class, because only entities are allowed to raise domain events.
You can further encapsulate raising domain events by making the `RaiseDomainEvent` method `protected`.

We're storing domain events in an internal collection, to prevent anyone else from accessing it.
The `GetDomainEvents` method is there to get a snapshot of the collection, and the `ClearDomainEvents` method to clear the internal collection.

```csharp
public abstract class Entity : IEntity
{
    private readonly List<IDomainEvent> _domainEvents = new();

    public IReadOnlyList<IDomainEvent> GetDomainEvents()
    {
        return _domainEvents.ToList();
    }

    public void ClearDomainEvents()
    {
        _domainEvents.Clear();
    }

    protected void RaiseDomainEvent(IDomainEvent domainEvent)
    {
        _domainEvents.Add(domainEvent);
    }
}
```

Now you're entities can inherit from the `Entity` base class and raise domain events:

```csharp
public class Course : Entity
{
    public Guid Id { get; private set; }

    public CourseStatus Status { get; private set; }

    public DateTime? CompletedOnUtc { get; private set; }

    public void Complete()
    {
        Status = CourseStatus.Completed;
        CompletedOnUtc = DateTime.UtcNow;

        RaiseDomainEvent(new CourseCompletedDomainEvent { CourseId = this.Id });
    }
}
```

And all that's left to do is **publish** the **domain events**.

## How To Publish Domain Events With EF Core

An elegant solution for publishing domain events is using **EF Core**.

Since EF Core acts as a **Unit of Work**, you can use it to gather all **domain events** in the current transaction and publish them.

I don't like to complicate things, and simply override the `SaveChangesAsync` method to publish the domain events after persisting the changes in the database.
But you could also use an interceptor.

```csharp
public class ApplicationDbContext : DbContext
{
    public override async Task<int> SaveChangesAsync(
        CancellationToken cancellationToken = default)
    {
        // When should you publish domain events?
        //
        // 1. BEFORE calling SaveChangesAsync
        //     - domain events are part of the same transaction
        //     - immediate consistency
        // 2. AFTER calling SaveChangesAsync
        //     - domain events are a separate transaction
        //     - eventual consistency
        //     - handlers can fail

        var result = await base.SaveChangesAsync(cancellationToken);

        await PublishDomainEventsAsync();

        return result;
    }
}
```

The most **important decision** you will have to make here is **when to publish the domain events**.

I think it makes the most sense to publish after calling `SaveChangesAsync`.
In other words, after saving changes to the database.

This comes with a few tradeoffs:

- Eventual consistency - because messages are processed after the original transactions
- Database inconsistency risk - because handling domain events can fail

Eventual consistency is something I can live with, so I choose to make this tradeoff.

However, introducing a risk of database inconsistency is a big concern.

You can solve this with the [**Outbox pattern,**](outbox-pattern-for-reliable-microservices-messaging)
where you persist your changes to the database and the domain events (as outbox messages) in a single transaction.
Now you have a guaranteed atomic transaction, and the domain events are processed asynchronously using a background job.

If you're wondering what's inside the `PublishDomainEventsAsync` method:

```csharp
private async Task PublishDomainEventsAsync()
{
    var domainEvents = ChangeTracker
        .Entries<Entity>()
        .Select(entry => entry.Entity)
        .SelectMany(entity =>
        {
            var domainEvents = entity.GetDomainEvents();

            entity.ClearDomainEvents();

            return domainEvents;
        })
        .ToList();

    foreach (var domainEvent in domainEvents)
    {
        await _publisher.Publish(domainEvent);
    }
}
```

## How To Handle Domain Events

With all of the plumbing we created so far, we're ready to implement a handler for the domain events.
Luckily, this is the simplest step in the process.

All you have to do is define a class implementing `INotificationHandler<T>` and specify your domain event type as the generic argument.

Here's a handler for the `CourseCompletedDomainEvent`, which takes the domain event and publishes a `CourseCompletedIntegrationEvent` to notify other systems.

```csharp
public class CourseCompletedDomainEventHandler
    : INotificationHandler<CourseCompletedDomainEvent>
{
    private readonly IBus _bus;

    public CourseCompletedDomainEventHandler(IBus bus)
    {
        _bus = bus;
    }

    public async Task Handle(
        CourseCompletedDomainEvent domainEvent,
        CancellationToken cancellationToken)
    {
        await _bus.Publish(
            new CourseCompletedIntegrationEvent(domainEvent.CourseId),
            cancellationToken);
    }
}
```

## In Summary

**Domain events** can help you build a loosely coupled system.
You can use them to separate the core domain logic from the side effects, which can be handled asynchronously.

There's no need to reinvent the wheel for implementing domain events, and you can use the **EF Core** and **MediatR** libraries to build this.

You will have to make the decision when you want to publish domain events.
Publishing before or after saving changes to the database both have their set of **tradeoffs**.

I prefer **publishing** domain events **after saving changes** to the database, and I use the [**Outbox pattern**](outbox-pattern-for-reliable-microservices-messaging)
to add transactional guarantees.
This approach introduces eventual consistency, but it's also more reliable.

Hope this was helpful.

See you next week!

**Today's action step:**
Take a look at [**this video,**](https://youtu.be/AHzWJ_SMqLo) where I explain how to implement domain events to build a decoupled system that scales.

---



<Sponsorships>

Today's issue is sponsored by [**Pluralsight.**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/)
Pluralsight's [**2023 State of Cloud report**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/) comes with some surprising findings.
Did you know 70% of organizations have more than half of their infrastructure in the cloud?
Or that 44% of organizations adopt the latest cloud products as soon as they're available?
Find out where you stand in the industry - and [**how to get ahead.**](https://www.linkedin.com/feed/update/urn:li:activity:7079820953911771137/)

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel realigns your engineering & product teammates on the technical decisions made across the business.
[**Explain your complex software systems with an interactive map**](https://u.icepanel.io/1eec0e6f) that teammates can drill down to the level of detail they need.

</Sponsorships>

**Clean code** is code that's easy to read, maintain, and understand.

I consider writing **clean code** a skill.

And it's a **skill** that **you can learn** and improve with deliberate practice.

My favorite approach for practicing **clean coding** is doing **refactoring exercises**.

So I prepared one for you today, and we're going to improve it one step at a time by applying **clean code principles**.

Let's dive in!

## Starting Point

I like starting with a problem when trying to learn new concepts.

And the more illustrative the problem, the better.

So we'll use some poorly written code as the starting point for our refactoring.

And in each step, I will highlight what the current issue is and how we will fix it.

Here's what I see when I look at the `Process` method:

- Deep nesting of code - 4 levels, to be precise
- Precondition checks are applied one after the other
- Throwing exceptions to represent a failure

How can we turn this into **clean code**?

```csharp
public void Process(Order? order)
{
    if (order != null)
    {
        if (order.IsVerified)
        {
            if (order.Items.Count > 0)
            {
                if (order.Items.Count > 15)
                {
                    throw new Exception(
                        "The order " + order.Id + " has too many items");
                }

                if (order.Status != "ReadyToProcess")
                {
                    throw new Exception(
                        "The order " + order.Id + " isn't ready to process");
                }

                order.IsProcessed = true;
            }
        }
    }
}
```

## #1: Early Return Principle

It should be painfully obvious by now that the initial version is deeply nested because of the `if` statements applying precondition checks.

We'll solve this using the **early return principle**, which states that we should return from a method as soon as the conditions for that have been met.

In the case of the `Process` method, this means moving from a deeply nested structure to a set of **guard clauses**.

```csharp {3-6,8-11,13-16}
public void Process(Order? order)
{
    if (order is null)
    {
        return;
    }

    if (!order.IsVerified)
    {
        return;
    }

    if (order.Items.Count == 0)
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new Exception(
            "The order " + order.Id + " has too many items");
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new Exception(
            "The order " + order.Id + " isn't ready to process");
    }

    order.IsProcessed = true;
}
```

## #2: Merge If Statements To Improve Readability

The **early return principle** makes the `Process` method more readable.

But there's no need to have one **guard clause** after another.

So we can merge all of them into one `if` statement.

The behavior of the `Process` method remains unchanged, but we remove a lot of excess code.

```csharp {3-8}
public void Process(Order? order)
{
    if (order is null ||
        !order.IsVerified ||
        order.Items.Count == 0)
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new Exception(
            "The order " + order.Id + " has too many items");
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new Exception(
            "The order " + order.Id + " isn't ready to process");
    }

    order.IsProcessed = true;
}
```

## #3: Use LINQ For More Concise Code

A quick improvement can be using **LINQ** to make the code more concise and expressive.

Instead of checking for `Items.Count == 0`, I prefer using the LINQ `Any` method.

You could argue that LINQ has worse performance, but I always optimize for readability.

There are far more expensive operations in an application than a simple method call.

```csharp {5}
public void Process(Order? order)
{
    if (order is null ||
        !order.IsVerified ||
        !order.Items.Any())
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new Exception(
            "The order " + order.Id + " has too many items");
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new Exception(
            "The order " + order.Id + " isn't ready to process");
    }

    order.IsProcessed = true;
}
```

## #4: Replace Boolean Expression With Descriptive Method

Merging multiple conditions into one `if` statement means writing less code, but it can **decrease readability** with **complex conditions**.

However, you can fix this and improve readability by using a variable or method with a **descriptive name**.

I prefer using methods, so I will introduce the `IsProcessable` method to represent the precondition check.

```csharp {3, 23-28}
public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new Exception(
            "The order " + order.Id + " has too many items");
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new Exception(
            "The order " + order.Id + " isn't ready to process");
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

## #5: Prefer Throwing Custom Exceptions

Now let's talk about throwing exceptions.
I like to use exceptions for _"exceptional"_ situations only, and I don't use them for flow control in my code.

Having said that, if you _do_ want to use exceptions for flow control, it's better to use **custom exceptions**.

You can introduce valuable contextual information and better describe the reason for throwing the exception.

And if you want to handle these exceptions globally, you can create a base class to be able to catch specific exceptions.

```csharp {10,15}
public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count > 15)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

## #6: Fix Magic Numbers With Constants

A common **code smell** I see is the use of **magic numbers**.

They are usually easy to spot because they're used to check if numeric some condition applies.

The problem with **magic numbers** is that they **carry no meaning**.

The code is harder to reason about, and more error-prone.

Fixing **magic numbers** should be straightforward, and one solution is introducing a constant.

```csharp {1,10}
const int MaxNumberOfLineItems = 15;

public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count > MaxNumberOfLineItems)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != "ReadyToProcess")
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

## #7: Fix Magic Strings With Enums

Similar to **magic numbers**, we have the **magic strings** **code smell**.

A typical use case for **magic strings** is to represent some sort of state.

You'll notice that we're comparing the `Order.Status` value to a **magic string** to check if the order is ready to process.

A few **problems** with **magic strings**:

- Easy to make mistakes (typo)
- Lack of strong typing
- Not refactoring proof

Let's create an `OrderStatus` `enum` to represent the possible states:

```csharp
enum OrderStatus
{
    Pending = 0,
    ReadyToProcess = 1,
    Processed = 2
}
```

And now we have to use the appropriate `OrderStatus` in the check:

```csharp {15,21}
const int MaxNumberOfLineItems = 15;

public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count > MaxNumberOfLineItems)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != OrderStatus.ReadyToProcess)
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
    order.Status = OrderStatus.Processed;
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

## #8: Use The Result Object Pattern

I said I don't prefer using exceptions for flow control. But how can we fix this?

One solution is using the **result object pattern**.

You can use a generic `Result` class to represent all types of results or a specific one like `ProcessOrderResult`.

To make your result objects encapsulated, expose a set of factory methods to create the concrete result type.

```csharp
public class ProcessOrderResult
{
    private ProcessOrderResult(
        ProcessOrderResultType type,
        long orderId,
        string message)
    {
        Type = type;
        OrderId = orderId;
        Message = message;
    }

    public ProcessOrderResultType Type { get; }

    public long OrderId { get; }

    public string? Message { get; }

    public static ProcessOrderResult NotProcessable() =>
      new(ProcessOrderResultType.NotProcessable, default, "Not processable");

    public static ProcessOrderResult TooManyLineItems(long oderId) =>
      new(ProcessOrderResultType.TooManyLineItems, orderId, "Too many items");

    public static ProcessOrderResult NotReadyForProcessing(long oderId) =>
      new(ProcessOrderResultType.NotReadyForProcessing, oderId, "Not ready");

    public static ProcessOrderResult Success(long oderId) =>
      new(ProcessOrderResultType.Success, oderId, "Success");
}
```

Using an `enum` like `ProcessOrderResultType` will make consuming the result object easier with switch expressions.
Here's the `enum` to represent the `ProcessOrderResult.Type`:

```csharp
public enum ProcessOrderResultType
{
    NotProcessable = 0,
    TooManyLineItems = 1,
    NotReadyForProcessing = 2,
    Success = 3
}
```

And now the `Process` method becomes:

```csharp {7,12,17,23}
const int MaxNumberOfLineItems = 15;

public ProcessOrderResult Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return ProcessOrderResult.NotProcessable();
    }

    if (order.Items.Count > MaxNumberOfLineItems)
    {
        return ProcessOrderResult.TooManyLineItems(order);
    }

    if (order.Status != OrderStatus.ReadyToProcess)
    {
        return ProcessOrderResult.NotReadyForProcessing(order);
    }

    order.IsProcessed = true;
    order.Status = OrderStatus.Processed;

    return ProcessOrderResult.Success(order);
}

static bool IsProcessable(Order? order)
{
    return order is not null &&
           order.IsVerified &&
           order.Items.Any();
}
```

Here's how using an `enum` for the `ProcessOrderResult.Type` allows you to write a switch expression:

```csharp
var result = Process(order);

result.Type switch
{
    ProcessOrderResultType.TooManyLineItems =>
        Console.WriteLine($"Too many line items: {result.OrderId}"),

    ProcessOrderResultType.NotReadyForProcessing =>
        Console.WriteLine($"Not ready for processing {result.OrderId}"),

    ProcessOrderResultType.Success =>
        Console.WriteLine($"Processed successfully {result.OrderId}"),

    _ => Console.WriteLine("Failed to process: {OrderId}", result.OrderId),
};
```

## Takeaway

That's it, 8 tips to write **clean code**:

- [Early return principle](#1-early-return-principle)
- [Merge multiple if statements](#2-merge-if-statements-to-improve-readability)
- [Use LINQ for conciseness](#3-use-linq-for-more-concise-code)
- [Replace boolean expression with method](#4-replace-boolean-expression-with-descriptive-method)
- [Prefer throwing custom exceptions](#5-prefer-throwing-custom-exceptions)
- [Replace magic numbers with constants](#6-fix-magic-numbers-with-constants)
- [Replace magic string with enums](#7-fix-magic-strings-with-enums)
- [Use the result object pattern](#8-use-the-result-object-pattern)

Writing **clean code** is a matter of deliberate practice and experience.

Most people will read about **clean coding principles**, but few will strive to apply them daily.

This is where you can set yourself apart.

I also made a video about these **clean code** tips, you can watch it [here.](https://youtu.be/McDvyFglkvU)

Hope this was helpful.

See you next week!

**Today's action step:**
Take a look at your project and see if you're making some of the mistakes I highlighted here.
And then fix them using the clean code tips I shared with you.

---



<Sponsorships>

Today's issue is sponsored by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel realigns your engineering & product teammates on the technical decisions made across the business.
[**Explain your complex software systems with an interactive map**](https://u.icepanel.io/1eec0e6f) that teammates can drill down to the level of detail they need.

</Sponsorships>

Large **Microservice-based** systems can consist of tens or even hundreds of individual services.
A client application needs to have all of this information to be able to make requests to the relevant **microservice** directly.

However, this has numerous issues, such as security concerns, increased complexity, and coupling.

We can solve this by introducing an **API gateway** that acts as a **reverse proxy** to accept API calls from the client application and forward them to the appropriate service.

The **API gateway** also enforces security and ensures scalability and high availability.

In this week's newsletter, I'll show you how to implement an **API gateway** for your **microservices system** using the **YARP reverse proxy**.

Here's what we will cover:

- Difference between **API gateway** and **reverse proxy**
- Installing and configuring **YARP**
- Creating an **API gateway** with **YARP**
- **Authentication** and **rate limiting** on the **API gateway**

Let's dive in.

## What's The Difference Between an API Gateway And a Reverse Proxy?

A **reverse proxy** and an **API gateway** are similar concepts, but they serve different purposes.

A **reverse proxy** acts as an intermediary between clients and servers.
The clients can only call the backend servers through the **reverse proxy**, which forwards the request to the appropriate server.
It hides the implementation details of individual servers inside the internal network.

A **reverse proxy** is commonly used for:

- Load balancing
- Caching
- Security
- SSL termination

<Image src="blogs/mnw_045/reverse_proxy.png" width={2602} height={1502} />

An **API gateway** is a specific type of **reverse proxy** designed for managing APIs.
It acts as a single entry point for API consumers to the various backend services.

The key characteristics of an **API gateway** are:

- Request routing and composition
- Request/response transformations
- Authentication and authorization
- Rate limiting
- Monitoring

Also, note that an **API gateway** can perform **load balancing** and other functionalities mentioned for reverse proxies.

Now let's see how to use a **reverse proxy** to implement an **API gateway**.

## Installing And Configuring YARP

**YARP** (Yet Another Reverse Proxy) is a library developed by Microsoft to address the needs of various teams needing to build a **reverse proxy**.
It's open source and built with .NET, so it integrates nicely with the existing ecosystem.

Let's install `Yarp.ReverseProxy` **NuGet** package to get started:

```powershell
Install-Package Yarp.ReverseProxy
```

Next, we're going to call:

- `AddReverseProxy` to add the required services for **YARP**
- `LoadFromConfig` to load the **reverse proxy** configuration from application settings
- `MapReverseProxy` to introduce the **reverse proxy** middleware

```csharp {3-4,8}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

var app = builder.Build();

app.MapReverseProxy();

app.Run();
```

We need to tell the **YARP** reverse proxy how to route the incoming requests to the individual microservices.

**YARP** uses the concept of `Routes` to represent request patterns for the proxy and `Clusters` to represent the services to forward those requests.

```json {3,6}
{
    "ReverseProxy": {
        "Routes": {
            ...
        },
        "Clusters": {
            ...
        }
    }
}
```

Here's an example **YARP configuration** with a `{**catch-all}` pattern that will route all incoming requests to the one destination server.

```json
{
  "ReverseProxy": {
    "Routes": {
      "ROUTE_NAME": {
        "ClusterId": "CLUSTER_NAME",
        "Match": {
          "Path": "{**catch-all}"
        }
      }
    },
    "Clusters": {
      "CLUSTER_NAME": {
        "Destinations": {
          "destination1": {
            "Address": "https://www.milanjovanovic.tech/"
          }
        }
      }
    }
  }
}
```

## Implementing an API Gateway With YARP

We can use **YARP** to build an **API gateway** by providing the configuration for the services we want to route traffic to.

I created a sample [API gateway implementation with YARP](https://github.com/m-jovanovic/yarp-api-gateway-sample) on GitHub, so you can give it a try.
The system has two services, the `Users.Api` and `Products.Api`, which are .NET 7 applications.

If a request comes in matching the `/users-service/{**catch-all}`, for example `/users-service/users`, it will be routed to the `users-cluster`.
The same logic applies for the `products-cluster`. We can apply more advanced transformations through the `Transforms` section.

```json
{
  "ReverseProxy": {
    "Routes": {
      "users-route": {
        "ClusterId": "users-cluster",
        "Match": {
          "Path": "/users-service/{**catch-all}"
        },
        "Transforms": [{ "PathPattern": "{**catch-all}" }]
      },
      "products-route": {
        "ClusterId": "products-cluster",
        "Match": {
          "Path": "/products-service/{**catch-all}"
        },
        "Transforms": [{ "PathPattern": "{**catch-all}" }]
      }
    },
    "Clusters": {
      "users-cluster": {
        "Destinations": {
          "destination1": {
            "Address": "https://localhost:5201/"
          }
        }
      },
      "products-cluster": {
        "Destinations": {
          "destination1": {
            "Address": "https://localhost:5101/"
          }
        }
      }
    }
  }
}
```

We now have a functioning **API gateway** built with **YARP**, routing requests to two individual services.

But what else can we do with **YARP**?

## Adding Authentication

The **API gateway** can enforce **authentication** and **authorization** at the entry point to the system before letting authenticated requests proceed.

And **YARP** supports integrating with the existing authentication & authorization middleware.

You first need to define an **authorization policy**:

```csharp
builder.Services.AddAuthorization(options =>
{
    options.AddPolicy("authenticated", policy =>
        policy.RequireAuthenticatedUser());
});
```

And call `UseAuthentication` and `UseAuthorization` to add the respective middleware to the request pipeline.
It's important to add them before calling `MapReverseProxy`.

```csharp {1,3}
app.UseAuthentication();

app.UseAuthorization();

app.MapReverseProxy();
```

Now all you have to do is add the `AuthorizationPolicy` section to the reverse proxy configuration:

```json {3}
"users-route": {
  "ClusterId": "users-cluster",
  "AuthorizationPolicy": "authenticated",
  "Match": {
    "Path": "/users-service/{**catch-all}"
  },
  "Transforms": [
    { "PathPattern": "{**catch-all}" }
  ]
}
```

**YARP** will forward most credentials to the proxied services, such as cookies or bearer tokens because it might be important to identify the user in the individual microservices.

## Adding Rate Limiting

You can also use an **API gateway** to introduce **rate limiting** to your system.
It's a technique to limit the number of requests to your API to **improve security** and reduce the load on the servers.

You can learn more about [how to use rate limiting in .NET here.](how-to-use-rate-limiting-in-aspnet-core)

As you can already guess, **YARP** supports the native **rate limiting** mechanism added in .NET 7.

All you need to do is define a **rate limit policy**:

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddFixedWindowLimiter("fixed", options =>
    {
        options.Window = TimeSpan.FromSeconds(10);
        options.PermitLimit = 5;
    });
});
```

Then you need to call `UseRateLimiter` to add the rate limiter middleware to the request pipeline.
It's important to do it before calling `MapReverseProxy`.

```csharp {1}
app.UseRateLimiter();

app.MapReverseProxy();
```

And then, you can apply rate limiting to the desired route using the `RateLimiterPolicy` section in the reverse proxy configuration:

```json {3}
"products-route": {
  "ClusterId": "products-cluster",
  "RateLimiterPolicy": "fixed",
  "Match": {
    "Path": "/products-service/{**catch-all}"
  },
  "Transforms": [
    { "PathPattern": "{**catch-all}" }
  ]
}
```

## In Summary

An **API gateway** is a critical component for a robust **microservices system** implementation.

And **YARP** is an excellent option if you want to build an **API gateway** with .NET.

I created a sample API gateway implementation with YARP, which you can find [here.](https://github.com/m-jovanovic/yarp-api-gateway-sample)
The system consists of two APIs, and the API gateway is configured to route requests between them.
It also implements:

- [Authentication](https://microsoft.github.io/reverse-proxy/articles/authn-authz.html)
- [Rate limiting](how-to-use-rate-limiting-in-aspnet-core)

In this newsletter, we only scratched the surface of what's possible with **YARP**.

Here are some useful resources if you want to learn more:

- [YARP docs](https://microsoft.github.io/reverse-proxy/articles/index.html)
- [Load balancing](https://microsoft.github.io/reverse-proxy/articles/load-balancing.html)
- [Session affinity](https://microsoft.github.io/reverse-proxy/articles/session-affinity.html)
- [Request/response transformations](https://microsoft.github.io/reverse-proxy/articles/transforms.html)

That's all for today.

Hope it was helpful.

**Today's action step:**
Download the [source code](https://github.com/m-jovanovic/yarp-api-gateway-sample) for the sample application implementing an API gateway with YARP, and take it for a spin.
You can challenge yourself by creating multiple instances of a single service and configuring load balancing with the various load balancing algorithms.

---



<Sponsorships>

Today's issue is sponsored by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel realigns your engineering & product teammates on the technical decisions made across the business.
[**Explain your complex software systems with an interactive map**](https://u.icepanel.io/1eec0e6f) that teammates can drill down to the level of detail they need.

And by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

</Sponsorships>

Reducing the size of your API responses can noticeably improve the performance of your application.

And since network bandwidth is a limited resource, you should at least consider the benefits of **response compression**.

Here's what you'll learn in this week's newsletter:

- How to configure **response compression** in .NET
- Server-based vs. application-based compression
- Possible **security risks** and **mitigation strategies**
- How to configure the available compression providers
- How much network bandwidth you could be saving

Let's get started!

## Configuring Response Compression

Using **response compression** in an .NET applications is remarkably easy.

You only have to call these two methods:

- `AddResponseCompression` - to configure the default services for response compression
- `UseResponseCompression` - to add the response compression middleware to the request pipeline

The `UseResponseCompression` method should be called before any middleware that compresses responses.

**Response compression** isn't turned on by default for HTTPS, so you have to enable it by setting `EnableForHttps` to `true`.

```csharp {3-6,10}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddResponseCompression(options =>
{
    options.EnableForHttps = true;
});

var app = builder.Build();

app.UseResponseCompression();

app.MapGet("/", () => "This response will be compressed 📦");

app.Run();
```

It really is that simple.

But...

## When Should You Use Response Compression?

Ideally, you want to be using **server-based response compression** if your application server supports it.

Because the middleware performs **response compression** at the **application level**, it will typically have worse performance.

If you are hosting your application and you can't use server-based compression, then using the response compression middleware is justified.

One more concern should be **security**, because using **response compression over HTTPS** can expose you to [CRIME](https://en.wikipedia.org/wiki/CRIME) and [BREACH](https://en.wikipedia.org/wiki/BREACH) attacks

Here's what you can do to improve security:

- You can mitigate CRIME and BREACH attacks by introducing **anti-forgery tokens** in ASP.NET Core
- Don't send application secrets as part of the request body
- Implement a [**rate limiter**](how-to-use-rate-limiting-in-aspnet-core)

## Configuring Compression Providers

There are two compression providers added by default when you call `AddResponseCompression`:

- `BrotliCompressionProvider`
- `GzipCompressionProvider`

You can further customize the available providers by adding custom compression providers if you want to.

Compression will default to **Brotli** compression when it's supported by the client.
Otherwise, it will default to **Gzip** if that is the supported compression format.

```csharp {4,5,6}
builder.Services.AddResponseCompression(options =>
{
    options.EnableForHttps = true;
    options.Providers.Add<BrotliCompressionProvider>();
    options.Providers.Add<GzipCompressionProvider>();
    options.MimeTypes = ResponseCompressionDefaults.MimeTypes;
});
```

The interesting thing is you can configure the `CompressionLevel` for the **Brotli** and **Gzip** compression providers.

There are four possible values:

- `Optimal` - tries to balance response size and compression speed
- `Fastest` - sacrifices optimal compression for improved speed
- `NoCompression` - self explanatory
- `SmallestSize` - sacrifices compression speed to make the output as small as possible

The default value is `CompressionLevel.Fastest`.

```csharp
builder.Services.Configure<BrotliCompressionProviderOptions>(options =>
{
    options.Level = CompressionLevel.Optimal;
});

builder.Services.Configure<GzipCompressionProviderOptions>(options =>
{
    options.Level = CompressionLevel.SmallestSize;
});
```

## How Much Can You Save?

Let's find out how much network bandwidth we can save by using **response compression**.

Here's a minimal API endpoint returning a list of `Message` objects:

```csharp
app.MapGet("/", () => Results.Ok(
    Enumerable
        .Range(1, 100)
        .Select(num => new Message
        {
            Id = num,
            Content = $"This is the message #{num}"
        })));
```

And here are the results with different providers and compression levels:

- No compression - 4.5kB
- **Gzip** + `CompressionLevel.Fastest` - 569B
- **Gzip** + `CompressionLevel.SmallestSize` - 539B
- **Gzip** + `CompressionLevel.Optimal` - 554B
- **Brotli** + `CompressionLevel.Fastest` - 400B
- **Brotli** + `CompressionLevel.SmallestSize` - 296B
- **Brotli** + `CompressionLevel.Optimal` - 319B

**Brotli** is the clear winner, which is why it's the **default compression provider**.

In the best case scenario, you can reduce the response size by ~93.5%.
Multiply this by the number of requests you're serving daily, and then you can begin to estimate the possible network savings.

One more thing I noticed is that using `CompressionLevel.SmallestSize` has a noticeable **negative impact** on response time.
I can't say this was surprising, so I suggest to simply keep using the default compression level.

## In Summary

[Response compression](https://learn.microsoft.com/en-us/aspnet/core/performance/response-compression?view=aspnetcore-7.0)
is an interesting technique to improve API performance and reduce network costs.

Ideally, you'd want to be using **server-based response compression** if it's supported by your application server.
If that's not the case, **application-based compression** is available in .NET with the response compression middleware.

What's the cost of response compression?

It will increase the CPU load, and can expose some security risks over HTTPS, but there are ways to mitigate this.

In my experience, the **default configuration values** for the compression provider and compression level give excellent results.

That's all for this week.

See you next Saturday.

**Today's action step:**
To see the value of response compression, I suggest enabling it in your application and examining the changes in response size.
You can try the different compression providers by varying the `Accept-Encoding` header, and also configure different compression levels in your application.

---



<Sponsorships>

Today's issue is sponsored by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

And by [**Fiverr.**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fweb-application%3Fref%3Dprogramming_language%253Aasp_net%252Cc)
Fiverr is your one-stop solution for finding skilled [**Web application developers.**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fweb-application%3Fref%3Dprogramming_language%253Aasp_net%252Cc)
Build your custom Web application from scratch with help from skilled freelance coders.
[**Check it out here!**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fweb-application%3Fref%3Dprogramming_language%253Aasp_net%252Cc)

</Sponsorships>

Today's modern applications must deliver the latest information without refreshing the user interface.

If you need to introduce **real-time** functionality to your application in .NET, there's one library you will most likely reach for - **SignalR**.

**SignalR** allows you to push content from your server-side code to any connected clients as changes happen in real-time.

Here's what I'll teach you in this week's newsletter:

- Creating your first **SignalR** `Hub`
- Testing **SignalR** from **Postman**
- Creating strongly typed hubs
- Sending messages to a specific user

Let's see why **SignalR** is so powerful and how easy it is to build **real-time** applications with it.

## Installing And Configuring SignalR

To start using **SignalR** you'll need to:

- Install the NuGet package
- Create the `Hub` class
- Register the SignalR services
- Map and expose the hub endpoint so clients can connect to it

Let's start by installing the `Microsoft.AspNetCore.SignalR.Client` NuGet package:

```powershell
Install-Package Microsoft.AspNetCore.SignalR.Client
```

Then you need a SignalR `Hub`, which is the central component in your application responsible for managing clients and sending messages.

Let's create a `NotificationsHub` by inheriting from the base `Hub` class:

```csharp
public sealed class NotificationsHub : Hub
{
    public async Task SendNotification(string content)
    {
        await Clients.All.SendAsync("ReceiveNotification", content);
    }
}
```

The SignalR `Hub` exposes a few useful properties:

- `Clients` - used to invoke methods on the clients connected to this hub
- `Groups` - an abstraction for adding and removing connections from groups
- `Context` - used for accessing information about the hub caller connection

You can learn more about the `Hub` class [here](https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub?view=aspnetcore-7.0).

Lastly, you need to register the SignalR services by calling the `AddSignalR` method.
You also need to call the `MapHub<T>` method, where you specify the `NotificationsHub` class and the path clients will use to connect to the hub.

```csharp {3, 7}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSignalR();

var app = builder.Build();

app.MapHub<NotificationsHub>("notifications-hub");

app.Run();
```

Now let's see how we can test the `NotificationsHub`.

## Connecting To SignalR Hub From Postman

To test SignalR, you need a client that will connect to the `Hub` instance.
You could create a simple application with Blazor or JavaScript, but I will show you a different approach.

We will use Postman's **WebSocket Request** to connect to the `NotificationsHub`.

<Image
  src="blogs/mnw_043/postman_websocket_request.png"
  width={1011}
  height={747}
/>

Here's what we need to do:

- Connect to the `NotificationsHub`
- Set the communication protocol to JSON
- Send messages to call the `NotificationsHub` methods

All messages need to end with a null termination character, which is just the ASCII character `0x1E`.

Let's start off by sending this message to set the communication protocol to JSON:

```json
{
  "protocol": "json",
  "version": 1
}?
```

You'll receive this response from the hub.

<Image
  src="blogs/mnw_043/postman_set_protocol_request.png"
  width={1731}
  height={890}
/>

We need a slightly different message format to call a message on the `Hub`.
The key is specifying the `arguments` and `target`, which is the actual hub method we want to call.

Let's say we want to call the `SendNotification` method on the `NotificationsHub`:

```json
{
  "arguments": ["This is the notification message."],
  "target": "SendNotification",
  "type": 1
}?
```

This will be the response we get back from the `NotificationsHub`:

<Image
  src="blogs/mnw_043/postman_send_notification_request.png"
  width={1726}
  height={816}
/>

## Strongly Typed Hubs

The base `Hub` class uses the `SendAsync` method to send messages to connected clients.
Unfortunately, we have to use strings to specify client-side methods to invoke, and it's easy to make a mistake.
There's also nothing enforcing which parameters are used.

SignalR supports **strongly typed hubs** that aim to solve this.

First, you need to define a client interface, so let's create a simple `INotificationsClient` abstraction:

```csharp
public interface INotificationsClient
{
    Task ReceiveNotification(string content);
}
```

The arguments don't have to be primitive types and can also be objects. SignalR will take care of serialization on the client side.

After that, you need to update the `NotificationsHub` class to inherit from the `Hub<T>` class to make it strongly typed:

```csharp
public sealed class NotificationsHub : Hub<INotificationsClient>
{
    public async Task SendNotification(string content)
    {
        await Clients.All.ReceiveNotification(content);
    }
}
```

You will lose access to the `SendAsync` method, and only the methods defined in your client interface will be available.

## Sending Server-Side Messages With `HubContext`

What good is a `NotificationsHub` if we can't send notifications from the backend to connected clients?\
Not much.

You can use the `IHubContext<THub>` interface access to the `Hub` instance in your backend code.

And you can use `IHubContext<THub, TClient>` for a strongly typed hub.

Here's a simple Minimal API endpoint that injects an `IHubContext<NotificationsHub, INotificationsClient>` for our strongly typed hub and uses it
to send a notification to all connected clients:

```csharp {3}
app.MapPost("notifications/all", async (
    string content,
    IHubContext<NotificationsHub, INotificationsClient> context) =>
{
    await context.Clients.All.ReceiveNotification(content);

    return Results.NoContent();
});
```

## Sending Messages To a Specific User

The real value of SignalR is being able to **send messages**, or notifications in this example, to a **specific user**.

I've seen some complicated implementations that manage a dictionary with a user identifier and a map of active connections.
Why would you do that when SignalR already supports this functionality?

You can call the `User` method and pass it the `userId` to scope the `ReceiveNotification` message to that specific user.

```csharp {2,6}
app.MapPost("notifications/user", async (
    string userId,
    string content,
    IHubContext<NotificationsHub, INotificationsClient> context) =>
{
    await context.Clients.User(userId).ReceiveNotification(content);

    return Results.NoContent();
});
```

How does **SignalR** know which user to send the message to?

It uses the `DefaultUserIdProvider` internally to extract the user identifier from the claims.
To be specific, it's using the `ClaimTypes.NameIdentifier` claim.
This also implies that you should be authenticated when connecting to the `Hub`, for example, by passing a JWT.

```csharp
public class DefaultUserIdProvider : IUserIdProvider
{
    public virtual string? GetUserId(HubConnectionContext connection)
    {
        return connection.User?.FindFirst(ClaimTypes.NameIdentifier)?.Value;
    }
}
```

By default, all the methods in a hub can be called by unauthenticated users.
So you need to decorate it with an `Authorize` attribute to only allow authenticated clients to access the hub.

```csharp {1}
[Authorize]
public sealed class NotificationsHub : Hub<INotificationsClient>
{
    public async Task SendNotification(string content)
    {
        await Clients.All.ReceiveNotification(content);
    }
}
```

## In Summary

Adding **real-time functionality** to your application creates room for innovation and adds value to your users.

With **SignalR**, you can start building real-time apps in .NET in minutes.

You need to grasp one concept - the `Hub` class.
SignalR abstracts away the message transport mechanism, so you don't have to worry about it.

Make sure to send authenticated requests to **SignalR** hubs and turn on authentication on the `Hub`.
SignalR will internally track the users connecting to your hubs, allowing you to send them messages based on the user identifier.

That's all for today.

Thanks for reading, and have an awesome Saturday.

**Today's action step:**
Look at your project and try to find an opportunity to add real-time functionality.
Commit 30 min. to build a simple proof of concept with SignalR and see if it can improve your project.

---



<Sponsorships>

Today's issue is sponsored by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

And by [**ABP Commercial.**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w42&utm_campaign=newsletter-sponsorship)
ABP Commercial is a complete [**web development platform built on ABP Framework**](https://commercial.abp.io/pricing?utm_source=milanw&utm_medium=w42&utm_campaign=newsletter-sponsorship),
perfect for enterprise-grade ASP.NET Core based web applications.
Pre-built application modules, advanced startup templates, rapid application development tooling, professional UI themes and premium support.

</Sponsorships>

Is the **anemic domain model** an **antipattern**?

It's a domain model without any behavior and only data properties.

Anemic domain models work great in simple applications, but they are difficult to maintain and evolve if you have rich business logic.

The important parts of your business logic and rules end up being scattered all over the application.
It reduces cohesiveness and reusability, and makes adding new features more difficult.

**Rich domain model** attempts to solve this by encapsulating as much of the business logic as possible.

But how can you design a **rich domain model**?

This is a never-ending process of moving business logic into the domain and refining your domain model.

Let's see how to **refactor** from an **anemic domain model** to a **rich domain model**.

## Working With Anemic Domain Model

To understand what working with an **anemic domain model** looks like, I'll use an example of handling a `SendInvitationCommand`.

I omitted the class and its dependencies so that we can focus on the `Handle` method.
It loads some entities from the database, performs validation, executes the business logic, and finally persists the changes in the database and sends an email.

It already implements some good practices like using repositories and returning result objects.

However, it's working with an **anemic domain model**.

A few things indicating this:

- Parameterless constructors
- Public property setters
- Exposed collections

In other words - the classes representing domain entities contain only data properties and no behavior.

The **problems** of an **anemic domain model** are:

- Discoverability of operations
- Potential code duplication
- Lack of encapsulation

We'll apply a few techniques to push logic down into the domain, and try to make the model more domain-driven.
I hope you'll be able to see the value and benefits this will bring.

```csharp
public async Task<Result> Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    if (gathering.Creator.Id == member.Id)
    {
        throw new Exception("Can't send invitation to the creator.");
    }

    if (gathering.ScheduledAtUtc < DateTime.UtcNow)
    {
        throw new Exception("Can't send invitation for the past.");
    }

    var invitation = new Invitation
    {
        Id = Guid.NewGuid(),
        Member = member,
        Gathering = gathering,
        Status = InvitationStatus.Pending,
        CreatedOnUtc = DateTime.UtcNow
    };

    gathering.Invitations.Add(invitation);

    _invitationRepository.Add(invitation);

    await _unitOfWork.SaveChangesAsync();

    await _emailService.SendInvitationSentEmailAsync(member, gathering);

    return Result.Success();
}
```

## Moving Business Logic Into The Domain

The goal is to move as much of the business logic as possible into the domain.

Let's start with the `Invitation` entity and defining a constructor for it.
I can simplify the design by setting the `Status` and `CreatedOnUtc` properties inside the constructor.
I'm also going to make it `internal` so that an `Invitation` instance can only be created within the domain.

```csharp {3-10}
public sealed class Invitation
{
    internal Invitation(Guid id, Gathering gathering, Member member)
    {
        Id = id;
        Member = member;
        Gathering = gathering;
        Status = InvitationStatus.Pending;
        CreatedOnUtc = DateTime.Now;
    }

    // Data properties omitted for brevity.
}
```

The reason I made the `Invitation` constructor `internal` is so that I can introduce a new method on the `Gathering` entity.
Let's call it `SendInvitation` and it will be responsible for instantiating a new `Invitation` instance and adding it to the internal collection.

Currently, the `Gathering.Invitations` collection is `public`, which means anyone can obtain a reference and modify the collection.

We don't want to allow this, so what we can do is encapsulate this collection behind a `private` field.
This moves the responsibility for managing the `_invitations` collection to the `Gathering` class.

Here's how the `Gathering` class looks like now:

```csharp {3,7-12}
public sealed class Gathering
{
    private readonly List<Invitation> _invitations;

    // Other members omitted for brevity.

    public void SendInvitation(Member member)
    {
        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);
    }
}
```

## Moving Validation Rules Into The Domain

The next thing we can do is move the validation rules into the `SendInvitation` method, further enriching the domain model.

Unfortunately, this is still a bad practice because of throwing "expected" exceptions when a validation fails.
If you want to use exceptions to enforce your validation rules you should at least do it right, and use specific exceptions instead of generic ones.

But it would be even better to use a **result object** to express validation errors.

```csharp {7-10,12-15}
public sealed class Gathering
{
    // Other members omitted for brevity.

    public void SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            throw new Exception("Can't send invitation to the creator.");
        }

        if (gathering.ScheduledAtUtc < DateTime.UtcNow)
        {
            throw new Exception("Can't send invitation for the past.");
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);
    }
}
```

Here's how using **result objects** would look like:

```csharp {5,9,14,21}
public sealed class Gathering
{
    // Other members omitted for brevity.

    public Result SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            return Result.Failure(DomainErrors.Gathering.InvitingCreator);
        }

        if (gathering.ScheduledAtUtc < DateTime.UtcNow)
        {
            return Result.Failure(DomainErrors.Gathering.AlreadyPassed);
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);

        return Result.Success();
    }
}
```

The benefit of this approach is we can introduce constants for possible domain errors.
The catalog of domain errors will act as **documentation** for your domain, and make it more expressive.

Finally, here's how the `Handle` method looks like with all the changes so far:

```csharp {12-17}
public async Task<Result> Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    var result = gathering.SendInvitation(member);

    if (result.IsFailure)
    {
        return Result.Failure(result.Errors);
    }

    await _unitOfWork.SaveChangesAsync();

    await _emailService.SendInvitationSentEmailAsync(member, gathering);

    return Result.Success();
}
```

If you take a closer look at the `Handle` method you'll notice it's doing two things:

- Persisting changes to the database
- Sending an email

This means it's **not atomic**.

There's a potential for the database transaction to complete, and the email sending to fail.
Also, sending the email will slow down the method which could affect performance.

How can make this method atomic?

By sending the email in the background. It's not important for our business logic, so this is safe to do.

## Expressing Side Effects With Domain Events

You can use **domain events** to express that something occurred in your domain that might be interesting to other components in your system.

I often use **domain events** to trigger actions in the background, like sending a notification or email.

Let's introduce an `InvitationSentDomainEvent`:

```csharp
public record InvitationSentDomainEvent(Invitation Invitation) : IDomainEvent;
```

We're going to raise this **domain event** inside the `SendInvitation` method:

```csharp {23}
public sealed class Gathering
{
    private readonly List<Invitation> _invitations;

    // Other members omitted for brevity.

    public Result SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            return Result.Failure(DomainErrors.Gathering.InvitingCreator);
        }

        if (gathering.ScheduledAtUtc < DateTime.UtcNow)
        {
            return Result.Failure(DomainErrors.Gathering.AlreadyPassed);
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);

        Raise(new InvitationSentDomainEvent(invitation));

        return Result.Success();
    }
}
```

The goal is to remove the code responsible for sending the email from the `Handle` method:

```csharp
public async Task<Result> Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    var result = gathering.SendInvitation(member);

    if (result.IsFailure)
    {
        return Result.Failure(result.Errors);
    }

    await _unitOfWork.SaveChangesAsync();

    return Result.Success();
}
```

We only want to worry about executing the business logic and persisting any changes to the database.
Part of those changes will also be the **domain event**, which the system will publish in the background.

Of course, we need a respective **handler** for the **domain event**:

```csharp
public sealed class InvitationSentDomainEventHandler
    : IDomainEventHandler<InvitationSentDomainEvent>
{
    private readonly IEmailService _emailService;

    public InvitationSentDomainEventHandler(IEmailService emailService)
    {
        _emailService = emailService;
    }

    public async Task Handle(InvitationSentDomainEvent domainEvent)
    {
        await _emailService.SendInvitationSentEmailAsync(
            domainEvent.Invitation.Member,
            domainEvent.Invitation.Gathering);
    }
}
```

We achieved two things:

- Handling the `SendInvitationCommand` is now atomic
- Email is sent in the background, and can be safely retried in case of an error

## Takeaway

Designing a **rich domain model** is a gradual process, and you can slowly evolve the domain model over time.

The first step could be making your domain model more defensive:

- Hiding constructors with the `internal` keyword
- Encapsulating collection access

The benefit is your domain models will have a fine-grained public API (methods) which act as an entry point for executing the business logic.

It's easy to test behavior when it's encapsulated in a class without having to mock external dependencies.

You can raise **domain events** to notify the system that something of important occurred, and any interested components can subscribe to that domain event.
Domain events allow you to develop a **decoupled** system, where you focus on the core domain logic, and don't have to worry about the side effects.

However, this doesn't mean that every system needs a **rich domain model**.

You should be pragmatic and decide when the complexity is worth it.

That's all for this week.

See you next Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Fiverr.**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fapi-integrations)
Fiverr is your one-stop solution for finding skilled freelance developers.
Build custom APIs and integrations tailored to your needs.
Seamlessly connect with payment providers (Stripe, PayPal) and automation platforms (Zapier) for streamlined operations.
Check it out [**here!**](https://go.fiverr.com/visit/?bta=738595&brand=fiverrcpa&landingPage=https%3A%2F%2Fwww.fiverr.com%2Fcategories%2Fprogramming-tech%2Fsoftware-development%2Fapi-integrations)

And by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

</Sponsorships>

If you're building a **.NET** application, chances are high that you'll need to call an **external API** over **HTTP**.

The easy way to make HTTP requests in .NET is to use the `HttpClient` to send those requests.
And it's a great abstraction to work with, especially with the methods supporting **JSON** payloads and responses.

Unfortunately, it's easy to misuse the `HttpClient`.

**Port exhaustion** and **DNS behavior** are some of the most common problems.

So here's what you need to know about working with `HttpClient`:

- How not to use `HttpClient`
- How to simplify configuration with `IHttpClientFactory`
- How to configure **typed clients**
- Why you should avoid **typed clients** in singleton services
- When to use which option

Let's dive in!

## The Naive Way To Use HttpClient

The simplest way to work with the `HttpClient` is to just create a new instance, set the required properties and use it to send requests.

What could possibly go wrong?

`HttpClient` instances are meant to be **long-lived**, and reused throughout the lifetime of the application.

Each instance uses its own **connection pool** for isolation purposes, but also to prevent **port exhaustion**.
If a server is under high load, and your application is constantly creating new connections, it could lead to exhausting the available ports.
This will cause an exception at runtime, when trying to send a request.

So how can you avoid this?

```csharp
public class GitHubService
{
    private readonly GitHubSettings _settings;

    public GitHubService(IOptions<GitHubSettings> settings)
    {
        _settings = settings.Value;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        var client = new HttpClient();

        client.DefaultRequestHeaders.Add("Authorization", _settings.GitHubToken);
        client.DefaultRequestHeaders.Add("User-Agent", _settings.UserAgent);
        client.BaseAddress = new Uri("https://api.github.com");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## The Smart Way To Create HttpClient Using IHttpClientFactory

Instead of managing the `HttpClient` lifetime yourself, you can use an `IHttpClientFactory` to create the `HttpClient` instance.

Simply call the `CreateClient` method and use the returned `HttpClient` instance to send your HTTP requests.

Why is this a better approach?

The expensive part of the `HttpClient` is the actual message handler - `HttpMessageHandler`.
Each `HttpMessageHandler` has an internal HTTP **connection pool** that can be reused.

The `IHttpClientFactory` will **cache** the `HttpMessageHandler` and reuse it when creating a new `HttpClient` instance.

An important note here is that `HttpClient` instances created by `IHttpClientFactory` are meant to be **short-lived**.

```csharp
public class GitHubService
{
    private readonly GitHubSettings _settings;
    private readonly IHttpClientFactory _factory;

    public GitHubService(
        IOptions<GitHubSettings> settings,
        IHttpClientFactory factory)
    {
        _settings = settings.Value;
        _factory = factory;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        var client = _factory.CreateClient();

        client.DefaultRequestHeaders.Add("Authorization", _settings.GitHubToken);
        client.DefaultRequestHeaders.Add("User-Agent", _settings.UserAgent);
        client.BaseAddress = new Uri("https://api.github.com");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Reducing Code Duplication With Named Clients

Using `IHttpClientFactory` will solve most of the issues of manually creating an `HttpClient`.
However, we still need to configure the default request parameters every time we obtain a new `HttpClient` from the `CreateClient` method.

You can configure a **named client** by calling the `AddHttpClient` method and passing in the desired name.
The `AddHttpClient` accepts a delegate that you can use to configure the default parameters on the `HttpClient` instance.

```csharp
services.AddHttpClient("github", (serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
});
```

The main difference is you now have to obtain the client by passing the name of the client to `CreateClient`.

But the use of the `HttpClient` looks a lot simpler:

```csharp
public class GitHubService
{
    private readonly IHttpClientFactory _factory;

    public GitHubService(IHttpClientFactory factory)
    {
        _factory = factory;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        var client = _factory.CreateClient("github");

        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Replacing Named Clients With Typed Clients

The downside of using **named clients** is having to resolve an `HttpClient` by passing in a name every time.

There's a better way to achieve the same behavior by configuring a **typed client**.
You can do this by calling the `AddClient<TClient>` method and configuring the service that will consume the `HttpClient`.

Under the hood, this is still using a **named client**, where the name is the same as the type name.

And this will also register `GitHubService` with a **transient lifetime**.

```csharp
services.AddHttpClient<GitHubService>((serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
});
```

Inside of `GitHubService`, you inject and use the typed `HttpClient` instance which will have all of the configuration applied.

No more dealing with `IHttpClientFactory` and creating `HttpClient` instances manually.

```csharp
public class GitHubService
{
    private readonly HttpClient client;

    public GitHubService(HttpClient client)
    {
        _client = client;
    }

    public async Task<GitHubUser?> GetUserAsync(string username)
    {
        GitHubUser? user = await client
            .GetFromJsonAsync<GitHubUser>($"users/{username}");

        return user;
    }
}
```

## Why You Should Avoid Typed Clients In Singleton Services

You could run into a **problem** if you inject a **typed client** into a **singleton service**.
Since the **typed client** is **transient**, injecting it in a **singleton service** will cause it to be cached for the lifetime of the **singleton service**.

This will prevent the **typed client** from reacting to DNS changes.

If you want to use a **typed client** in a **singleton service**, the recommened approach is using `SocketsHttpHandler` as the primary handler,
and configuring the `PooledConnectionLifetime`.

Since the `SocketsHttpHandler` will handle connection pooling, you can disable recycling at the `IHttpClientFactory` level by setting `HandlerLifetime` to `Timeout.InfiniteTimeSpan`.

```csharp
services.AddHttpClient<GitHubService>((serviceProvider, client) =>
{
    var settings = serviceProvider
        .GetRequiredService<IOptions<GitHubSettings>>().Value;

    client.DefaultRequestHeaders.Add("Authorization", settings.GitHubToken);
    client.DefaultRequestHeaders.Add("User-Agent", settings.UserAgent);

    client.BaseAddress = new Uri("https://api.github.com");
})
.ConfigurePrimaryHttpMessageHandler(() =>
{
    return new SocketsHttpHandler()
    {
        PooledConnectionLifetime = TimeSpan.FromMinutes(15)
    };
})
.SetHandlerLifetime(Timeout.InfiniteTimeSpan);
```

## When Should You Use Which Option?

I showed you a few possible options for working with `HttpClient`.

But which one should you use and when?

Microsoft was kind enough to provide us with a set of best practices and [recommended use](https://learn.microsoft.com/en-us/dotnet/fundamentals/networking/http/httpclient-guidelines#recommended-use)
for `HttpClient`.

- Use a `static` or **singleton** `HttpClient` instance with a `PooledConnectionLifetime` configured, since this solves both port exhaustion and tracking DNS changes
- Use `IHttpClientFactory` if you want to move the configuration to one place, but remember that clients are meant to be **short-lived**
- Use a **typed client** if you want the `IHttpClientFactory` configurability

I prefer working with a **typed client**, and I'm mindful of the fact that it's configured as a **transient service**.

Thanks for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Treblle.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Treblle is a lightweight SDK that helps engineering and product teams [**build, ship & maintain REST-based APIs faster.**](https://www.treblle.com/?utm_source=MailInf&utm_medium=Milan)
Simple integration for all popular languages & frameworks, including .NET 6.

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

</Sponsorships>

If you're building a scalable application, it's a common requirement to offload some work in your application to a **background job**.

Here are a few examples of that:

- Publishing email notifications
- Generating reports
- Updating a cache
- Image processing

How can you create a recurring **background job** in .NET?

[**Quartz.NET**](https://www.quartz-scheduler.net/) is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems.

There are three concepts you need to understand in **Quartz.NET**:

- **Job** - the actual background task you want to run
- **Trigger** - the trigger controlling when a job runs
- **Scheduler** - responsible for coordinating jobs and triggers

Let's see how we can use **Quartz.NET** to create and schedule **background jobs**.

## Adding The Quartz.NET Hosted Service

The first thing we need to do is install the **Quartz.NET** NuGet package.
There are a few to pick from, but we're going to install the `Quartz.Extensions.Hosting` library:

```powershell
Install-Package Quartz.Extensions.Hosting
```

The reason we're using this library is because it integrates nicely with .NET using an `IHostedService` instance.

To get the **Quartz.NET** hosted service up and running, we need two things:

- Add the required services with the DI container
- Add the hosted service

```csharp
services.AddQuartz(configure =>
{
    configure.UseMicrosoftDependencyInjectionJobFactory();
});

services.AddQuartzHostedService(options =>
{
    options.WaitForJobsToComplete = true;
});
```

**Quartz.NET** will create jobs by fetching them from the DI container.
This also means you can use scoped services in your jobs, not just singleton or transient services.

Setting the `WaitForJobsToComplete` option to `true` will ensure that **Quartz.NET** waits for the jobs to complete gracefully before exiting.

## Creating Background Jobs With `IJob`

To crate a background job with **Quartz.NET** you need to implement the `IJob` interface.

It only exposes a single method - `Execute` - where you can place the code for your background job.

A few things worth noting here:

- We're using DI to inject the `ApplicationDbContext` and `IPublisher` services
- The job is decorated with `DisallowConcurrentExecution` to prevent running the same job concurrently

```csharp
[DisallowConcurrentExecution]
public class ProcessOutboxMessagesJob : IJob
{
    private readonly ApplicationDbContext _dbContext;
    private readonly IPublisher _publisher;

    public ProcessOutboxMessagesJob(
        ApplicationDbContext dbContext,
        IPublisher publisher)
    {
        _dbContext = dbContext;
        _publisher = publisher;
    }

    public async Task Execute(IJobExecutionContext context)
    {
        List<OutboxMessage> messages = await _dbContext
            .Set<OutboxMessage>()
            .Where(m => m.ProcessedOnUtc == null)
            .Take(20)
            .ToListAsync(context.CancellationToken);

        foreach (OutboxMessage outboxMessage in messages)
        {
            IDomainEvent? domainEvent = JsonConvert
                .DeserializeObject<IDomainEvent>(
                    outboxMessage.Content,
                    new JsonSerializerSettings
                    {
                        TypeNameHandling = TypeNameHandling.All
                    });

            if (domainEvent is null)
            {
                continue;
            }

            await _publisher.Publish(domainEvent, context.CancellationToken);

            outboxMessage.ProcessedOnUtc = DateTime.UtcNow;

            await _dbContext.SaveChangesAsync();
        }
    }
}
```

Now that the **background job** is ready, we need to register it with the **DI** container and add a trigger that will run the job.

## Configuring the Job

I mentioned at the start that there are three key concepts in **Quartz.NET**:

- Job
- Trigger
- Scheduler

We already implemented the `ProcessOutboxMessagesJob` background job in the previous section.

The **Quartz.NET** library will take care of the scheduler.

And this leaves us with configuring the **trigger** for our `ProcessOutboxMessagesJob`.

```csharp
services.AddQuartz(configure =>
{
    var jobKey = new JobKey(nameof(ProcessOutboxMessagesJob));

    configure
        .AddJob<ProcessOutboxMessagesJob>(jobKey)
        .AddTrigger(
            trigger => trigger.ForJob(jobKey).WithSimpleSchedule(
                schedule => schedule.WithIntervalInSeconds(10).RepeatForever()));

    configure.UseMicrosoftDependencyInjectionJobFactory();
});
```

We need to uniquely identify our **background job** with a `JobKey`.
I like to keep it simple and use the job name.

Calling `AddJob` will register the `ProcessOutboxMessagesJob` with DI and also with Quartz.

After that we configure a trigger for this job by calling `AddTrigger`.
You need to associate the job with the trigger by calling `ForJob`, and then you configure the schedule for the background job.
In this example, I'm scheduling the job to run every ten seconds and repeat forever while the hosted service is running.

**Quartz** also has support for configuring triggers using [cron expressions.](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/crontriggers.html)

## Job Persistence

By default, Quartz configures all jobs using the `RAMJobStore` which is the most performant because it keeps all of its data in RAM.
However, this also means it's volatile and you can lose all scheduling information when your application stops or crashes.

It could be useful to have a persistent job store in some scenarios and there's a built in `AdoJobStore` which works with SQL databases.
You need to create a set of database tables for Quartz.NET to use.

You can learn more about this in the [job stores documentation.](https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/job-stores.html)

## Takeaway

**Quartz.NET** makes running **background jobs** in .NET easy, and you can use all the power of DI in your **background jobs**.
It's also flexible for various scheduling requirements with configuration via code or using cron expressions.

There's some room for improvement to make scheduling jobs easier and reduce boilerplate:

- Add an extension method to simplify configuring jobs with a simple schedule
- Add an extension method to simplify configuring jobs with a cron schedule from application settings

If you want to see a tutorial on using **Quartz.NET**, I made an in-depth video about [**using Quartz for processing Outbox messages**](https://youtu.be/XALvnX7MPeo).

That's all for this week.

See you next Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Tower.**](https://www.git-tower.com/?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly)
Try Tower, a [**powerful Git client for Windows**](https://www.git-tower.com/?utm_source=newsletter&utm_medium=sponsor&utm_campaign=milans-net-weekly) focused on productivity.
Make the complex simple with helpful features like Undo, Drag and Drop and Remote Services Integration!

And by [**IcePanel.**](https://u.icepanel.io/1eec0e6f)
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

</Sponsorships>

Do you want to streamline your software development process and accelerate your release cycles?

Imagine being able to automatically build, test, and deploy your .NET applications with every code change.

With **CI/CD**, you can significantly reduce manual effort and focus more on creating software, ensuring faster and more reliable releases.

And it's never been easier to get started with **CI/CD**.

**GitHub Actions** are completely free and simple to use.

So here's what we'll cover:

- Introduction to **CI/CD** & **GitHub Actions**
- Creating a **build & test pipeline** for **.NET**
- Creating a **deployment pipeline** for **Azure App Service**

Let's dive in.

## What Is Continuous Integration And Delivery?

I'll try to briefly explain what **CI/CD** is, before we take a look at **GitHub Actions**.

**CI/CD** is a method to increase the frequency of delivering new features by adding automation to your software development workflow.

**Continuous Integration ("CI")** refers to the automation process of syncing new code to a repository.
Any new changes to the application code are immediately built, tested and merged.

**Continuous Delivery, or Deployment, ("CD")** refers to the process of automating the deployment part of the workflow.
When you make a change which gets merged to the repository, this step takes care of deploying those changes to the production environment (or any other environment).

## Continuous Integration With GitHub Actions

If you're using **GitHub**, getting started with **Continuous Integration** has never been easier.

You can use [**GitHub Actions**](https://github.com/features/actions) to automate your build, test, and deployment **pipeline**.
You can create workflows that build and test every commit to your repository, or deploy to production when a new tag is created.

To create a **GitHub Action**, you write a _workflow_ to be triggered when some _event_ occurs in your repository.
An example event is a commit to the main branch, creation of a tag, or you can manually run the workflow.

Here's a **GitHub Actions** workflow to build and test a .NET project:

```yaml
name: Build & Test 🧪

on:
  push:
    branches:
      - main

env:
  DOTNET_VERSION: '7.0.x'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Install dependencies 📂
        run: dotnet restore WebApi

      - name: Build 🧱
        run: dotnet build WebApi --configuration Release --no-restore

      - name: Test 🧪
        run: dotnet test WebApi --configuration Release --no-build
```

Let's unwrap what is happening here:

- Defining an event to trigger the workflow
- Setting up the **.NET SDK** with the version from `env.DOTNET_VERSION`
- Restoring, building and testing the project using the `dotnet` CLI tool

You can add this to your **GitHub** repository today, and start getting instant feedback when you commit code to the repository.

When a workflow run fails due to a build error or a failed test, you'll get an email notification.

## Continuous Delivery To Azure With GitHub Actions

**Continuous Integration** is a great way to start out with **CI/CD**, but the real value lies in **automating** your **deployment** process.

Imagine this:

- You make a change to your codebase
- The commit triggers the **deployment pipeline**
- A few minutes later your changes are live in production

Usually it's a little more nuanced, because we need to think about configuration, database migrations, etc.
But try to see the big picture here.

If you're running your application in the cloud, for example on **Azure**, chances are there's an existing **GitHub Action** you can use.

Here's a **deployment pipeline** I use to publish my application to an **Azure App Service** instance:

```yaml
name: Publish 🚀

on:
  push:
    branches:
      - main

env:
  AZURE_WEBAPP_NAME: web-api
  AZURE_WEBAPP_PACKAGE_PATH: './publish'
  DOTNET_VERSION: '7.0.x'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Build and Publish 📂
        run: |
          dotnet restore WebApi
          dotnet build WebApi -c Release --no-restore
          dotnet publish WebApi -c Release --no-build
            --output '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'

      - name: Deploy to Azure 🌌
        uses: azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          publish-profile: ${{ secrets.AZURE_PUBLISH_PROFILE }}
          package: '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'
```

This workflow is pretty similar to the previous one, with the differences being:

- Adding a publish step and configuring the output path
- Using the `azure/webapps-deploy@v2` action to deploy to **Azure**

If you need to safely and securely expose secret values in your workflows, you can use **GitHub secrets**.
You can define the secrets on GitHub, and use them in actions without having to add them to source control.

In the deployment workflow I'm using `secrets.AZURE_PUBLISH_PROFILE` to access my publish profile for the App Service instance.

## In Summary

**Continuous Integration and Delivery** can transform your development process by increasing the speed at which you release changes.

Try adding up how much time you spend on deployments.
I'm pretty sure you'll be surprised by the time savings potential of automating them.

And the good part is you will typically set up your **build and deployment pipelines** once, and then continue benefiting from them for the lifetime of your project.

If you want a step by step guide, I made a video showing [**how to implement a CI/CD pipeline from scratch.**](https://youtu.be/QP0pi7xe24s)
And the **source code** is also public, so you can add the **GitHub Actions** workflow file to your project.

Thanks for reading.

Hope that was helpful!

---



<Sponsorships>

Today's issue is sponsored by [**Refactoring.**](https://refactoring.club/?utm_source=milan)
Refactoring is a weekly newsletter [**about making great software and working well with humans.**](https://refactoring.club/?utm_source=milan)
It is read every week by 35,000+ developers, managers and leaders at companies like Google, Meta, and Amazon.
Check it out [**here.**](https://refactoring.club/?utm_source=milan)

And by [**Jam.**](https://jam.dev/)
Reduce time spent on bug reporting with [**Jam**](https://jam.dev/) by 60-80%.
Auto-capture dev logs & network requests in one link, access Instant Replays for hard-to-repro bugs and integrate with Jira, Slack & more.
Free Team Tier for the rest of 2023.
[**Get Jam Now.**](https://jam.dev/)

</Sponsorships>

Most software applications today are built around the concept of **multi-tenancy**.

One application serves multiple customers, while keeping their data **isolated**.

You can approach **multi-tenancy** in two ways:

- **Single database** and **logical isolation** of tenants
- **Multiple databases** and **physical isolation** of tenants

Which option you decide to use will depend mostly on your requirements.
Some industries like healthcare require a high degree of **data isolation**, and using a **database per tenant** is a must.

So how are we going to **implement multi-tenancy** support with **EF Core**?

We can use **Query Filters** to apply a **tenant filter** to all database queries.

Implement it once and you can almost forget about it.

Let's see what are some of the problems we need to solve.

## How To Use EF Core Query Filters

If you want an in-depth dive into **Query filters**, take a look at the newsletter issue where I talked about
[**using query filters with EF Core.**](how-to-use-global-query-filters-in-ef-core)

Here's a quick refresher on **Query filters**:

- Configure the **query filter** by calling `HasQueryFilter` for your entity
- **EF** will apply it to all queries for that entity
- You can turn it off with `IgnoreQueryFilters`
- Only **one** query filter **per entity** is allowed

And here's a simple example:

```csharp
modelBuilder
   .Entity<Order>()
   .HasQueryFilter(order => !order.IsDeleted);
```

All queries to the `Order` table will include an `IsDeleted = FALSE` condition.

## Single Database Multi-Tenancy With EF Core

You will need two things to **implement multi-tenancy** on a **single database**:

- A way to know **who** the **current tenant** is
- A way to **filter** the **data** for that **tenant** only

The typical approach for **multi-tenancy** on a **single database** is having a `TenantId` column in your tables.
And then filtering on that column when querying the database.

You can use the **Query filters** feature in **EF Core** to apply a **global filter** for some entity.

Inside of the `OnModelCreating` method we configure the query filter on the `Order` entity:

```csharp
public class OrdersDbContext : DbContext
{
    private readonly string _tenantId;

    public OrdersDbContext(
        DbContextOptions<OrdersDbContext> options,
        TenantProvider tenantProvider)
        : base(options)
    {
        _tenantId = tenantProvider.TenantId;
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder
            .Entity<Order>
            .HasQueryFilter(o => o.TenantId == _tenantId);
    }
}
```

We're using the `TenantProvider` class to get the **current tenant** value.

Here's what the `TenantProvider` implementation looks like:

```csharp
public sealed class TenantProvider
{
    private const string TenantIdHeaderName = "X-TenantId";

    private readonly IHttpContextAccessor _httpContextAccessor;

    public TenantProvider(IHttpContextAccessor httpContextAccessor)
    {
        _httpContextAccessor = httpContextAccessor;
    }

    public string TenantId => _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];
}
```

The `TenantId` is coming from the HTTP request header in this example.

A few other options to get the `TenantId` are:

- Query string - `api/orders?tenantId=example-tenant-id`
- JWT Claim
- API Key

If you want a more **secure implementation** you should go with **JWT Claims** or **API Keys** to provide the `TenantId` value.

## Separate Databases Multi-Tenancy With EF Core

What if we want to **isolate** each tenant to a **separate database**?

Here are the changes we need to make:

- Applying different **connection string per tenant**
- Resolving the connection string for each tenant _somehow_

You can't use **Query filters** here, since we are working with different databases.

So you will need to store the **tenant information** and **connection strings** somewhere.

A simple example would be store them in the **application settings**:

```json
"Tenants": {
    { "Id": "tenant-1", "ConnectionString": "Host=tenant1.db;Database=tenant1" },
    { "Id": "tenant-2", "ConnectionString": "Host=tenant2.db;Database=tenant2" }
}
```

You can then register an `IOptions` instance with a list of `Tenant` objects.

And we need to slightly modify the `TenantProvider` class to return a **connection string** for the current tenant:

```csharp
public sealed class TenantProvider
{
    private const string TenantIdHeaderName = "X-TenantId";

    private readonly IHttpContextAccessor _httpContextAccessor;
    private readonly TenantSettings _tenantSettings;

    public TenantProvider(
        IHttpContextAccessor httpContextAccessor,
        IOptions<TenantSettings> tenantsOptions)
    {
        _httpContextAccessor = httpContextAccessor;
        _tenants = tenantsOptions.Value;
    }

    public string TenantId => _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];

    public string GetConnectionString()
    {
        return _tenantSettings.Tenants.Single(t => t.Id == TenantId);
    }
}
```

And the last part is registering your `DbContext` to **dynamically resolve** the **connection string** for the current **tenant**.

```csharp
builder.Services.AddDbContext<OrdersDbContext>((sp, o) =>
{
    var tenantProvider = sp.GetRequiredService<TenantProvider>();

    var connectionString = tenantProvider.GetConnectionString();

    o.UseSqlServer(connectionString);
});
```

On every request, we create a new `OrdersDbContext` and connect to the appropriate database for that tenant.

You should definitely consider storing the tenant **connection strings** in a **secure** place like **Azure Key Vault**.

## Closing Thoughts

I hope you now have a better understanding of how to build a **multi-tenant system** with **EF Core**.

I showed you the bare bones implementation, which you can improve to make it more robust and secure.

Building **multi-tenant systems** isn't easy, but when you understand the basic principles it shouldn't be too difficult either.

That's all for this week.

See you next Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**IcePanel**](https://u.icepanel.io/1eec0e6f).
IcePanel is a collaborative [**C4 model modelling & diagramming tool**](https://u.icepanel.io/1eec0e6f) that helps explain complex software systems.
With an interactive map, you can align your software engineering & product teams on technical decisions across the business.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

**Software architecture** diagrams are a great way to communicate how you are planning to build a software system or how an existing software system works.

However, the majority of software architecture **diagrams** I've seen are a total mess.

If only there was a standard way to visualize your software architecture...

Enter the **C4 model**, which stands for **context**, **containers**, **components**, and **code**.

The **C4 model** is a lightweight approach to describing your **software architecture**.

And it's essentially just two things, a predefined set of common **abstractions** and four diagram types:

- **Context diagram** - high-level view of the system
- **Container diagram** - shows the objects running inside a system
- **Component diagram** - shows the building blocks that make each container
- **Code diagram** - rarely used, typically UML or ER diagram

Let's see how we can use the **C4 model** to visualize our **software architecture**.

## First You Need Abstractions

Before we dive into the **C4 diagrams**, we first need to understand the ubiquitous language of the **C4 model**.

> A software system is made up of one or more containers (applications and data stores),
> each of which contains one or more components, which in turn are implemented by one or more code elements (classes, interfaces, objects, functions, etc).
> And people may use the software systems that we build.

The **C4 model** defines a set of **abstractions** that you can use to describe your software systems:

- Person - the end user using your system
- Software system - highest level of abstraction that delivers value to end users
- Container - applications and data stores that make up a system
- Component - building blocks/modules that make up the container

With these high-level concepts in place, let's take a look at how to use them in our **C4 diagrams**.

## System Context Diagram

The **System Context diagram** is a high-level view of your software system.

It shows your software system as the central part, and any external systems and users that your system interacts with.

It should be **technology agnostic**, and the focus on the people and software systems instead of low-level details.

<Image
  src="blogs/mnw_037/system_context_diagram.jpg"
  width={2116}
  height={1172}
/>

The intended audience for the **System Context Diagram** is _everybody_.

If you can show it to non-technical people and they are able to understand it, then you know you're on the right track.

## Container Diagram

When you zoom into one software system, you get to the **Container diagram**.

Your software system is comprised of multiple running parts - **containers**.

A **container** can be a:

- Web application
- Single-page application
- Database
- File system
- Object store
- Message broker

You can look at a **container** as a **deployment unit** that executes code or stores data.

The **Container diagram** shows the high-level view of the software architecture and the **major technology choices**.

<Image src="blogs/mnw_037/container_diagram.jpg" width={1972} height={1732} />

The **Container diagram** is intended for technical people inside and outside of the software development team:

- Software architects
- Developers
- Operations/support staff

## Component Diagram

Next you can zoom into an individual **container** to decompose it into its building blocks.

The **Component diagram** show the individual **components** that make up a **container**:

- What each of the components are
- The technology and implementation details

<Image src="blogs/mnw_037/component_diagram.jpg" width={2100} height={1524} />

The **Component diagram** is intended for software architects and developers.

## Code Diagram

Finally, you can zoom into each **component** to show how it is implemented with **code**, typically using a UML class diagram or an ER diagram.

This level is rarely used as it goes into too much technical detail for most use cases.

However, there are supplementary diagrams that can be useful to fill in missing information by showcasing:

- Sequence of events
- Deployment information
- How systems interact at a higher level

It's only recommended for the most **important or complex components**.

Of course, the target audience are software architects and developers.

## Takeaway

The **C4 model** is a tool for visual and verbal communication, enabling your team to talk in the same language and have a common understanding of the design.

It's valuable for many reasons, to name a few:

- Product and business people benefit from understanding user behavior at the Context level
- Context and Container diagrams provide a simple view of how software systems work
- New developers can quickly understand system flows

Now that you understand what the **C4 model** is, there's one question left to answer.

Should you use the **C4 model** to express your **software architecture**?

I can't give you a definitive answer.

I used it with a lot of success.

But I think you should at least give it a try.

You can learn more about the **C4 model** [here.](https://c4model.com/)

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

**Software architecture** is a blueprint for how you should structure your system.
You can follow this blueprint strictly, or you can give yourself varying levels of freedom.

But when deadlines are tight, and you start cutting corners, that beautiful software architecture you built crumbles like a house of cards.

How can you **enforce** your **software architecture**?

By writing **architecture tests**.

**Architecture tests** are automated tests that verify the structure and design of your code.

You can use them to enforce your software architecture and the direction of dependencies of your projects.

In this week's issue I'll explain how to:

- Write architecture tests
- Enforce architecture
- Enforce design rules

Let's dive in!

## Writing Architecture Tests

You write **architecture tests** the same as any unit test in your application.
There's an excellent library for writing architecture tests that already implements the boilerplate code we need to start writing tests.

We're going to use the `NetArchTest.Rules` library for writing architecture tests.

First, you have to install the NuGet package:

```powershell
Install-Package NetArchTest.Rules
```

And now you can use it to write rules in your test project.

The starting point for writing architecture tests is the static `Types` class, which you can use to load a set of types.

Once you have loaded your types you can further filter them to find a more specific set of types.

Some of the available filtering methods:

- `ResideInNamespace`
- `AreClasses`
- `AreInterfaces`
- `HaveNameStartingWith`
- `HaveNameEndingWith`

Finally, when you are satisfied with your selection, you can write the rule you want to enforce by calling `Should` or `ShouldNot`
and applying the condition you want to check.

Here's an example checking that all classes in the domain assembly are sealed:

```csharp
var result = Types
  .InAssembly(DomainAssembly)
  .That()
  .AreClasses()
  .Should()
  .BeSealed()
  .GetResult();

Assert.True(result.IsSuccessful);
```

## Enforcing Architecture Rules

**Architecture tests** are particularly useful to **enforce software architecture rules** in a layered architecture or **Modular Monolith**.

Let's take the example of the Clean architecture:

- Domain should not have any dependencies
- Application should not depend on Infrastructure
- Infrastructure should depend on Application and Domain

Here's how you can write tests for enforcing architecture rules.

**Domain should not have any dependencies**

```csharp
var result = Types
  .InAssembly(DomainAssembly)
  .ShouldNot()
  .HaveDependencyOnAny("Application", "Infrastructure")
  .GetResult();

Assert.True(result.IsSuccessful);
```

**Application should not depend on Infrastructure**

```csharp
var result = Types
  .InAssembly(AplicationAssembly)
  .Should()
  .NotHaveDependencyOn("Infrastructure")
  .GetResult();

Assert.True(result.IsSuccessful);
```

**Infrastructure should depend on Application and Domain**

How the `NetArchTest.Rules` library works is by scanning the imported namespaces of your types.

Because of this, writing negative conditions like in the previous two examples is straightforward.

But writing positive conditions has to be scoped to a more specific set of types.

For example, we can validate this dependency by checking that all repositories must have a dependency on the `Domain` namespace.

```csharp
var result = Types
  .InAssembly(InfrastructureAssembly)
  .HaveNameEndingWith("Repository")
  .Should()
  .HaveDependencyOn("Domain")
  .GetResult();

Assert.True(result.IsSuccessful);
```

## Enforcing Design Rules

Another valuable **use case** for **architecture tests** is **enforcing design rules** in your application.

Design rules are more specific than project references, and focus on the implementation details of your classes.

Here are some **design rules** that you can enforce:

- Services must be internal
- Entities and Value objects must be sealed
- Controllers can't depend on repositories directly
- Command (or query) handlers must follow a naming convention

The possibilities are endless, and it's up to you how many design rules you want to enforce.

Here's how you can write tests for enforcing design rules.

**Command handlers must end with `CommandHandler`**

```csharp
var result = Types
    .InAssembly(ApplicationAssembly)
    .That()
    .ImplementInterface(typeof(ICommandHandler))
    .Should()
    .HaveNameEndingWith("CommandHandler")
    .GetResult();

Assert.True(result.IsSuccessful);
```

**Controllers can't directly reference repositories**

```csharp
var result = Types
    .InAssembly(ApiAssembly)
    .That()
    .HaveNameEndingWith("Controller")
    .ShouldNot()
    .HaveDependencyOn("Infrastructure.Repositories")
    .GetResult();

Assert.True(result.IsSuccessful);
```

## Takeaway

**Architecture tests** are an easy way to **enforce software architecture** and design rules with automated tests.

One of the best investments you can make as a software engineer is writing automated tests.
You write the tests once, and use them to verify your system forever.
Granted, you also have to maintain the tests over time as your system grows.

Manually enforcing software architecture with pair programming and constant PR reviews is:

- Error prone
- Time consuming
- Not cost effective

**Architecture tests** really shine here, since you can write them quickly and reduce the cost of enforcing your software architecture rules to zero.

Thanks for reading.

Hope that was helpful!

---



<Sponsorships>

Today's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

We all want to build **robust** and **reliable** applications that can scale indefinitely and handle any number of requests.

But with **distributed systems** and **microservices architectures** growing in complexity, it's becoming increasingly harder to **monitor** the **health** of our applications.

It's vital that you have a system in place to receive quick feedback of your application **health.**

That's where **health checks** come in.

**Health checks** provide a way to monitor and verify the health of various components of an application including:

- Databases
- APIs
- Caches
- External services

Here's what I'll show you in this week's newsletter:

- [What are health checks](#what-are-health-checks)
- [Adding a custom health check](#adding-custom-health-checks)
- [Using existing health check libraries](#using-existing-health-check-libraries)
- [Customizing the health checks response format](#formatting-health-checks-response)

Let's see how to implement **health checks** in **ASP.NET Core**.

## What Are Health Checks?

**Health checks** are a proactive mechanism for monitoring and verifying the **health** and **availability** of an application in **ASP.NET Core.**

ASP.NET Core has **built-in support** for implementing **health checks.**

Here's the basic configuration, which registers the health check services and adds the `HealthCheckMiddleware` to respond at the specified URL.

```csharp {3,7}
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHealthChecks();

var app = builder.Build();

app.MapHealthChecks("/health");

app.Run();
```

The health check returns a `HealthStatus` value indicating the health of the service.

There are three distinct `HealthStatus` values:

- `HealthStatus.Healthy`
- `HealthStatus.Degraded`
- `HealthStatus.Unhealthy`

You can use the `HealthStatus` to indicate the different states of your application.

For example, if the application is functioning slower than expected you can return `HealthStatus.Degraded`.

## Adding Custom Health Checks

You can create **custom health checks** by implementing the `IHealthCheck` interface.

For example, you can implement a check to see if your **SQL** database is available.

It's important to use a query that can complete quickly in the database, like `SELECT 1`.

Here's a **custom health check** implementation example in the `SqlHealthCheck` class:

```csharp
public class SqlHealthCheck : IHealthCheck
{
    private readonly string _connectionString;

    public SqlHealthCheck(IConfiguration configuration)
    {
        _connectionString = configuration.GetConnectionString("Database");
    }

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        try
        {
            using var sqlConnection = new SqlConnection(_connectionString);

            await sqlConnection.OpenAsync(cancellationToken);

            using var command = sqlConnection.CreateCommand();
            command.CommandText = "SELECT 1";

            await command.ExecuteScalarAsync(cancellationToken);

            return HealthCheckResult.Healthy();
        }
        catch(Exception ex)
        {
            return HealthCheckResult.Unhealthy(
                context.Registration.FailureStatus,
                exception: ex);
        }
    }
}
```

After you implement the **custom health check**, you need to register it.

The previous call to `AddHealthChecks` now becomes:

```csharp {2}
builder.Services.AddHealthChecks()
    .AddCheck<SqlHealthCheck>("custom-sql", HealthStatus.Unhealthy);
```

We're giving it a custom name and setting which status to use as the failure result in `HealthCheckContext.Registration.FailureStatus`.

But stop and think for a moment.

Do you want to implement a **custom health check** on your own for **every external service** that you have?

Of course not! There's a better solution.

## Using Existing Health Check Libraries

Before you start implementing a custom **health check** for everything, you should first see if there's already an **existing library.**

In the [`AspNetCore.Diagnostics.HealthChecks`](https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks) repository you can find
a wide collection **health check** packages for frequently used services and libraries.

Here are just a few examples:

- SQL Server - `AspNetCore.HealthChecks.SqlServer`
- Postgres - `AspNetCore.HealthChecks.Npgsql`
- Redis - `AspNetCore.HealthChecks.Redis`
- RabbitMQ - `AspNetCore.HealthChecks.RabbitMQ`
- AWS S3 - `AspNetCore.HealthChecks.Aws.S3`
- SignalR - `AspNetCore.HealthChecks.SignalR`

Here's how to add health checks for **PostgreSQL** and **RabbitMQ**:

```csharp {3,4}
builder.Services.AddHealthChecks()
    .AddCheck<SqlHealthCheck>("custom-sql", HealthStatus.Unhealthy);
    .AddNpgSql(pgConnectionString)
    .AddRabbitMQ(rabbitConnectionString)
```

## Formatting Health Checks Response

By default, the endpoint returning you **health check** status will return a string value representing a `HealthStatus`.

This isn't practical if you have **multiple health checks** configured, as you'd want to view the health status individually per service.

To make matters worse, if one of the services is failing the entire response will return `Unhealthy` and you don't know what's causing the issue.

You can solve this by providing a `ResponsWriter`, and there's an existing one in the `AspNetCore.HealthChecks.UI.Client` library.

Let's install the **NuGet** package:

```powershell
Install-Package AspNetCore.HealthChecks.UI.Client
```

And you need to slightly update the call to `MapHealthChecks` to use the `ResponseWriter` coming from this library:

```csharp {3-6}
app.MapHealthChecks(
    "/health",
    new HealthCheckOptions
    {
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });
```

After making these changes, here's what the response from the health check endpoint looks like:

```json
{
  "status": "Unhealthy",
  "totalDuration": "00:00:00.3285211",
  "entries": {
    "npgsql": {
      "data": {},
      "duration": "00:00:00.1183517",
      "status": "Healthy",
      "tags": []
    },
    "rabbitmq": {
      "data": {},
      "duration": "00:00:00.1189561",
      "status": "Healthy",
      "tags": []
    },
    "custom-sql": {
      "data": {},
      "description": "Unable to connect to the database.",
      "duration": "00:00:00.2431813",
      "exception": "Unable to connect to the database.",
      "status": "Unhealthy",
      "tags": []
    }
  }
}
```

## Takeaway

Application monitoring is important to track availability, resource usage, and changes to performance in your application.

I've used **health checks** before to implement **failover scenarios** in a **cloud deployment**.
When one application instance stops responding with a healthy result, a new one is created to continue serving requests.

It's easy to monitor the health of your ASP.NET Core applications by **exposing health checks** for your services.

You can decide to implement **custom health checks**, but first consider if there are **existing solutions**.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

What happens when a **message is retried** in an event-driven system?

It happens more often than you think.

The **worst case scenario** is that the **message is processed twice**, and the **side effects** can also be applied more than once.

Do you want your bank account to be double charged?

I'll assume the answer is no, of course.

You can use the **Idempotent Consumer** pattern to solve this problem.

In this week's issue I will show you:

- How the Idempotent Consumer pattern works
- How to implement an Idempotent Consumer
- The tradeoffs you need to consider

Let's see why the **Idempotent Consumer** pattern is valuable.

## How The Idempotent Consumer Pattern Works

What's the idea behind the **Idempotent Consumer pattern**?

> An idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.

We want to avoid handling the same message more than once.

This would require **Exactly-once** message delivery guarantees from our messaging system.
And this is a really hard problem to solve in distributed systems.

A looser delivery guarantee is **At-least-once**, where we are aware that retries can happen and we can receive the same message more than once.

The **Idempotent Consumer** pattern works well with **At-least-once** message delivery, and solves the **problem of duplicate messages.**

Here's what the algorithm looks like from the moment we receive a message:

1. Was the message already processed?
2. If yes, it's a duplicate and there's nothing to do
3. If not, we need to handle the message
4. We also need to store the message identifier

<Image
  src="blogs/mnw_034/idempotent_consumer_algorithm.png"
  width={400}
  height={634}
/>

We need a **unique identifier** for every **message** we receive, and a table in the database to store processed messages.

However, it's interesting how we choose the implement the message handling and storing of the processed message identifier.

You can implement the idempotent consumer as a decorator around a regular message handler.

I'll show you two implementations:

- Lazy idempotent consumer
- Eager idempotent consumer

## Lazy Idempotent Consumer

The **lazy idempotent consumer** matches the flow shown in the algorithm above.

Lazy refers to how we store the message identifer to mark it as processed.

In the happy path, we handle the message and store the message identifier.

If the message handler throws an exception, we never store the message identifier and the consumer can be executed again.

Here's what the implementation looks like:

```csharp
public class IdempotentConsumer<T> : IHandleMessages<T>
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages<T> _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages<T> decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        if (_messageRepository.IsProcessed(message.Id))
        {
            return;
        }

        await _decorated.Handle(message);

        _messageRepository.Store(message.Id);
    }
}
```

## Eager Idempotent Consumer

The **eager idempotent consumer** is slightly different from the lazy implementation, but the end result is the same.

In this version, we eagerly store the message identifier in the database and then continue to handle the message.

If the handler throws an exception, we need to perform cleanup in the database and remove the eagerly stored message identifier.

Otherwise, we risk leaving the system in an inconsistent state since the message was never handled correctly.

Here's what the implementation looks like:

```csharp
public class IdempotentConsumer<T> : IHandleMessages<T>
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages<T> _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages<T> decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        try
        {
            if (_messageRepository.IsProcessed(message.Id))
            {
                return;
            }

            _messageRepository.Store(message.Id);

            await _decorated.Handle(message);
        }
        catch (Exception e)
        {
            _messageRepository.Remove(message.Id);

            throw;
        }
    }
}
```

## In Summary

Idempotency is an interesting problem to solve in a software system.

Some operations are **naturally idempotent**, and we don't need the overhead of the **Idempotent Consumer** pattern.

However, for those operations that aren't naturally idempotent, the **Idempotent Consumer** is a great solution.

The high-level algorithm is simple, and you can take two approaches in the implementation:

- Lazy storing of message identifiers
- Eager storing of message identifiers

I prefer to use the **lazy approach**, and only **store the message identifier** in the database when the **handler completes successfully.**

It's easier to reason about and there is one less call to the database.

Thanks for reading.

Hope that was helpful.

---



<Sponsorships>

This week's issue is sponsored by the [**Why You Should Join Newsletter.**](https://whyyoushouldjoin.substack.com/)
[**Why You Should Join**](<(https://whyyoushouldjoin.substack.com/)>) is a free, monthly newsletter doing deep dives on the most promising startups to join.
They receive inside info from Sequoia, Index, and [**other top VCs**](https://whyyoushouldjoin.substack.com/about) about which companies are doing best and recommend just one each month.
Prior suggestions [**Warp**](https://whyyoushouldjoin.substack.com/p/warp) and [**Pinecone**](https://whyyoushouldjoin.substack.com/p/pinecone)
have since raised significant follow-on funding - [**subscribe for free**](https://whyyoushouldjoin.substack.com/) so you don't miss the next Uber or Airbnb.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

**EF Core** is getting many new and exciting features in the upcoming version.

**EF7** introduced support for returning **scalar types** using **SQL** queries.

And now we're getting support for **querying unmapped types** with **raw SQL queries** in **EF8.**

This is exactly what **Dapper** offers out of the box, and it's good to see **EF Core** catching up.

In this week's newsletter, I'm going to cover how to use **EF Core** for:

- [Raw SQL queries](#ef-core-and-sql-queries)
- [Composing SQL queries with LINQ](#composing-sql-queries-with-linq)
- [Executing data modifications with SQL](#sql-queries-for-data-modifications)

Let's dive in!

## EF Core And SQL Queries

**EF7** added support for **raw SQL queries** returning scalar types.
**EF8** is taking this a step further with raw SQL queries that can return any mappable type, without having to include it in the **EF model**.

You can query unmapped types with the `SqlQuery` and `SqlQueryRaw` methods.

The `SqlQuery` method uses string interpolation to parameterize the query, protecting against SQL injection attacks.

Here's an example query returning an `OrderSummary` list:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>(
        $"SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= {startDate}")
    .ToListAsync();
```

This will be the **SQL** sent to the database:

```sql
SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= @p0
```

The type used for the query result can have a parameterized constructor.
The property names don't need to match the column names in the database, but they do have to match the names of the values in the result set.

You can also execute raw SQL queries and return results with:

- Views
- Functions
- Stored procedures

## Composing SQL Queries With LINQ

An interesting thing about `SqlQuery` is that it returns `IQueryable`, which can be further composed with **LINQ.**

You can add a `Where` statement after calling `SqlQuery`:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>("SELECT * FROM OrderSummaries AS o")
    .Where(o => o.CreatedOn >= startDate)
    .ToListAsync();
```

However, the generated **SQL** isn't optimal:

```sql
SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o
) AS s
WHERE s.CreatedOn >= @p0
```

Another possibility is to combine an `OrderBy` statement with `Skip` and `Take`:

```csharp
var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery<OrderSummary>(
        $"SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= {startDate}")
    .OrderBy(o => o.Id)
    .Skip(10)
    .Take(5)
    .ToListAsync();
```

This would be the generated **SQL** for the previous query:

```sql
SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn >= @p0
) AS s
ORDER BY s.Id
OFFSET @p1 ROWS FETCH NEXT @p2 ROWS ONLY
```

In case you're wondering, the performance is similar to **LINQ** queries using `Select` projections.

I ran some benchmarks, and didn't notice any significant performance improvement.

This feature will be very useful if you're more comfortable with writing **SQL** or you want to fetch data from views, functions, and stored procedures.

## SQL Queries For Data Modifications

If you want to modify data in the database with **SQL**, you will typically write a query that doesn't return a result.

The SQL query can be an `UPDATE` or `DELETE` statement, or even a stored procedure call.

You can use the `ExecuteSql` method to execute this type of query with **EF Core**:

```csharp
var startDate = new DateOnly(2023, 1, 1);

dbContext.Database.ExecuteSql(
    $"UPDATE Orders SET Status = 5 WHERE CreatedOn >= {startDate}");
```

`ExecuteSql` also protects from SQL injection by parameterizing arguments, just like `SqlQuery`.

With **EF7** you can write the above query with **LINQ** and the `ExecuteUpdate` method.
There's also the `ExecuteDelete` method for deleting records.

## In Summary

**EF7** introduced support for raw SQL queries returning **scalar** values.

**EF8** will add support for **raw SQL queries** returning **unmapped types** with `SqlQuery` and `SqlQueryRaw`.

I like the direction that **EF** is going, introducing more flexibility for querying the database.

The performance isn't as good as **Dapper**, unfortunately.
But it's close enough that network costs will play the bigger factor.

I will probably be using only **EF** moving forward since it covers more use cases.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

</Sponsorships>

Rate limiting is a technique to limit the number of requests to a server or an API.

A limit is introduced within a given time period to prevent server overload and protect against abuse.

In ASP.NET Core 7, we have a built-in rate limiter middleware that's easy to integrate into your API.

We're going to cover four rate limiter algorithms:

- [Fixed window](#fixed-window-limiter)
- [Sliding window](#sliding-window-limiter)
- [Token bucket](#token-bucket-limiter)
- [Concurrency](#concurrency-limiter)

Let's see how we can work with rate limiting.

## What Is Rate Limiting?

Rate limiting is about restricting the number of requests to an API, usually within a specific time window or based on other criteria.

This is practical for a few reasons:

- Prevents overloading of servers or applications
- Improves security and guards against DDoS attacks
- Reduces costs by preventing unnecessary resource usage

In a multi-tenant application, each unique user can have a limitation on the number of API requests.

## Configuring Rate Limiting

ASP.NET Core 7 introduced built-in rate limiting middleware in the `Microsoft.AspNetCore.RateLimiting` namespace.

To add rate limiting to your application, you first need to register the rate limiting services:

```csharp
builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    // We'll talk about adding specific rate limiting policies later.
});
```

I suggest updating the `RejectionStatusCode` to `429 (Too Many Requests)` because it's more correct.
The default value is `503 (Service Unavailable)`.

And you also have to apply the `RateLimitingMiddleware`:

```csharp
app.UseRateLimiter();
```

That's everything you'll need.

Let's see the rate limiting algorithms we can use.

## Fixed Window Limiter

The `AddFixedWindowLimiter` method configures a fixed window limiter.

The `Window` value determines the time window.

When a time window expires, a new one starts, and the request limit is reset.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddFixedWindowLimiter("fixed", options =>
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

## Sliding Window Limiter

The sliding window algorithm is similar to the fixed window, but it introduces segments in a window.

Here's how it works:

- Each time window is divided into multiple segments
- The window slides one segment each segment interval
- The segment interval is (window_time)/(segments_per_window)
- When a segment expires, the requests taken in that segment are added to the current segment

The `AddSlidingWindowLimiter` method configures a sliding window limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddSlidingWindowLimiter("sliding", options =>
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.SegmentsPerWindow = 2;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

## Token Bucket Limiter

The token bucket algorithm is similar to the sliding window, but instead of adding back the requests from the expired segment,
a fixed number of tokens are added after each replenishment period.

The total number of tokens can never exceed the token limit.

The `AddTokenBucketLimiter` method configures a token bucket limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddTokenBucketLimiter("token", options =>
    {
        options.TokenLimit = 100;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
        options.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
        options.TokensPerPeriod = 20;
        options.AutoReplenishment = true;
    });
});
```

When `AutoReplenishment` is `true`, an internal timer will execute every `ReplenishmentPeriod` and replenish the tokens.

## Concurrency Limiter

The concurrency limiter is the most straightforward algorithm, and it just limits the number of concurrent requests.

The `AddConcurrencyLimiter` method configures a concurrency limiter.

```csharp
builder.Services.AddRateLimiter(rateLimiterOptions =>
{
    rateLimiterOptions.AddConcurrencyLimiter("concurrency", options =>
    {
        options.PermitLimit = 10;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
```

There's no time component involved in this case. The only parameter is the number of concurrent requests.

## Using Rate Limiting In Your API

Now that we have configured our rate limiting policies, let's see how we can use them in our API.

There are slight differences between controllers and minimal API endpoints, so I'll cover them in separate examples.

**Controllers**

To add rate limiting on a controller we use the `EnableRateLimiting` and `DisableRateLimiting` attributes.

`EnableRateLimiting` can be applied on the controller or on the individual endpoints.

```csharp
[EnableRateLimiting("fixed")]
public class TransactionsController
{
    private readonly ISender _sender;

    public TransactionsController(ISender sender)
    {
        _sender = sender;
    }

    [EnableRateLimiting("sliding")]
    public async Task<IActionResult> GetTransactions()
    {
        return Ok(await _sender.Send(new GetTransactionsQuery()));
    }

    [DisableRateLimiting]
    public async Task<IActionResult> GetTransactionById(int id)
    {
        return Ok(await _sender.Send(new GetTransactionByIdQuery(id)));
    }
}
```

In the previous example:

- All endpoints in the `TransactionsController` will use a **fixed window** policy
- The `GetTransactions` endpoint will use a **sliding window** policy
- The `GetTransactionById` endpoint won't have any rate limiting applied

**Minimal APIs**

In a Minimal API endpoint you can configure the rate limit policy by calling `RequireRateLimiting` and specifying the policy name.

We're using the **token bucket** policy in this example.

```csharp
app.MapGet("/transactions", async (ISender sender) =>
{
    return Results.Ok(await sender.Send(new GetTransactionsQuery()));
})
.RequireRateLimiting("token");
```

## Closing Thoughts

It's great that we can quickly introduce **rate limiting** in ASP.NET Core.

You can choose from one of the existing rate limiter algorithms:

- Fixed window
- Sliding window
- Token bucket
- Concurrency

Here are some resources if you want to learn more about rate limiting:

- [Rate Limiting pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/rate-limiting-pattern)
- [Announcing Rate Limiting for .NET](https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/)

I'm excited to try out rate limiting in my projects.

That's all for today.

Have an awesome Saturday!

---



<Sponsorships>

This week's issue is sponsored by [**QueueExplorer.**](https://www.cogin.com/mq/)
QueueExplorer is an explorer-like [**management console for RabbitMQ, Azure Service Bus, MSMQ, and ActiveMQ.**](https://www.cogin.com/mq/)
With QueueExplorer you can see what happens in your queues, edit and fix problematic messages, extract business data from JSON or XML payloads,
or send thousands of messages for stress testing.

And by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus is a free .NET "service bus", and [**Rebus Pro is the perfect one-up for serious Rebus users.**](https://pro.rebus.fm)
Use Fleet Manager to get Slack alerts when something fails and retry dead-lettered messages with a click of the mouse.

</Sponsorships>

Designing long-lived processes in a distributed environment is an interesting engineering challenge.

And a well known pattern for solving this problem is a [**Saga**](https://microservices.io/patterns/data/saga.html).

A **Saga** is a sequence of local transactions, where each local transaction updates the **Saga** state and publishes a message triggering the next step in the **Saga**.

Sagas come in two forms:

- **Orchestrated**
- **Choreographed**

With an orchestrated Saga, there's a central component responsible for orchestrating the individual steps.

In a choreographed Saga, processes work independently but coordinate with each other using events.

In this week's issue, I'll show you how to create an **orchestrated Saga** using the **Rebus** library with **RabbitMQ** for message transport.

Let's dive in.

## Rebus Configuration

[Rebus](https://github.com/rebus-org/Rebus) is a free .NET “service bus”, and it's practical for implementing asynchronous messaging-based
communication between the components of an application.

Let's install the following libraries:

- `Rebus.ServiceProvider` for managing the **Rebus** instance
- `Rebus.RabbitMq` for **RabbitMQ** message transport
- `Rebus.SqlServer` for **SQL Server** state persistence

```powershell
Install-Package Rebus.ServiceProvider -Version 8.4.0
Install-Package Rebus.RabbitMq -Version 8.0.0
Install-Package Rebus.SqlServer -Version 7.3.1
```

Inside of your **ASP.NET Core** application you will need the following configuration.

```csharp
services.AddRebus(
    rebus => rebus
        .Routing(r =>
            r.TypeBased().MapAssemblyOf<Program>("newsletter-queue"))
        .Transport(t =>
            t.UseRabbitMq(
                configuration.GetConnectionString("RabbitMq"),
                inputQueueName: "newsletter-queue"))
        .Sagas(s =>
            s.StoreInSqlServer(
                configuration.GetConnectionString("SqlServer"),
                dataTableName: "Sagas",
                indexTableName: "SagaIndexes"))
        .Timeouts(t =>
            t.StoreInSqlServer(
                builder.Configuration.GetConnectionString("SqlServer"),
                tableName: "Timeouts"))
);

services.AutoRegisterHandlersFromAssemblyOf<Program>();
```

Unpacking the individual configuration steps:

- `Routing` - Configures messages to be routed by their type
- `Transport` - Configures the message transport mechanism
- `Sagas` - Configures the Saga persistence store
- `Timeouts` - Configures the timeouts persistence store

You also need to specify the queue name for sending and receiving messages.

`AutoRegisterHandlersFromAssemblyOf` will scan the specified assembly and automatically register the respective message handlers.

## Creating The Saga With Rebus

We're going to create a **Saga** for a newsletter onboarding process.

When a user subscribes to the newsletter we want to:

- Send a welcome email immediately
- Send a follow-up email after 7 days

The first step in creating the **Saga** is defining the data model by implementing `ISagaData`.
We'll keep it simple and store the `Email` for **correlation**, and add two flags for the distinct steps in our **Saga**.

```csharp
public class NewsletterOnboardingSagaData : ISagaData
{
    public Guid Id { get; set; }
    public int Revision { get; set; }

    public string Email { get; set; }

    public bool WelcomeEmailSent { get; set; }

    public bool FollowUpEmailSent { get; set; }
}
```

Now we can define the `NewsletterOnboardingSaga` class by inheriting from the `Saga` class and implementing the `CorrelateMessages` method.

It's a best practice to use a unique value for correlation.
In our case this will be the `Email`.

You also configure how the `Saga` starts with `IAmInitiatedBy`, and the individual messages the `Saga` handles with `IHandleMessages`.

```csharp
public class NewsletterOnboardingSaga :
    Saga<NewsletterOnboardingSagaData>,
    IAmInitiatedBy<SubscribeToNewsletter>,
    IHandleMessages<WelcomeEmailSent>,
    IHandleMessages<FollowUpEmailSent>
{
    private readonly IBus _bus;

    public NewsletterOnboardingSaga(IBus bus)
    {
        _bus = bus;
    }

    protected override void CorrelateMessages(
        ICorrelationConfig<NewsletterOnboardingSagaData> config)
    {
        config.Correlate<SubscribeToNewsletter>(m => m.Email, d => d.Email);

        config.Correlate<WelcomeEmailSent>(m => m.Email, d => d.Email);

        config.Correlate<FollowUpEmailSent>(m => m.Email, d => d.Email);
    }

    /* Handlers omitted for brevity */
}
```

## Message Types And Naming Conventions

There are two types of messages you send in a **Saga**:

- Commands
- Events

Commands instruct the receiving component what to do.\
Think: **verb, imperative.**

Events notify the Saga which process was just completed.\
Think: **what happened, past tense.**

## Saga Orchestration With Messages

The `NewsletterOnboardingSaga` starts by handling the `SubscribeToNewsletter` command, and publishes a `SendWelcomeEmail` command.

```csharp
public async Task Handle(SubscribeToNewsletter message)
{
    if (!IsNew)
    {
        return;
    }

    await _bus.Send(new SendWelcomeEmail(message.Email));
}
```

The `SendWelcomeEmail` command is handled by a different component, which publishes a `WelcomeEmailSent` event when it completes.

In the `WelcomeEmailSent` handler we update the `Saga` state and publish a **deferred message** by calling `Defer`.
Rebus will persist the `SendFollowUpEmail` command, and publish it when the **timeout expires**.

```csharp
public async Task Handle(WelcomeEmailSent message)
{
    Data.WelcomeEmailSent = true;

    await _bus.Defer(TimeSpan.FromDays(3), new SendFollowUpEmail(message.Email));
}
```

Finally, the `SendFollowUpEmail` command is handled and we publish the `FollowUpEmailSent` event.

We update the `Saga` state again, and also call `MarkAsComplete` to complete the `Saga`.

```csharp
public Task Handle(FollowUpEmailSent message)
{
    Data.FollowUpEmailSent = true;

    MarkAsComplete();

    return Task.CompletedTask;
}
```

Completing the `Saga` will delete it from the database.

## Handling Commands With Rebus

Here's how the `SendWelcomeEmail` command handler looks like.

```csharp
public class SendWelcomeEmailHandler : IHandleMessages<SendWelcomeEmail>
{
    private readonly IEmailService _emailService;
    private readonly IBus _bus;

    public SendWelcomeEmailHandler(IEmailService emailService, IBus bus)
    {
        _emailService = emailService;
        _bus = bus;
    }

    public async Task Handle(SendWelcomeEmail message)
    {
        await _emailService.SendWelcomeEmailAsync(message.Email);

        await _bus.Reply(new WelcomeEmailSent(message.Email));
    }
}
```

The important thing to highlight here is that we're using the `Reply` method to send a message back.
This will reply back to the endpoint specified as the return address on the current message.

## In Summary

**Sagas** are practical for implementing a long-lived process in a distributed system.
Each business process is represented by a local transaction, and publishes a message to trigger the next step in the **Saga**.

Although **Sagas** are very powerful, they are also _complicated to develop, maintain and debug._

We didn't cover a few important topics in this newsletter:

- Error handling
- Message retries
- Compensating transactions

I think you'll have some fun researching these on your own.

Take a look at the [**source code for the example used in this newsletter**](https://github.com/m-jovanovic/newsletter-orchestrated-saga)
if you want to learn more about implementing Sagas.

If you have **Docker** installed, you should be able to run it without a problem and try it out.

Thank you for reading, and have an awesome Saturday.

---



<Sponsorships>

This week's issue is sponsored by [**Rebus Pro.**](https://pro.rebus.fm)
Rebus Pro is the perfect one-up for serious Rebus users - use Fleet Manager to get Slack alerts when something fails,
and [**retry dead-lettered messages with a click of the mouse.**](https://pro.rebus.fm)

And by [**Uno Platform**](https://platform.uno/docs/articles/get-started-vs-2022.html?utm_source=MilanJovanovic&utm_medium=newsletter&utm_campaign=GetStarted).
Uno Platfrom is a free and open-source development platform that allows you to build single codebase applications for Windows, iOS, Android, WebAssembly,
macOS, and Linux with C# and XAML.
[**Easily create applications that look and feel native**](https://platform.uno/docs/articles/get-started-vs-2022.html?utm_source=MilanJovanovic&utm_medium=newsletter&utm_campaign=GetStarted)
to each platform without having to rewrite your codebase from scratch.

</Sponsorships>

**MediatR** is a popular library with a simple **mediator pattern** implementation in .NET.

Here's a definiton taken from MediatR's GitHub: **"In-process messaging with no dependencies."**

With the rise in popularity of the **CQRS pattern**, MediatR became the go-to library to implement commands and queries.

However, MediatR also has support for the **publish-subscribe** pattern using notifications.
You can publish an `INotification` instance and have multiple subscribers handle the published message.

Until recently, the handlers subscribing to an `INotification` message could only execute serially, one by one.

In this week's newsletter, I'll show you how to configure MediatR to **execute the handlers in parallel**.

Let's dive in.

## How Publish-Subscribe Works With MediatR

Before I talk about notification publishing strategies, let's see how **publish-subscribe** works with **MediatR**.

You need a class implementing the `INotification` interface:

```csharp
public record OrderCreated(Guid OrderId) : INotification;
```

Then you need a respective `INotificationHandler` implementation:

```csharp
public class OrderCreatedHandler : INotificationHandler<OrderCreated>
{
    private readonly INotificationService _notificationService;

    public OrderCreatedHandler(INotificationService notificationService)
    {
        _notificationService = notificationService;
    }

    public async Task Handle(
        OrderCreated notification,
        CancellationToken cancellationToken)
    {
        await _notificationService.SendOrderCreatedEmail(
            notification.OrderId,
            cancellationToken);
    }
}
```

And then you simply publish a message using either `IMediator` or `IPublisher`.
I prefer using the `IPublisher` because it's more expressive:

```csharp
await publisher.Publish(new OrderCreated(order.Id), cancellationToken);
```

MediatR will invoke all the respective handlers.

## Introducing Notification Publisher Strategies

Before MediatR v12, the publishing strategy would invoke each handler individually.

However, there's a new interface `INotificationPublisher` controlling how the handlers are called.

The default implementation of this interface is `ForeachAwaitPublisher`:

```csharp
public class ForeachAwaitPublisher : INotificationPublisher
{
    public async Task Publish(
        IEnumerable<NotificationHandlerExecutor> handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        foreach (var handler in handlerExecutors)
        {
            await handler
                .HandlerCallback(notification, cancellationToken)
                .ConfigureAwait(false);
        }
    }
}
```

But now you can also use the `TaskWhenAllPublisher`:

```csharp
public class TaskWhenAllPublisher : INotificationPublisher
{
    public Task Publish(
        IEnumerable<NotificationHandlerExecutor> handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        var tasks = handlerExecutors
            .Select(handler => handler.HandlerCallback(
                notification,
                cancellationToken))
            .ToArray();

        return Task.WhenAll(tasks);
    }
}
```

Here's a comparison between these two strategies.

`ForeachAwaitPublisher`:

- Invokes each handler one by one
- Fails when an exception occurs in one of the handlers

`TaskWhenAllPublisher`:

- Invokes all the handlers at the same time
- Executes all the handlers regardless of one of them throwing an exception

If you store the task returned by `TaskWhenAllPublisher` you can access the `Task.Exception` property, which will contain an `AggregateException` instance.
You can then implement more robust exception handling.

## Configuring MediatR Notification Publishing Strategy

How do we configure which `INotificationPublisher` strategy MediatR will use?

There's a new way to apply configuration options when calling the `AddMediatR` method.

You supply an `Action<MediatRServiceConfiguration>` delegate and configure the `MediatRServiceConfiguration` instance.

If you want to use the `TaskWhenAllPublisher` strategy, you can either:

- Provide a value for the `NotificationPublisher` property
- Specify the strategy type on the `NotificationPublisherType` property

```csharp
services.AddMediatR(config => {
    config.RegisterServicesFromAssemblyContaining<Program>();

    // Setting the publisher directly will make the instance a Singleton.
    config.NotificationPublisher = new TaskWhenAllPublisher();

    // Seting the publisher type will:
    // 1. Override the value set on NotificationPublisher
    // 2. Use the service lifetime from the ServiceLifetime property below
    config.NotificationPublisherType = typeof(TaskWhenAllPublisher);

    config.ServiceLifetime = ServiceLifetime.Transient;
});
```

You can also implement a custom `INotificationPublisher` instance and use your own implementation instead.

## How Is This Useful?

Being able to **run notification handlers in parallel** provides a significant **performance improvement** over the default behavior.

However, note that all handlers will use the same service scope.

If you have service instances that don't support concurrent access you may run into problems.

Unfortunately, one such service instance is the **EF Core** `DbContext`.

In any case, I think this is a great addition to the already amazing **MediatR** library.

That's all for today.

See you next week.

---



<Sponsorships>

This week's issue is sponsored by [**Localizely.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)
Localizely is the translation management platform that helps you translate texts in your app for targeting multilingual markets.
With Localizely you can collaborate with your team on translations, and reduce translation errors with better context and app screenshots.
[**Learn more here.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)

And by [**QA Wolf.**](https://www.qawolf.com/lp/qa-as-a-service?utm_campaign=QAWolfGetsYouTo80Percent03042023&utm_source=milannetweekly&utm_medium=newsletter)
QA Wolf gets you to 80% automated end-to-end test coverage in 4 months - and keeps you there.
They will build, run and maintain your test suite so that you don't have to.
[**Chat with QA Wolf.**](https://www.qawolf.com/lp/qa-as-a-service?utm_campaign=QAWolfGetsYouTo80Percent03042023&utm_source=milannetweekly&utm_medium=newsletter)

</Sponsorships>

**Data-driven testing** is a testing method where test data is provided through some external source.
Hence it's also known as **parameterized testing**.

A popular testing library in .NET that supports parameterized testing is **xUnit**.
It uses attributes to define test methods.
The `Fact` attribute defines a simple test, and the `Theory` attribute defines a parameterized test.

In this week's newsletter, I'm going to show you four ways to write **parameterized tests** with xUnit:

- `InlineData`
- `MemberData`
- `ClassData`
- `TheoryData`

And I'll discuss which approach I think is the best.

Let's dive in.

## Writing Parameterized Tests With InlineData

The simplest way to write parameterized tests with xUnit is using the `InlineData` attribute.
You provide test data by passing in values to the `InlineData` constructor.

Here's how that would look like:

```csharp
[Theory]
[InlineData("test@test.com", "test.com")]
[InlineData("milan@milanjovanovic.tech", "milanjovanovic.tech")]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}
```

In this example, we provide two `string` values to the `InlineData` attribute, which represent the `email` and `expectedDomain` parameters in the test.
We can specify the `InlineData` attribute as many times as we want, to introduce more test cases.

The downside of this approach is that it becomes very verbose when we have many test cases.
And we are limited to only using constant data for the parameters.

## Writing Parameterized Tests With MemberData

With the `MemberData` attribute we have the ability to programmatically provide the test data.
You can load the test data from a static property or member of a type.

Here's an example of using the `MemberData` attribute to load test data from a property:

```csharp
[Theory]
[MemberData(nameof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public static IEnumerable<object[]> EmailTestData => new List<object>
{
    new object[] { "test@test.com", "test.com" },
    new object[] { "milan@milanjovanovic.tech", "milanjovanovic.tech" }
};
```

You specify the name of the member in the `MemberData` attribute, and it's a best practice to use the `nameof` operator
so that you can rename the property (or method) in the future without breaking your test.

The one constraint is that the property (or method) has to return `IEnumerable<object[]>`, so there is no strong typing.

## Writing Parameterized Tests With ClassData

The `ClassData` attribute allows you to extract test data into its own class.
This is helpful for organizing your test data separately from your tests, and it allows for easier reuse.
You load the test from a class the inherits from `IEnumerable<object[]>` and implements the `GetEnumerator` method.

Here's an example of using the `ClassData` attribute to load test data from a class:

```csharp
[Theory]
[ClassData(typeof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public class EmailTestData : IEnumerable<object[]>
{

    public IEnumerable<object[]> GetEnumerator()
    {
        yield return new object[] { "test@test.com", "test.com" };
        yield return new object[] { "milan@milanjovanovic.tech", "milanjovanovic.tech" };
    }

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
};
```

Unfortunately, this approach is complicated because you have to implement the `IEnumerable` interface.

It almost defeats the purpose of separating test data from the actual tests.

And we still suffer from lack of type-safety.

Is there a better solution?

## Writing Parameterized Tests With TheoryData

Let me introduce you to `TheoryData`, which is my preferred way of providing test data for parameterized tests.
Using the `TheoryData` class you can implement a class to provide test data while having the benefit of type-safety.

Here's an example of using `TheoryData` in combination with `ClassData`:

```csharp
[Theory]
[ClassData(typeof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public class EmailTestData : TheoryData<string, string>
{
    public EmailTestData()
    {
        Add("test@test.com", "test.com");
        Add("milan@milanjovanovic.tech", "milanjovanovic.tech");
    }
};
```

How does `TheoryData` work?

It's a generic class that allows us to specify the types for our parameterized test.

You just call the `Add` method in the `EmailTestData` constructor to provide test data for a single test case.
And introducing more test cases comes down to calling the `Add` method multiple times.

You can also use `TheoryData` in combination with `MemberData`, and return `TheoryData` from a property or method.

## Which Approach Should You Use?

I showed you four approaches to write parameterized tests with xUnit:

- `InlineData`
- `MemberData`
- `ClassData`
- `TheoryData`

So which one should you use?

Here's my personal preference that you can follow if you want:

- `InlineData` for simple test cases
- `TheoryData` using `ClassData` for complex test cases

Thank you for reading, and have a wonderful Saturday.

---



<Sponsorships>

Today's issue is sponsored by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs on new or existing SQL Server**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
enabling teams to ship apps faster.

And by [**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2),
the translation management platform that helps you
[**translate texts in your app for targeting multilingual markets.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website2)

</Sponsorships>

**Entity Framework Core (EF Core)** is a popular ORM in .NET that allows you to work with SQL databases.
EF Core uses a `DbContext`, which represents a session with the database and is responsible for tracking changes,
performing database operations, and managing database connections.

It's common to have only one `DbContext` for the entire application.

But what if you need to have **multiple DbContexts**?

In this week's newsletter we're going to explore:

- When you may want to use multiple DbContexts
- How to create multiple DbContexts
- What are the benefits of using multiple DbContexts

Let's dive in!

## Why Use Multiple DbContexts?

There are a few cases where using **multiple DbContexts can be useful.**

**Multiple Databases**\
Does your application need to work with multiple SQL databases?
Then you're forced to use multiple DbContexts, each one dedicated to a specific SQL database.

**Separating Concerns**\
If the application you're building has a complex domain model, you may see an improvement by separating concerns
between a few DbContexts, where each one is responsible for a specific area of the domain model.

**Modular Monolith**\
When you're building a **Modular Monolith**, using multiple DbContexts can be practical because you can configure
a different **database schema** per `DbContext`, giving you logical separation at the database level.

**Read Replicas**\
You can configure a separate `DbContext` instance to access the read replica of your database, and use that
`DbContext` for read-only queries.
You can also configure `QueryTrackingBehavior.NoTracking` on the `DbContext` level to improve query performance.

## Creating Multiple DbContexts In a Single Application

Here's how you can easily configure multiple DbContexts.
Let's say we have a `CatalogDbContext` and an `OrderDbContext` in our application.
We want to configure them using the following constrainsts:

- Both DbContexts use the **same database**
- Each DbContext has a **separate database schema**

```csharp
public class CatalogDbContext : DbContext
{
    public DbSet<Product> Products { get; set; }

    public DbSet<Category> Categories { get; set; }
}

public class OrderDbContext : DbContext
{
    public DbSet<Order> Orders { get; set; }

    public DbSet<LineItem> LineItems { get; set; }
}
```

First we need to configure the `CatalogDbContext` and `OrderDbContext` with the DI container.
You can do this by calling the `AddDbContext` method and specifying which `DbContext` is being configured,
and then using the SQL provider specific method to pass the connection string.
In this case I'm connecting to SQL Server with the `UseSqlServer` method.

```csharp
using Microsoft.EntityFrameworkCore;

services.AddDbContext<CatalogDbContext>(options =>
    options.UseSqlServer("CONNECTION_STRING"));

services.AddDbContext<OrderDbContext>(options =>
    options.UseSqlServer("CONNECTION_STRING"));
```

If you just want to use both DbContexts in the same schema, then this is all the configuration you need.
You can now inject the `DbContext` instances in your application and use them.

However, if you want to configure a different schema for each `DbContext` then you also need to override
the `OnModelCreating` method and specify the custom schema with `HasDefaultSchema`.

```csharp
public class CatalogDbContext : DbContext
{
    public DbSet<Product> Products { get; set; }

    public DbSet<Category> Categories { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.HasDefaultSchema("catalog");
    }
}

public class OrderDbContext : DbContext
{
    public DbSet<Order> Orders { get; set; }

    public DbSet<LineItem> LineItems { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.HasDefaultSchema("order");
    }
}
```

Limitations with multiple DbContexts:

1. It's not possible to do a join between different `DbContext` instances, because **EF Core** doesn't know if they
   are using the same database
2. Transactions will only work if the DbContexts are using the same database. You have to create a new transaction
   and share it between the DbContexts by [calling the `UseTransaction` method](/blog/working-with-transactions-in-ef-core#using-existing-transactions-with-ef-core)

**Migrations History Table**

If you decide to use different schemas per `DbContext`, you will be unpleasantly surprised to learn
that the default schema doesn't apply to the migrations history table.

You need to configure this by calling the `MigrationsHistoryTable` method and specifying the table name
and schema where the migrations history for that context will live.
I used the `HistoryRepository.DefaultTableName` constant in this example, but you can specify a custom
table name if you want to.

```csharp
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Migrations;

services.AddDbContext<CatalogDbContext>(options =>
    options.UseSqlServer(
        "CONNECTION_STRING",
        o => o.MigrationsHistoryTable(
            tableName: HistoryRepository.DefaultTableName,
            schema: "catalog")));

services.AddDbContext<OrderDbContext>(options =>
    options.UseSqlServer(
        "CONNECTION_STRING",
        o => o.MigrationsHistoryTable(
            tableName: HistoryRepository.DefaultTableName,
            schema: "order")));
```

## Benefits Of Using Multiple DbContexts

Using multiple DbContexts can offer several benefits to your application:

- Separation of concerns
- Better performance
- More control & security

Each DbContext can be responsible for a specific subset of the application's data, which can help organize the code
and make it more modular.

When you separate data access into multiple DbContexts, the application can reduce the risk of contention and improve
concurrency, and this can improve performance.

And if you're using multiple DbContexts, you can configure more granular access control to improve application security.
You can also optimize performance and resource usage.

## In Summary

Using **multiple EF Core DbContexts** in a single application is straightforward and has many benefits.

For ready-heavy applications you can configure a separate `DbContext` to turn off query tracking by default
and get improved performance.

Also, using multiple DbContexts is practical if you're building a **Modular monolith**.
You can configure the DbContexts to be in **separate database schemas**, giving you logical separation at the database level.

That's all for today.

See you next week.

---



<Sponsorships>

Today's issue is sponsored by [**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website),
the translation management platform that helps you translate texts in your app for
[**targeting multilingual markets.**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website)

And by [**Hasura**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly),
an open-source engine that gives
[**instant GraphQL and REST APIs**](https://hasura.io/graphql/database/sql-server/?utm_source=newsletter&utm_medium=email&utm_campaign=milan-dotnet-weekly)
on new or existing SQL Server, enabling teams to ship apps faster.

</Sponsorships>

Although **C#** is an **object-oriented programming** language, it received many new functional features in recent versions.

To mention just a few of these features:

- Pattern matching
- Switch expressions
- Records

You're probably already doing **functional programming** without even knowing it.

Do you use LINQ?
If you do, then you're doing functional programming.
Because LINQ is a functional .NET library.

In today's issue, I will show you how to **refactor** some **imperative code** with **functional programming**.

Let's dive in.

## Benefits Of Functional Programming

Before we take a look at some code, let's see what are the **benefits** of using **functional programming**:

- Emphasis on immutability
- Emphasis on function purity
- Code is easier to reason about
- Less prone to bugs and errors
- Ability to compose functions and create higher-order functions
- Easier to test and debug

From my experience, I think functional programming is more enjoyable once you get used to it.
Starting out, it feels strange because your old object-oriented programming habits will kick in.
But after a while, **functional programming** feels easier to work with than imperative code.

## Starting With Imperative Code

**Imperative programming** is the most basic programming approach.
We describe a step-by-step process to execute a program.
It's easier for beginners to reason with imperative code by following along with the steps in the process.

Here's an example of an `EmailValidator` class written with imperative code:

```csharp
public class EmailValidator
{
    private const int MaxLength = 255;

    public (bool IsValid, string? Error) Validate(string email)
    {
        if (string.IsNullOrEmpty(email))
        {
            return (false, "Email is empty");
        }

        if (email.Length > MaxLength)
        {
            return (false, "Email is too long");
        }

        if (email.Split('@').Length != 2)
        {
            return (false, "Email format is invalid");
        }

        if (Uri.CheckHostName(email.Split('@')[1]) == UriHostNameType.Unknown)
        {
            return (false, "Email domain is invalid");
        }

        return (true, null);
    }
}
```

You can clearly see the distinct steps:

- Check if email is null or empty
- Check if email is not too long
- Check if email format is valid
- Check if email domain is valid

Let's see how we can refactor this using **functional programming**.

## Applying Functional Programming

The basic building block in **functional programming** is - **a function**.
And programs are written by composing function calls.
There are a few other things you need to keep in mind, like keeping your functions pure.
A function is pure if it always returns the same output for the same input.

We can capture each step from the imperative version of `EmailValidator` with a `Func` delegate.
To also capture the respective error message together with the validation check, we can use a tuple.
And since we know all of our validation steps, we can create an array of `Func` delegates to store all of the individual **functions**.

```csharp
public class EmailValidator
{
    const int MaxLength = 255;

    static readonly Func<string, (bool IsValid, string? Error)>[] _validations =
    {
        email => (!string.IsNullOrEmpty(email), "Email is empty"),
        email => (email.Length <= MaxLength, "Email is too long"),
        email => (email.Split('@').Length == 2, "Email format is invalid"),
        email => (
            Uri.CheckHostName(email.Split('@')[1]) != UriHostNameType.Unknown,
            "Email domain is invalid")
    };

    static readonly (bool IsValid, string? Error) _successResult = (true, null);

    public (bool IsValid, string? Error) Validate(string email)
    {
        var validationResult = _validations
            .Select(func => func(email))
            .FirstOrDefault(func => !func.IsValid);

        return validationResult is { IsValid: false, Error: { Length: >0 } } ?
            validationResult : _successResult;
    }
}
```

Notice that this allows us to do all sorts of interesting things with the `_validations` array.
How hard would it be to modify this function to return _all of the errors_ instead of just the first one?

If you're thinking we can use LINQ's `Select` method somehow, you're thinking in the right direction.

## Further Reading

We only scratched the surface of what functional programming is, and what you can do with it.
If you want to learn more, here are some learning materials:

- [Functional Programming in C#, by Enrico Buonanno](https://www.manning.com/books/functional-programming-in-c-sharp)
- [Functional Programming With C# Using Railway-Oriented Programming](https://youtu.be/dDasAmowFts)
- [How Function Composition Can Make Your Code Better](https://youtu.be/zuy2j8vxgYc)
- [Make Your ASP.NET Core API Controllers Incredibly Simple With Functional Programming](https://youtu.be/AVA2mKG4WOc)

Thank you for reading, and have a wonderful Saturday.

---



<Sponsorships>

[**Localizely**](https://localizely.com/?utm_source=milanjovanovic.tech&utm_medium=email&utm_campaign=website) is a translation management platform that
helps you translate texts in your app for targeting multilingual markets.

</Sponsorships>

Working with **Microservices**, or any distributed system for that matter, is difficult.
In a distributed system many things can go wrong, and there are even research papers about this.
If you want to explore this topic further, I suggest that you read about the [fallacies of distributed computing](https://www.se.rit.edu/~se442/doc/fallacies.pdf).

Reducing the surface area for things to go wrong should be one of your goals, as an engineer.
In this week's newsletter, we'll try to achieve exactly that using the **Outbox pattern**.

How can you implement reliable communication between components in a distributed system?

The **Outbox pattern** is an elegant solution to this problem, allowing you to achieve transactional
guarantees in a single service and at-least-once message delivery to external systems.

Let's see how the **Outbox pattern** solves this and how can we implement it.

## What Problem Does The Outbox Pattern Solve?

To understand what problem the **Outbox pattern** solves, first we need a problem, of course.

Here's an example of a user registration flow.
There are a few things going on here:

- Saving the `User` to the database
- Sending a welcome email to the `User`
- Publishing a `UserRegisteredEvent` to a message bus

```csharp
public async Task RegisterUserAsync(User user, CancellationToken token)
{
    _userRepository.Insert(user);

    await _unitOfWork.SaveChangesAsync(token);

    await _emailService.SendWelcomeEmailAsync(user, token);

    await _eventBus.PublishAsync(new UserRegisteredEvent(user.Id), token);
}
```

In the happy path, all of the operations complete without any issues and all is well.

But what happens if any one of these operations fail?

- The database is unavailable, and saving the `User` fails
- The email service is down and sending an email crashes
- Publishing an event to the service bus doesn't succeed

Also, imagine a situation where you manage to save a `User` to the database,
send him a welcome email, but fail to publish the `UserRegisteredEvent` to notify other services.
How are you going to recover from this scenario?

The **Outbox pattern** allows you to **atommically** update the database and send messages to the message bus.

## Implementing The Outbox Pattern

The first step is to introduce a table in your database to represent the **Outbox**.
We can call this table `OutboxMessages`, and it's intended to store all messages that need to be delivered.
Now instead of directly making requests to external services, we simply store a message as a new row in the **Outbox** table.
The messages are usually stored as JSON in the database.

The second step is to introduce a **background process** that will periodically poll the `OutboxMessages` table.
If the worker process finds a row with an unprocessed message, it's going to publish that message and mark it as sent.
If publishing the message fails for some reason, the work process can **retry** in the next execution.

Notice that with retries, you now have **at-least-once message delivery** implemented.
The message will be published exactly once for the happy path, and more than one time in case or retries.

We can rewrite the `RegisterUserAsync` method from the previous example, now using an **Outbox**:

```csharp
public async Task RegisterUserAsync(User user, CancellationToken token)
{
    _userRepository.Insert(user);

    _outbox.Insert(new UserRegisteredEvent(user.Id));

    await _unitOfWork.SaveChangesAsync(token);
}
```

The **Outbox** is part of the same transaction as our unit of work, so we can atomically save the `User` to the database
and also persist the `OutboxMessage`.
If saving to the database fails, the entire transaction is rolled back and no messages are sent to the message bus.

And since we now moved the publishing of the `UserRegisteredEvent` to the worker process, we need to add a handler
so that we can send the welcome email to the user.
Here's an example of that in the `SendWelcomeEmailHandler` class:

```csharp
public class SendWelcomeEmailHandler : IHandle<UserRegisteredEvent>
{
    private readonly IUserRepository _userRepository;
    private readonly IEmailService _emailService;

    public SendWelcomeEmailHandler(
        IUserRepository userRepository,
        IEmailService emailService)
    {
        _userRepository = userRepository;
        _emailService = emailService;
    }

    public async Task Handle(UserRegisteredEvent message)
    {
        var user = await _userRepository.GetByIdAsync(message.UserId);

        await _emailService.SendWelcomeEmailAsync(user);
    }
}
```

## Architecture Diagram With Outbox

Here's a high level overview of the system architecture with the **Outbox** introduced to the system.
You can see the `Outbox` table in the database.
What changes now is that you store messages to the `Outbox` table in the same transaction along with your entities.

<Image src="blogs/mnw_026/outbox.png" width={1801} height={989} />

## Further Reading

After reading this newsletter you should have a pretty good understanding of what the **Outbox pattern** is
and what problems it solves.
If you need to implement reliable messaging in a distributed system, it's a great solution for your problem.

What's missing is more details about how to implement the **Outbox pattern**, so here are a few videos you can watch:

- [How to use the Domain Events pattern](https://youtu.be/BimfDeDV4yU)
- [How to implement the Outbox pattern](https://youtu.be/XALvnX7MPeo)
- [How to add retries to the Outbox with Polly](https://youtu.be/xajVttkZntU)

Thanks for reading, and have an amazing Saturday.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

**Structured logging** is a practice where you apply the same message format to all of your
application logs.
The end result is that all your logs will have a similar structure, allowing them to be
easily searched and analyzed.

[Serilog](https://serilog.net/) is a popular logging library in .NET, packed with many features.
It provides logging to files, logging to the console, and elsewhere.

However, **Serilog** is unique because it comes with support for **structured logging** out of the box.

Let's see how we can install **Serilog** and configure it an **ASP.NET Core** application.

## Installing Serilog

To install **Serilog** in **ASP.NET Core** you can add the following NuGet package:

```powershell
Install-Package Serilog.AspNetCore
```

This NuGet packages comes with a simple API to integrate **Serilog** into your application.
You can call the `UseSerilog` method on the `HostBuilder` instance to provide a lambda
method to configure **Serilog**.

I think the most flexible way to configure Serilog is through application settings,
which is achieved by calling `ReadFrom.Configuration()`.

You can also call the `UseSerilogRequestLogging()` method to introduce automatic HTTP request logging
in your API.

```csharp
var builder = WebApplication.CreateBuilder(args);

builder.Host.UseSerilog((context, configuration) =>
    configuration.ReadFrom.Configuration(context.Configuration));

var app = builder.Build();

app.UseSerilogRequestLogging();

app.Run();
```

The next question is how do you provide the actual configuration values to **Serilog**?

## Configuring Serilog With `appsettings.json`

You need to add a `Serilog` section in your `appsettings.json` file.

Here you can configure, among other things:

- Which **sinks** to use with **Serilog**
- Override default and minimum log levels
- Configure file logging arguments

In this example, we're adding the `Console` and `File` sinks to **Serilog**.
And we're adding some additional configuration for the `File` sink in the `Serilog.WriteTo` configuration section.
We can configure the output path for the log files, the naming format, which formatter to use for the logs and so on.

```json
"Serilog": {
  "Using": [ "Serilog.Sinks.Console", "Serilog.Sinks.File" ],
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Warning",
      "System": "Warning"
    }
  },
  "WriteTo": [
    { "Name": "Console" },
    {
      "Name": "File",
      "Args": {
        "path": "/logs/log-.txt",
        "rollingInterval": "Day",
        "rollOnFileSizeLimit": true,
        "formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
      }
    }
  ],
  "Enrich": [ "FromLogContext", "WithMachineName", "WithThreadId" ]
}
```

You can get a more detailed overview of what's supported with the `Serilog.Configuration` library in the
[documentation](https://github.com/serilog/serilog-settings-configuration).

## Using Serilog In ASP.NET Core

We managed to successfully install and configure **Serilog**.
But how do we actually use it?

**Serilog** integrates with the `ILogger` interaface coming from the `Microsoft.Extensions.Logging` namespace.
If you're already using `ILogger` for logging, everything will continue working correctly.

Here's a simple example of logging inside of a Minimal API endpoint:

```csharp
app.MapGet("/serilog-is-cool", (ILogger logger) =>
{
    logger.LogInformation("This is a log inside of the Minimal API endpoint.");

    return Results.Ok(new { Message = "success" });
});
```

You just inject an `ILogger` or `ILogger<T>` instance and Serilog will provide its own implementation at runtime.

## Structured Logging Syntax

The idea behind **structured logging** is that you can introduce additional contextual information inside
of your logs.
**Serilog** does this using a message template syntax, where you can specify named parameters and then
pass in their values separately.

Here's an example of what this message template would look like.
You specify parameters inside of curly bracers and provide a name, for example `{NamedParameter}`.
The value provided for the parameter will be serialized as a property inside of the corresponding
structured log.

```csharp
var book = new { Author = "Domain-Driven Design", Title = "Eric Evans" };
var orderNumber = 1;

log.LogInformation(
    "Processing book {@Book}, order number = {@OrderNumber}",
    book,
    orderNumber);
```

There are a few things to unpack here:

- `{@Book}` parameter which accepts an object
- `{OrderNumber}` parameter which accepts a scalar value

The `@` operator in front of `Book` tells Serilog to serialize the object passed in, instead of converting
it using `ToString()`.

## Benefits Of Structured Logging

Lastly, I want to highlight what are some of the benefits of **structured logging** and why you should be using it.

As I said at the beginning, the main idea with **structured logging** is that all log message follow the same
structure.
This structure can be a JSON document for example, or a row in a relational table.
Since structured logs are in a machine-readable format, it's much easier to search through them for
specific information.

When an error occurs, structured logs can provide more context and details about the error, making it easier to
identify the root cause and fix the problem.

It's very easy to start doing structured logging with **Serilog**, and I hope you'll give it a try.

See you next week, and have an excellent Saturday.

---



If you're working in a **distributed system**, you need to be able to communicate between
multiple services.
There are a few ways that you can implement this.
Depending on your chosen approach, you can either introduce tight coupling between your
services or stay **loosely coupled**.

**Loose coupling** is an important quality in distributed systems.
It allows you to evolve your services independently.
So how do you implement loosely coupled **communication between services**?

You need a **messaging system**.

And [Azure Service Bus](https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview)
is an excellent choice.

In this week's newsletter, I'll show you how to create an **Azure Service Bus** instance,
and how to implement messaging over a **queue**.

Let's dive in.

## Creating An Azure Service Bus Instance

You can create a new **Azure Service Bus** instance from the **Azure** portal.

I won't go into detail on that, since I find the **Azure** UI pretty intuitive.

<Image src="blogs/mnw_024/service_bus.png" width={898} height={695} />

After creating your **Azure Service Bus** instance, you'll need to do two more things:

- Create a new **queue**
- Find the **connection string**

After that you can proceed with implementing the pub-sub pattern over a queue.
And you'll use the connection string from the Azure portal for connecting to the Azure Service Bus instance.

## Publishing Messages To The Azure Service Bus Queue

The first thing we need to do is to create the publishing side of our system, and then
we'll see how we can process messages.
We're going to use the `Azure.Messaging.ServiceBus` library to connect to the queue
running in Azure Service Bus.

You can install the NuGet package by running the following command:

```powershell
Install-Package Azure.Messaging.ServiceBus
```

And now, let's write the code for publishing a message to an Azure Service Bus queue.

To work with the Azure Service Bus instance, you will use the [`ServiceBusClient`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusclient?view=azure-dotnet)
class.
It requires a connection string to be able to connect to the Azure Service Bus instance.

The `ServiceBusClient` is safe to cache and reuse in the application, so it can be registered
as a service with the singleton lifetime.

With the `ServiceBusClient` you can create a [`ServiceBusSender`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebussender?view=azure-dotnet)
instance, which is responsible
for sending the actual messages.
You also need to specify which queue it will be sending messages to.
This can also be the name of a topic, if you are publishing to a topic instead.

```csharp
using Azure.Messaging.ServiceBus;

await using var client = new ServiceBusClient(ConnectionString);

await using ServiceBusSender sender = client.CreateSender(QueueName);

// This will be the payload for the message ✉️
var productCreated = new ProductCreatedEvent(
    eventId: Guid.NewGuid(),
    product.Id,
    product.Name);

string json = JsonSerializer.Serialize(productCreated);

var message = new ServiceBusMessage(json);

await sender.SendMessageAsync(message);
```

As you can see, publishing a message to the queue is relatively simple.

We're creating a new instance of `ProductCreatedEvent`, which represents our message payload.
The payload itself is serialized into a `JSON` string, wrapped inside a `ServiceBusMessage`.

For publishing messages to Azure Service Bus, you simply call the `SendMessageAsync` method.

## Receiving Messages From The Azure Service Bus Queue

Publishing messages to a queue is only half of the job.
You also need to be able to receive messages from the queue.
However, you'll see that this is very similar to the publishing side.

On the receiving side, you'll also need to create a `ServiceBusClient`.
And then use it to create an instance of [`ServiceBusProcessor`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor?view=azure-dotnet),
which is used for consuming messages.
You need to tell the `ServiceBusProcessor` which queue it will subscribe to.

The `ServiceBusProcessor` exposes two events, which represent callbacks for when a message is received.
These events are [`ProcessMessageAsync`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor.processmessageasync?view=azure-dotnet)
and [`ProcessErrorAsync`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor.processerrorasync?view=azure-dotnet).

You need to provide a handler for these two events, to properly consume messages from the queue.
In the example below, we're using the `HandleMessageAsync` and `HandleErrorAsync` local functions.

```csharp
using Azure.Messaging.ServiceBus;

await using var client = new ServiceBusClient(ConnectionString);

await using ServiceBusProcessor processor  = client.CreateProcessor(QueueName);

processor.ProcessMessageAsync += HandleMessageAsync;

processor.ProcessErrorAsync += HandleErrorAsync;

await processor.StartProcessingAsync();

async Task HandleMessageAsync(ProcessMessageEventArgs args)
{
    string json = args.Message.Body.ToString();

    var productCreated = JsonSerializer.Deserialize<ProductCreatedEvent>(json);

    Console.WriteLine(productCreated);

    await args.CompleteMessageAsync(args.Message);
}

Task HandleErrorAsync(ProcessErrorEventArgs args)
{
    var exception = args.Exception;

    Console.WriteLine(exception.ToString());

    return Task.CompletedTask;
}
```

The `ServiceBusProcessor` begins listening to messages coming from the queue after calling the `StartProcessingAsync` method.

Notice that the success and error event handlers need to accept the
[`ProcessMessageEventArgs`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.processmessageeventargs?view=azure-dotnet)
and [`ProcessErrorEventArgs`](https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.processerroreventargs?view=azure-dotnet),
respectively.

From the `ProcessMessageEventArgs` you can access the `Message.Body` which contains the mesage payload.

## Further Reading

**Azure Service Bus** is a very feature rich service cloud.
I showed you how to work with **queues**, which are great if you only have one publisher and one subscriber.
However, if you need the ability to have multiple subscribers to a single message, you can't achieve
this with queues.

You will have to use [topics](https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-queues-topics-subscriptions#topics-and-subscriptions),
and I invite you to research this _topic_ further (pun intended).

**Azure Functions** have excellent support for integrating with **Azure Service Bus**.
You can define a `QueueTrigger` that will run your **Azure Function** when you receive a message to an Azure Service Bus **queue**.

I also released a video showing how to [publish and consume messages using RabbitMQ](https://youtu.be/CTKWFMZVIWA),
and I think you might enjoy it after reading this newsletter.

Have an excellent weekend, and stay awesome!

---



<Sponsorships>

[**Ultimate ASP.NET Core Web API - Second Edition**](https://code-maze.lemonsqueezy.com/checkout/buy/b0e9ff04-cb9f-4bc2-a69b-d434fad56790)
is a step-by-step and easy-to-digest guide to creating scalable, robust, and high-performance Web APIs.

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

Every software engineer working with **SQL databases** needs to know about **transactions**.
And since most of the time the **SQL database** will be abstracted by an ORM like **EF Core**,
it's important to understand how you can work with **transactions** using the available
abstractions.

So today, I'll show you how to work with **transactions** in **EF Core**.

Here's what we will cover:

- Default transaction behavior
- Creating transactions
- Using existing transactions

Let's dive in.

## Default Transaction Behavior

What is the **default** EF Core **transaction behavior**?

By default, all changes made in a single call to `SaveChanges` are applied in a
transaction. If any of the changes fail, the entire transaction is rolled back
and no changes are applied to the database. Only if all changes are successfully
persisted to the database, the call to `SaveChanges` can complete.

This is a wonderful feature of **SQL databases** and it saves us many headaches.
We don't have to think about the databases remaining in an inconsistent state,
because database **transactions** can do the work for us.

Let's take a look at an example.

```csharp
using var context = new ShoppingContext();

context.LineItems.Add(new LineItem
{
    ProductId = productId,
    Quantity = quantity
});

var stock = context.Stock.FirstOrDefault(s => s.ProductId == productId);

stock.Quantity -= quantity;

context.SaveChanges();
```

Because we are adding a `LineItem`, and in the same scope reducing the `Stock`
quantity, the call to `SaveChanges` will apply both changes inside of a transaction.
We can guarantee that the database will remain in a **consistent state**.

## Creating Transactions With EF Core

What if you want to have more control over **transactions** when working with **EF Core**?

You can manually create a transaction by accessing the `Database` facade available
on a `DbContext` instance and calling `BeginTransaction`.

Here's an example where we have multiple calls to `SaveChanges`. In the default
scenario, both calls would run in their own transaction. This leaves the possibility
of the second call to `SaveChanges` failing, and leaving the database in an
inconsistent state.

```csharp
using var context = new ShoppingContext();
using var transaction  = context.Database.BeginTransaction();

try
{
    context.LineItems.Add(new LineItem
    {
        ProductId = productId,
        Quantity = quantity
    });

    context.SaveChanges();

    var stock = context.Stock.FirstOrDefault(s => s.ProductId == productId);

    stock.Quantity -= quantity;

    context.SaveChanges();

    // When we commit the changes, they will be applied to the databases.
    // The transaction will auto-rollback when it is disposed,
    // if any command fails.
    transaction.Commit();
}
catch (Exception)
{
    transaction.Rollback();
}
```

We call `BeginTransaction` to manually start a new **database transaction**.
This will create a new transaction and return it, so that we can `Commit` the
transaction when we want to complete the operation. You also want to add a
`try-catch` block around your code, so that you can `Rollback` the transaction
if there are any exceptions.

## Using Existing Transactions With EF Core

Creating a transaction using the EF Core `DbContext` isn't the only option.
You can create a `SqlTransaction` instance and pass it to **EF Core**, so that
the changes applied with EF Core can be committed inside the same **transaction**.

Here's what I mean:

```csharp
using var sqlConnection = new SqlTransaction(connectionString);
sqlConnection.Open();

using var transaction = sqlConnection.BeginTransaction();

try
{
    using var context = new ShoppingContext();

    // Tell EF Core to use an existing transaction.
    context.UseTransaction(transaction);

    context.LineItems.Add(new LineItem
    {
        ProductId = productId,
        Quantity = quantity
    });

    context.SaveChanges();

    var stock = context.Stock.FirstOrDefault(s => s.ProductId == productId);

    stock.Quantity -= quantity;

    context.SaveChanges();

    transaction.Commit();
}
catch (Exception)
{
    transaction.Rollback();
}
```

## In Summary

**EF Core** has excellent support for **transactions** and it's very easy to work with.

You have three options available:

- Rely on the default transaction behavior
- Create a new transaction
- Use an existing transaction

Most of the time, you want to rely on the default behavior and not have to
think about it.

As soon as you need to perform multiple `SaveChanges` calls, you should manually
create a transaction, and manage the transaction yourself.

See you next week, and have an excellent Saturday.

---



In this week's newsletter I want to show you how to implement **API Key authentication**
in **ASP.NET Core**. This authentication approach uses an **API Key** to authenticate the
client of an API. You can pass the **API Key** to the API in a few ways, such as through
the query string or a request header.

I will show you how to implement **API Key authentication** where the **API key** is passed
in a request header. But the implementation would be similar if we were to use any
other approach.

When would you want to use **API Key authentication**? This kind of authentication
mechanism is common in **Server-to-Server (S2S)** communication. When your API serves
request for other server-side applications to consume and integrate with. It's
less common in client-server communication scenarios.

Let's see how we can implement **API Key authentication** in ASP.NET Core!

## Implementing API Key Authentication

We will start off by creating an attribute that we can place on endpoints
where we want to apply **API Key authentication**. It won't be any kind of
attribute, because we will use a `ServiceFilterAttribute`.

What a `ServiceFilterAttribute` allows us to do is specify a type for the
filter that will be created for that attribute.
This means we can implement our authentication logic in an `IAuthorizationFilter`.
With a `ServiceFilterAttribute` we also have support for dependency injection
in our `IAuthorizationFilter` implementation.

Let's first define the `ApiKeyAttribute` class:

```csharp
public class ApiKeyAttribute : ServiceFilterAttribute
{
    public ApiKeyAttribute()
        : base(typeof(ApiKeyAuthorizationFilter))
    {
    }
}
```

In the `ApiKeyAttribute` we specify `ApiKeyAuthorizationFilter` class as the
filter that will be resolved from the DI container. Here's what it looks like:

```csharp
public class ApiKeyAuthorizationFilter : IAuthorizationFilter
{
    private const string ApiKeyHeaderName = "X-API-Key";

    private readonly IApiKeyValidator _apiKeyValidator;

    public ApiKeyAuthorizationFilter(IApiKeyValidator apiKeyValidator)
    {
        _apiKeyValidator = apiKeyValidator;
    }

    public void OnAuthorization(AuthorizationFilterContext context)
    {
        string apiKey = context.HttpContext.Request.Headers[ApiKeyHeaderName];

        if (!_apiKeyValidator.IsValid(apiKey))
        {
            context.Result = new UnauthorizedResult();
        }
    }
}
```

The implementation comes down to validating the **API Key** obtained from
the header of the current request. If we determine that the **API Key**
is not valid, we set the value of `AuthorizationFilterContext.Result`
to a new instance of an `UnauthorizedResult`.

And lastly, all that's left for us to do is implement our custom
validation logic for the **API Key** inside of `ApiKeyValidator`:

```csharp
public class ApiKeyValidator : IApiKeyValidator
{
    public bool IsValid(string apiKey)
    {
        // Implement logic for validating the API key.
    }
}

public interface IApiKeyValidator
{
    bool IsValid(string apiKey);
}
```

The actual implementation for validating the **API Key** will vary based
on your use case, and where you are storing the API keys.
For example, if you store the API keys in the database you would check
if the provided **API Key** exists in the database.
If it exists, then validation passes.
If it doesn't exist, then validation fails and we return an
`UnauthorizedResult`.

## Registering Services With Dependency Injection

We have to make sure to register our `ApiKeyAuthorizationFilter` and
`ApiKeyValidator` services with the dependency injection container.

```csharp
builder.Services.AddSingleton<ApiKeyAuthorizationFilter>();

builder.Services.AddSingleton<IApiKeyValidator, ApiKeyValidator>();
```

This will register them as singleton services in our application.
You can use a different service scope such as `Transient` or `Scoped`
if you need to.

## Applying API Key Authentication To Endpoints

Finally, with our **API Key authentication** in place, we can apply the
`ApiKeyAttribute` attribute to our endpoints:

```csharp
public class NewslettersController : ControllerBase
{
    [ApiKey]
    [HttpGet]
    public IActionResult Get()
    {
        // ...
    }
}
```

In this case I'm applying the `ApiKeyAttribute` to an endpoint, but
you can also apply it on the `NewslettersController` and it will add
authentication to all the endpoints for that controller.

## Next Steps

Now that you know how to implement **API Key authentication**, I think you
should also learn how to implement **JWT authentication**. And while you're
at it, why not throw **authorization** into the mix.

I made a few videos about **JWT authentication** and **permission authorization**
that you should take a look at next:

- [Token Authentication In ASP.NET Core 7 With JWT](https://youtu.be/4cFhYUK8wnc)
- [Introduction To Permission Authorization In ASP.NET Core 7](https://youtu.be/PlbAuNvR16s)
- [Managing Permissions With EF Core Migrations](https://youtu.be/v4vXDRJ9_sg)
- [Implementing A Custom Authorization Handler In ASP.NET Core](https://youtu.be/SZtZuvcMBA0)
- [Using Custom JWT Claims For Authorization In ASP.NET Core](https://youtu.be/SUyFPp6BPV0)

---



<Sponsorships>

[**DotnetDevs**](https://dotnetdevs.co/) is a reverse job board that empowers .NET developers available for their next job.
Stop scouring traditional job boards and [**sit back as companies reach out to you first**](https://dotnetdevs.co/).

</Sponsorships>

In this week's newsletter I want to talk about the `yield` keyword in **C#**.
I think it's a powerful **C#** feature and I wanted to highlight the benefits.

The `yield` keyword tells the compiler that the method in which it appears
is an **iterator block**. An iterator block, or method, returns an `IEnumerable`
as the result. And the `yield` keyword is used to return the values for the
`IEnumerable`.

An interesting thing aboug `IEnumerable` is that it is lazily evaluted.
Calling a method with an iterator block doesn't run any code. It's only
when the `IEnumerable` is iterated over, or enumerated, that we get
the actual values. I'll talk about this more later.

Let's see how we can start using the `yield` keyword!

## How To Use The Yield Keyword

The `yield` keyword on it's own doesn't do anything, you have to combine
it with the `return` or `break` statement:

- `yield return` - provides the next value of the iterator
- `yield-break` - signals the end of iteration

In every project I worked on, there's a piece of code similar to the
following. You create a list to hold the results, add elements to the
list, and return the list in the end.

```csharp
var engineers = GetSoftwareEngineers();

public IEnumerable<SoftwareEngineer> GetSoftwareEngineers()
{
    var result = new List<SoftwareEngineer>();

    for(var i = 0; i < 10; i++)
    {
        result.Add(new SoftwareEngineer
        {
            Id = i
        });
    }

    return result;
}
```

You can simplify the method using the `yield return` statement, and
completely remove the intermediate list required to hold the results.

```csharp
var engineers = GetSoftwareEngineers();

public IEnumerable<SoftwareEngineer> GetSoftwareEngineers()
{
    for(var i = 0; i < 10; i++)
    {
        yield return new SoftwareEngineer
        {
            Id = i
        };
    }
}
```

However, it's important to note these two implementation are fundamentally
different from each other. In the first example, the entire list is
populated and materialized. In the second example, the `IEnumerable`
returned will not be materialized and you have to either iterate over
it inside a `foreach` loop or call `ToList()`.

## Stopping Iteration With Yield Break

You can use the `yield break` statement to stop iteration and exit
the iterator block. Typically you would do this when a certain
condition is met, or you only want to return a specific set of values
from the iterator block.

Here's an example where this would be useful:

```csharp
Console.WriteLine(string.Join(", ", TakeWhilePositive(new[] { 1, 2, -3, 4 })));
// Output: 1, 2

public IEnumerable<int> TakeWhilePositive(IEnumerable<int> numbers)
{
    foreach(int num in numbers)
    {
        if (num > 0)
        {
            yield return num;
        }
        else
        {
            yield break;
        }
    }
}
```

## Working With IAsyncEnumerable

In **C# 8** we got the `IAsyncEnumerable` type which allows us to
iterate over a collection asynchronously with the `yield` statement.

For example, this can be useful when you want to call a thid-party
API multiple times to fetch some data. A common situation is when
you get a list of users from the database, and then have to call
an external storage service to get profile picture information.

Without `IAsyncEnumerable` you would have to do something like this:

```csharp
public async Task<IEnumerable<User>> GetUsersAsync()
{
    var users = await GetUsersFromDbAsync();

    foreach(var user in users)
    {
        user.ProfileImage = await GetProfileImageAsync(user.Id);
    }

    return users;
}

// And you would call the method like this.
var users = await GetUsersAsync();

foreach(var user in users)
{
    Console.WriteLine(user);
}
```

Now, consider this same example with the use of `IAsyncEnumerable`:

```csharp
public async IAsyncEnumerable<User> GetUsersAsync()
{
    var users = await GetUsersFromDbAsync();

    foreach(var user in users)
    {
        user.ProfileImage = await GetProfileImageAsync(user.Id);

        yield return user;
    }
}

// And you would call the method like this.
await foreach(var user in GetUsersAsync())
{
    Console.WriteLine(user);
}
```

The second implementation will iterate over the users returned from
the database when they are yielded by the `IAsyncEnumerable`.

## When Should I Use Yield?

I've found a few interesting practical applications for the `yield` keyword.
One example is when implementing Domain-Driven Design value objects.

Value objects need to support structural equality. They need to implement
a method that returns all of the equality components. Here's an example of
that using the `yield return` statement:

```csharp
public class Address
{
    public string City { get; init; }

    public string Street { get; init; }

    public string Zip { get; init; }

    public string Country { get; init; }

    public IEnumerable<object> GetEqualityComponents()
    {
        yield return City;
        yield return Street;
        yield return Zip;
        yield return Country;
    }
}
```

---



<Sponsorships>

Have some [**C# and .NET experience**](https://packt.link/PJqza), and want to get started building apps?
Get introduced to the most common technologies like gRPC, Blazor, GraphQL, Cosmos DB, SignalR, .NET MAUI, and more in
[**Apps and Services with .NET 7**](https://packt.link/PJqza).

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter I want to introduce you to an interesting feature
in **EF Core** called **Compiled Queries**.

If you have queries that you execute frequently in your application with a
different set of parameters, it can be helpful to explicitly compile the
query and reuse it throughout the lifetime of your application.

**Compiled Queries** are more performant than standard EF queries, because they
can take advantage of some additional optimizations.

Let me show you how to use **Compiled Queries**, and how much performance
improvement to expect!

## How To Create Compiled Queries

Lets define a simple class that will represent out data model that we will
use for writing **EF** queries:

```csharp
public class Newsletter
{
    public long Id { get; init; }

    public string Title { get; init; }

    public int ReadTimeInMinutes { get; init; }
}
```

How would we write a simple query to fetch a `Newsletter` by the `Id`?
I think you can do this in your sleep.

```csharp
using var dbContext = new AppDbContext();

var newsletter = dbContext.Set<Newsletter>().FirstOrDefault(n => n.Id == id);
```

Now, how do we convert this query into a **Complied Query**?

There are a few steps involved:

- Create a **Compiled Query** by calling `EF.CompileQuery`
- Store the **Compiled Query** in a static field, so that it can be reused
- Execute the database query using the **Compiled Query**

You can define the **Compiled Query** in a static field inside of the `AppDbContext`.
And then expose a method that will accept an argument, and pass it to the
**Compiled Query** to invoke it.

```csharp
using Microsoft.EntityFrameworkCore;

public class AppDbContext
{
    private static Func<AppDbContext, long, Newsletter?> GetNewsletter =
        EF.CompileQuery(
            (dbContext, id) =>
                dbContext.Set<Newsletter>().FirstOrDefault(n => n.Id == id));

    public Newsletter? GetNewsletter(long id)
    {
        return GetNewsletter(this, id);
    }
}
```

This is how we would call the method which invokes the **Compiled Query**:

```csharp
using var dbContext = new AppDbContext();

var newsletter = dbContext.GetNewsletter(id);
```

I ran some benchmarks, with the following setup:

- **EF Core 7**
- **SQL Server 2022**
- Table with 10,000 records

The **Compiled Query** was consistently around **10% faster**.

I also tried running a no-tracking query by calling `AsNoTracking()` and
observed similar results.

## Why Are Compiled Queries Faster?

So we can conclude that **Compiled Queries** are faster. But why is that?

Let's examine what happens when we execute an **EF** LINQ query. Before **EF**
can convert the query into valid SQL that can be executed in the database,
it needs to compile the query. The compiled query is cached and **EF** will be
able to reuse that cached query. In some situations the query needs to be
recompiled, introducing additional performance costs.

When we explicitly compile the query by calling `EF.CompileQuery`, we
can utilize some optimization techniques that aren't available at runtime.

Note that **Compiled Queries** only improve the performance of the in-memory
portion of executing an EF query. The round trip time and materializing
results from the database remain unaffected.

## Can We Make Compiled Queries Asynchronous?

I showed you how to write a synchronous **Compiled Query**. But due to
performance considerations we almost always want to execute database
queries asynchronously.

Here's how we can create an asynchronous **Compiled Query**:

```csharp
using Microsoft.EntityFrameworkCore;

public class AppDbContext
{
    private static Func<AppDbContext, string, Task<Newsletter?>> GetByTitle =
        EF.CompileAsyncQuery(
            (AppDbContext context, string title) =>
                context.Set<Newsletter>().FirstOrDefault(c => c.Title == title));

    public async Task<Newsletter?> GetNewsletterByTitleAsync(string title)
    {
        return await GetByTitle(this, title);
    }
}
```

It's interesting that we aren't writing an asynchronous query in the expression
passed to `EF.CompileAsyncQuery`. It will be converted to an asynchronous query
during compilation.

## Compiled Queries Aren't a Silver Bullet

You might be tempted to go and convert all of your **EF** queries into
**Compiled Queries**, to squeeze out that last little bit of performance.
I urge you not do it. **Compiled Queries** are a useful tool, but they
aren't the solution to all your problems.

Instead, I think we should use **Compiled Queries** sparingly, only in
situations where we really need to do these kinds of micro-optimizations.

Although **Compiled Queries** seem great, we can't deny they increase
the complexity of our code considerably. If you think the slight
performance improvement gained from using **Compiled Queries** justifies
the increase in complexity, then by all means, you should use them.
Otherwise, I would look for other ways to improve performance.

---



<Sponsorships>

[**DotnetDevs**](https://dotnetdevs.co/) is a reverse job board that empowers .NET developers available for their next job.
Stop scouring traditional job boards and [**sit back as companies reach out to you first**](https://dotnetdevs.co/).

</Sponsorships>

In this week's newsletter I will show you how to easily add **validation**
to the strongly typed configuration objects injected with `IOptions`.

The **Options pattern** allows us to use classes to provide strongly typed
configuration values in our application at runtime.

But you have no guarantee that the configuration values injected with
`IOptions` will be correctly read from the application settings.

Let's see how we can introduce validation for `IOptions`, and make sure the application settings are correct.

## Strongly Typed Configuration

I first want to define a simple class that will represent our strongly
typed configuration. Let's say we want to integrate with the **GitHub API**,
so we create a `GitHubSettings` class to hold our configuration:

```csharp
public class GitHubSettings
{
    public string AccessToken { get; init; }

    public string RepositoryName { get; init; }
}
```

Inside of our `appsettings.json` file we need to create a section to
hold our configuration values:

```json
"GitHubSettings": {
    "AccessToken": "access-token-value",
    "RepositoryName": "youtube-projects"
}
```

And with this in place, we can configure our `GitHubSettings`:

```csharp
builder.Services.Configure<GitHubSettings>(
    builder.Configuration.GetSection("GitHubSettings"));
```

Finally, our `GitHubSettings` is properly configured and we can inject
it with `IOptions<GitHubSettings>`.

## What Could Go Wrong?

If we leave the implementation like this, we're moving the responsibility
for providing the correct configuration values to the developer. I'm not
saying we are the problem, but I've forgotten to add application settings
a few times. I'm sure this happened to you also.

Here are just a few things that can go wrong:

- Passing an incorrect section name to `IConfiguration.GetSection`
- Forgetting to add the settings values in `appsettings.json`
- Typo in a property name in the class or in the configuration
- Unbindale properties without a setter
- Data type mismatch resulting in incompatible values

Depending on which one of these mistakes is made, the application will
behave differently at runtime.

The best case scenario is that the incorrect application settings cause
a runtime exception, and you realize you have a problem and fix it.

The worst case scenario, and this happens more often than you may think,
is that the application silently fails. The application settings aren't
correctly set on the value provided by `IOptions`, but you don't get a
runtime exception. The problem may go undetected for some time.

How do we solve this?

## Validation For The Options Pattern

There is a simple way to introduce **validation** to the settings class
using **data annotations**. We just add the validation attributes that
we need to the properties of the settings class.

For example, we can add the `Required` attribute to the `GitHubSettings`
properties:

```csharp
public class GitHubSettings
{
    [Required]
    public string AccessToken { get; init; }

    [Required]
    public string RepositoryName { get; init; }
}
```

We have to slightly change how we configure the `GitHubSettings`:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .ValidateDataAnnotations();
```

A few things to note here:

- `AddOptions` - returns an `OptionsBuilder<TOptions>` that binds to
  the `GitHubSettings` class
- `BindConfiguration` - binds the values from the configuration section
- `ValidateDataAnnotations` - enables **validation** using **data annotations**

With this in place, if we try to inject `GitHubSettings` with any of
the properties missing a value, we will get a runtime exception.

You can also define a **custom delegate** for the **validation** logic, instead
of using data annotations:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .Validate(gitHubSettings =>
    {
        if (string.IsNullOrEmpty(gitHubSettings.AccessToken))
        {
            return false;
        }

        return true;
    });
```

## Running Validation At Application Start

It would be great if we could run **validation** on the configuration values
when our application is starting, instead of at runtime.

We can do that by calling `ValidateOnStart` method when configuring
our settings class:

```csharp
builder.Services
    .AddOptions<GitHubSettings>()
    .BindConfiguration("GitHubSettings")
    .ValidateDataAnnotations()
    .ValidateOnStart(); // 👈 the magic happens here
```

When we start the application, the validation will run on `GitHubSettings`
and an exception is thrown if validation fails. The validation exception
will look something like this:

```yaml
Unhandled exception. Microsoft.Extensions.Options.OptionsValidationException:
  DataAnnotation validation failed for 'GitHubSettings' members:
```

This shortens the feedback loop, and you will know right away that you have
a problem. This is much better than finding out that you have a problem at
runtime, like in the previous examples.

## Closing Thoughts

The **Options pattern** is very flexible and allows us to use strongly typed
settings in ASP.NET Core.

If you want to see how to implement the [**Options pattern**](https://youtu.be/wxYt0motww0),
I made a [**video about it where I go into the details.**](https://youtu.be/wxYt0motww0)
I covered the differences between `IOptions`, `IOptionsSnapshot` and `IOptionsMonitor`.

And now you know how to use the `ValidateOnStart` method, which was introduced
in **.NET 6**, to validate your application settings on app start up. This allows
you to learn about configuration issues as soon as possible, instead of at runtime.

I also made a video showing how to add [validation to the Option pattern](https://youtu.be/qRruEdjNVNE).

---



In this week's newsletter I will share 5 simple tips on how
you can be a better software engineer in 2023.

I find it a little amusing that the last newsletter of the year
and is also coming out on the last day of the year.

Here are 5 tips for being a better software engineer in 2023:

- [Keep learning and acquiring new skills](#1-keep-learning-and-acquiring-new-skills)
- [Invest in code quality](#2-invest-in-code-quality)
- [Work on complex systems](#3-work-on-complex-systems)
- [Be comfortable in the cloud](#4-be-comfortable-in-the-cloud)
- [Take care of yourself](#5-take-care-of-yourself)

I'm confident that you will become a better software engineer if you apply one of these, but I challenge you to work on all of them in the coming year.

Let's dive in.

## 1. Keep Learning And Acquiring New Skills

The field of software engineering is constantly evolving, so it's
important to stay up-to-date with [**new technologies**](https://youtu.be/dDasAmowFts)
and best practices.

We have a new .NET release every year, and it's easy to get lost
with the latest news. How I stay up to date is through online courses,
attending conferences and meetups, or working on [**personal projects**](https://youtu.be/Ru6_b50wdfo).

Personal projects transition best to my day job, and I don't shy away
from exploring new topics. I shared many of my personal projects on [**GitHub**](https://github.com/m-jovanovic),
maybe you can find some inspiration over there.

## 2. Invest In Code Quality

In addition to writing clean and readable code, it's important to also
consider the overall quality of your code.

This includes things like performance, security, and maintainability.
By writing high-quality code, you'll be able to build systems that are
more reliable, scalable, and easier to maintain over time.

One way to write clean and quality code is [**with the help of static code analysis**](https://youtu.be/0nVT1gM4vPg).

Investing in code quality will pay dividends in the later stages of any project.

## 3. Work On Complex Systems

If you aspire to be senior engineer or software architect, you have to
work on complex systems. You need to be in a position to solve the toughest problems.

What do I consider to be a [**complex system**](https://youtu.be/Ru6_b50wdfo)?

That's difficult to say, but here are some rough guidelines:

- Microservices
- Event-driven systems
- High performance systems

Of course, you don't need a [**fancy architecture**](https://youtu.be/Ru6_b50wdfo) to work in a complex system.

If you are working in a business domain with many domain rules,
I consider that a complex system also.

You should strive to always be in a position to work on a challenging project,
this will help you grow.

## 4. Be Comfortable In The Cloud

The cloud is here to stay, and you should be familiar with at least one
of the major cloud providers:

- Microsoft Azure
- Amazon Web Services
- Google Cloud Platform

Most of them give you free credits to get started and explore the services they offer.

I stayed away from [**cloud development**](https://youtu.be/QP0pi7xe24s) for too long in my career, and now I wish I started sooner.

You want to be [**comfortable in the cloud**](https://youtu.be/QP0pi7xe24s) to be a better software engineer.

## 5. Take Care Of Yourself

Being a software engineer is mentally and physically demanding, and it's
important to take care of yourself in order to maintain a healthy work-life balance.

This includes getting enough sleep, eating well, and taking breaks when needed.

Taking care of yourself can also help you stay focused and productive in your work.

I like to take short breaks every hour or so, and not think about work for a few minutes.
This helps me replenish my energy, and allows me to continue to operate on a high level.

## What I'm Doing To Make 2023 Amazing

What I'm Doing To Make 2023 Amazing
Making New Year's resolutions is popular at the start of the year. The problem is most people get too excited about it, but then proceed to not accomplish anything in a few months.

I've found the simplest rule to always make progress is taking action. I try to make a small step forward every day. And when I look back on the year, I realize I made a lot of progress.

Again, my 5 tips so you can be a better software engineer in 2023 are:

- [Keep learning and acquiring new skills](#1-keep-learning-and-acquiring-new-skills)
- [Invest in code quality](#2-invest-in-code-quality)
- [Work on complex systems](#3-work-on-complex-systems)
- [Be comfortable in the cloud](#4-be-comfortable-in-the-cloud)
- [Take care of yourself](#5-take-care-of-yourself)

Where I failed most in 2022 was taking care of myself, and this will be one of my main improvement points.

I also want to learn many new things, so I can share them with you in this newsletter and on my social media channels.

I wish you a very happy and prosperous New Year.
Write lots of code, squash many bugs,
and may you have green unit tests year round.

Stay awesome! 🎁\
Milan

---



<Sponsorships>

Coordinating deployments across 37,000 stores and 500,000 database
instances for 6+ years with zero incidents? Impossible! But it's
the reality for one Fortune 100 fast-food empire who chose a NoSQL
point-of-sale database designed to smartly scale.
[**Learn how RavenDB excels as a point-of-sale DB**](https://ravendb.net/whitepapers/ravendb-point-of-sale-database/?utm_campaign=mnw_newsletter_17&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_whitepaper&utm_content=ad_link).

</Sponsorships>

In the world of software development, there are countless approaches
and methodologies to choose from. It's easy to get swayed with the
latest trends, and loose sight of architectural principles that really matter.

One of the more popular ones is **Clean Architecture**, a design approach
that prioritizes maintainability, scalability, flexibility, and productivity.

In this week's newsletter, we will explore the key benefits of using
**Clean Architecture** and how it can help your team build better software.

Let's dive in.

## What Is Clean Architecture?

**Clean Architecture**, also known as "The Onion Architecture," was first
introduced by Robert C. Martin (aka "Uncle Bob") in his book
"Clean Architecture: A Craftsman's Guide to Software Structure and Design".

At its core, **Clean Architecture** is a way of organizing a software system
in a way that separates the concerns of the various components,
making it easier to understand and maintain.

In **Clean Architecture**, the core of the system is the **"inner circle"**,
which contains the business rules and logic.

Surrounding this **inner circle** are layers of abstraction,
each one representing a different concern.

<Image src="blogs/mnw_017/clean_architecture.png" width={1920} height={1080} />

The typical outer layers are **Infrastructure** and **Presentation** layers.
The **Infrastructure** layer handles external concerns such as APIs and databases.
While the **Presentation** layer exposes an interface for clients to interact with
the application.

The key principle of **Clean Architecture** is that the **inner circle**
should not depend on the outer layers. Instead, the outer layers should
depend on the **inner circle**. This helps to ensure that the **core** of the
system is flexible and easy to modify, without worrying about the impact
on other parts of the system.

## Benefits Of Using Clean Architecture

I want to highlight some of the key benefits of using **Clean Architecture**.

### Improved Maintainability

One of the primary benefits of using **Clean Architecture** is improved **maintainability**.
By separating the concerns of the various components and enforcing the dependency rule,
it becomes much easier to understand and modify the code.
Depending on abstractions allows you to design your business logic in a flexible way,
without having to know the implementation details.

### Modularity and Separation of Concerns

**Clean Architecture** helps to create a clear **separation of concerns**
within the codebase. Each layer has a specific purpose and is decoupled
from the others, making it easier to understand and modify individual
components without affecting the rest of the system. This modularity
also makes it easier to reuse components in other projects.

### Testability

**Clean Architecture** also makes it **easier to test** and debug the code.
Because the inner circle is independent of the outer layers,
it's easier to write unit tests that focus specifically on the business
rules. This can help to catch errors early on in the development
process and reduce the overall testing effort.

### Loose Coupling of Components

**Clean Architecture** also promotes **loose coupling** between the various
components of the system. This means that it's easier to swap out
external dependencies or make other modifications without affecting
the core business logic. This can be especially useful when it comes
to upgrading or replacing technology.

### Increased Flexibility

Another key benefit of **Clean Architecture** is increased **flexibility**.
By separating the concerns of the various components, it's easier
to modify and adapt the code to changing requirements. This can be
especially useful in fast-paced environments where requirements
are constantly evolving.

### Improved Team Productivity

**Clean Architecture** can help to improve team **productivity**.
By establishing clear separation of responsibilities and well-defined
boundaries, it's easier for team members to understand their roles
and responsibilities. This can improve communication and collaboration,
leading to more efficient and effective work.

## Clean Architecture In The Real World

This all sounds nice in theory, but how does **Clean Architecture**
perform in the real world?

I have used **Clean Architecture** on roughly 10 projects in the
last 5 years, and I've had a lot of success with it. It was
easy to add new features, and scale the applications when
necessary. **Clean Architecture** can easily be broken down
into multiple modules or services, if performance is suffering
and there is a need to scale out.

One problem with the **Clean Architecture** is that it is **easy to overengineer**.

Dogmatism is a real issue, as I see many people with strong
opinions of what **Clean Architecture** should be.
I've been guilty of this myself in the past.

Recently, I try to be more **pragmatic** when using **Clean Architecture**.
I apply what I like, and give myself the flexibility of **"breaking"**
**Clean Architecture** if I think it will simplify things in the long run.

## Closing Thoughts

By following the principles of **Clean Architecture**, you can create a flexible
and maintainable codebase that is well-suited to evolving requirements and technology.

However, it's important to be **pragmatic** with **Clean Architecture** and
allow yourself to be flexible in the design, in order to simplify things in the long run.

If you want to see how to apply **Clean Architecture** in practice,
I have a
[**playlist with more than 20 videos on Clean Architecture**](https://youtu.be/tLk4pZZtiDY?list=PLYpjLpq5ZDGstQ5afRz-34o_0dexr1RGa).

---



<Sponsorships>

New to C# and .NET, or want a refresher on modern .NET?\
[**C# 11 and .NET 7 - Modern Cross-Platform Development Fundamentals**](https://packt.link/402oY)
is a concise, beginner-friendly introduction to the C# language,
.NET libraries, and ASP.NET Core.

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
Visual Studio extension that lets you highlight important objects and arbitrary texts
to help you navigate your code more easily.

</Sponsorships>

Did you know you can turn **PostgreSQL** into a fully-fledged **Document database**?

**Marten** is a **.NET** library that allows developers to use the **PostgreSQL**
database as both a **document database** and a fully-featured **event store**.

You don't need to install anything else to be able to use **PostgreSQL**
as a **document database**, outside of the Nuget package. **Marten** relies
on the **JSONB** support available since **PostgreSQL** 9.4.

In this week's newsletter, I want to introduce you to the basics of working
with **Marten** and show you how easy it is to get started.

Let's dive in.

## Installing And Configuring Marten

What are you going to need to start using **PostgreSQL** as a **Document datbase**?

Other than a running instance of **PostgreSQL**, of course, you will need
to install the **Marten** Nuget package:

```csharp
dotnet add package Marten
```

**Marten** can build the required database schema and necessary tables on the fly,
and I suggest using this approach in development.
For a production environment, you definitely want to apply schema
changes on your own with migration scripts.

To register **Marten** with dependency injection, you need to call the `AddMarten`
method.

Here's an example **Marten** configuration inside of a **.NET 7** application:

```csharp
builder.Services.AddMarten(options =>
{
    options.Connection(builder.Configuration.GetConnectionString("Marten"));
});
```

This will register a few services with dependency injection:

- `IDocumentStore` - used to create sessions, generate schema migrations, and do bulk inserts
- `IDocumentSession` - used for read and write operations
- `IQuerySession` - used for read operations

Let's see how we can work with documents using **Marten**.

## Storing Documents With Marten

Storing documents in the database is very straightforward. You need to create
a new `DocumentStore` instance, and open an `IDocumentSession` which exposes
methods for storing and persisting documents.

Let's see how we can store a `Product` document:

```csharp
var store = DocumentStore.For("Connection string to PostgreSQL");

using var session = store.OpenSession();

var product = new Product
{
    Name = "C# 11 and .NET 7 - Modern Cross-Platform Fundamentals",
    Price = 46.87
};

session.Store(product);

await session.SaveChangesAsync();
```

We're creating a new `DocumentStore` instance which we use to open
a session to **PostgreSQL**. And then we just call `Store` and pass in
the `Product` instance. Note that **Marten** will populate the
`Product.Id` at this point. **Marten** can populate keys of `Guid`,
`int`, `long`, and other data types. It uses the HiLo algorithm
for numeric keys. Finally, when we call `SaveChangesAsync` the
`Product` is serialized into **JSON** and persisted as a document.

An important thing to be aware of is that the `IDocumentSession`
created by `OpenSession` doesn't track changes on the entities automatically.
You need to create a session with dirty checking enabled by
calling `DirtyTrackedSession` on the `DocumentStore` to enable
automatic change detection.

## Querying Documents With Marten

**Marten** has rich support for querying documents in the database.
You can write and execute queries using **LINQ**, which you are
familiar with if you worked with **EF Core**. And you can also
write and execute **SQL** queries, because it's still a **PostgreSQL**
database underneath.

Here's an example query to return products that have a higher price
than the one which is specified:

```csharp
var store = DocumentStore.For("Connection string to PostgreSQL");

using var session = store.QuerySession();

var products = session.Query<Product>().Where(p => p.Price > 9.99).ToList();
```

**Marten** also has support for:

- Including related documents
- Batched queries
- Paging
- Full text search

## Advanced Options With Marten

**Marten** can utilize the full capabilities **PostgreSQL** has to offer,
notably transactions and indexing. **Marten** sessions are transactional
by default, either all of the documents are persisted together or
none of them are. And you can configure indexes on your documents
for faster queries.

**Marten** isn't just a **document database** on top of **PostgreSQL**!

You have fully-fledged support for **event sourcing** with **Marten**,
as well as projections. This makes it a perfect choice for
implementing **CQRS**. But this is a topic for a separate newsletter.

## Closing Thoughts

I'm absolutely amazed with [Marten](https://martendb.io/) and what it has to offer.
And **PostgreSQL** is also my favorite database, so it's like a match
made in heaven. I don't get too excited about learning new
technologies, but **Marten** has been an endless source
of joy this past week.

I still need to explore a few more topics before I can consider it
for production use:

- Schema migrations
- Relationships and foreign keys
- Advanced configuration options

Considering that **PostgreSQL** is cheaper than most **document databases**,
I think using **Marten** is an interesting alternative. And if you are
familiar with **SQL** databases, you can still use all of that knowledge.

---



<Sponsorships>

Keeping your business logic and data in-sync doesn't have to be a drag.
Database configuration-as-code and auto-indexes mean your logic is always
querying against what you expect after every release.
[**Take off with RavenDB today**](https://ravendb.net/learn/bootcamp/?utm_campaign=mnw_newsletter_15&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_bootcamp&utm_content=ad_link)
and fly faster tomorrow.

</Sponsorships>

In this week's newsletter we are going to explore **Minimal APIs**,
which were introduced in **.NET 6**.

**Minimal APIs** were introduced to remove some of the ceremony
of creating traditional APIs with controllers.
To define an endpoint, you can use the new extension
methods, such as `MapGet` to define a **GET** endpoint.

I see one big issue with **Minimal APIs**, and that is the lack
of clear guidance around how to structure applications
built with **Minimal APIs**.

In this newsletter, I want to offer a few solutions for that problem.

Let's dive in.

## How To Create Minimal APIs?

Let's define a simple **Minimal API** application with two endpoints.
We're going to create one `GET` endpoint for getting a list of products.
And one `POST` endpoint for saving a product to the database.

We're using the powerful **DI** feature that allows us to inject services
as expression arguments, which you can see in the two expressions below
where we are injecting the `AppDbContext`.

```csharp
var builder = WebApplication.CreateBuilder(args);

// Configure EF and other services...

var app = builder.Build();

app.MapGet("/products", async (AppDbContext dbContext) =>
{
    return Results.Ok(await dbContext.Products.ToListAsync());
});

app.MapPost("/products", async (Product product, AppDbContext dbContext) =>
{
    dbContext.Products.Add(product);

    await dbContext.SaveChangesAsync();

    return Results.Ok(product);
});

app.Run();
```

And with this we have a functioning **Minimal API** that we can develop further
as we continue to add more endpoints.

## The Problem With Maintaining Minimal APIs

There is one potential problem with structuring our **Minimal APIs** like
in the previous example. If we keep adding the **Minimal API** endpoints
in the same file, our API will become hard to maintain as it grows
in complexity. How can we solve the maintance problem with **Minimal APIs**?

One solution can be to use extension methods to encapsulate the
definiton of the **Minimal API** endpoints.

Here's an example of that:

```csharp
public static class ProductsModule
{
    public static void RegisterProductsEndpoints(this IEndpointRouteBuilder  endpoints)
    {
        endpoints.MapGet("/products", async (AppDbContext dbContext) =>
        {
            return Results.Ok(await dbContext.Products.ToListAsync());
        });

        endpoints.MapPost("/products", async (Product product, AppDbContext dbContext) =>
        {
            dbContext.Products.Add(product);

            await dbContext.SaveChangesAsync();

            return Results.Ok(product);
        });
    }
}
```

And then inside of `Program` we need to register the endpoints:

```csharp
app.RegisterProductsEndpoints();
```

You can see that this simplifies our **Minimal API** definition,
and we also have our endpoints grouped by feature in one place.
I think this improve the maintainability of **Minimal APIs**,
but it comes at a cost. And that cost is having to define
extension methods for each group of endpoints you want to encapsulate,
and then you have to remember to call that extensions method in `Program`.

Can we do better?

## Structuring Minimal API Projects With Modules

I want to introduce you to an interesting open source library
called [Carter](https://github.com/CarterCommunity/Carter),
which has a concept of modules that we can use to group endpoints.

Here's how we can define our `ProductsModule` with **Carter**:

```csharp
public class ProductsModule : ICarterModule
{
    public void AddRoutes(IEndpointRouteBuilder app)
    {
        app.MapGet("/products", async (AppDbContext dbContext) =>
        {
            return Results.Ok(await dbContext.Products.ToListAsync());
        });

        app.MapPost("/products", async (Product product, AppDbContext dbContext) =>
        {
            dbContext.Products.Add(product);

            await dbContext.SaveChangesAsync();

            return Results.Ok(product);
        });
    }
}
```

This takes care of configuring our **Minimal API** endpoints, but we still
need to tell the framework to use these endpoints. We have to slightly
modify the `Program` to register the required **Carter** services by
calling `AddCarter`, and also map our endpoints by calling `MapCarter`.

```csharp
var builder = WebApplication.CreateBuilder(args);

// Configure EF and other services...

builder.Services.AddCarter();

var app = builder.Build();

app.MapCarter();

app.Run();
```

When we want to define additional **Minimal API** endpoints we just need to
implement a new `ICarterModule`, and register our endpoints. **Carter** will
automatically take care of registering the new endpoints after that.

## Would I Use Minimal APIs In a Real Project?

I think **Minimal APIs** have evolved nicely since they were first introduced
in **.NET 6**. I would be careful with using them in very large applications,
but I'm definitely going to explore options for using them on smaller projects.

A good use case can be for building a microservice that has a limited
number of endpoints.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source Visual Studio extension that lets you highlight
important objects and arbitrary texts to help you navigate your code more easily.

Sick of staying up late firefighting deployments?
Reclaim your nights with a database that keeps your app up even when things go sideways.
[**Outsource hosting to the experts with RavenDB Cloud today**](https://cloud.ravendb.net/?utm_campaign=mnw_newsletter_14&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_cloud&utm_content=ad_link)
so you can sleep well tomorrow.

</Sponsorships>

In this week's newsletter we will talk about running **background tasks** in **ASP.NET Core**.
After reading this newsletter, you will be able to set up a **background task**
and have it up and running within minutes.

**Background tasks** are used to offload some work in your application to the background,
outside of the normal application flow. A typical example can be asynchronously
processing messages from a queue.

I will show you how to create a simple **background task** that runs once and completes.

And you will also see how to configure a continuous **background task**, that repeats after a specific period.

Let's dive in.

## Background Tasks With IHostedService

You can define a **background task** by implementing the `IHostedService` interface.
It has only two methods.

Here's what the `IHostedService` interface looks like:

```csharp
public interface IHostedService
{
    Task StartAsync(CancellationToken cancellationToken);

    Task StopAsync(CancellationToken cancellationToken);
}
```

All you have to do is implement the `StartAsync` and `StopAsync` methods.

Inside of `StartAsync` you would usually perform the background processing.
And inside of `StopAsync` you would perform any cleanup that is necessary,
such as disposing of resources.

To configure the **background task** you have to call the `AddHostedService` method:

```csharp
builder.Services.AddHostedService<MyBackgroundTask>();
```

Calling `AddHostedService` will configure the **background task**
as a **singleton** service.

So does dependency injection still work in `IHostedService` implementations?\
Yes, but you can only inject **transient** or **singleton** services.

However, I don't like to implement the `IHostedService` interface myself.
I prefer using the `BackgroundService` class instead.

## Background Tasks With BackgroundService

The `BackgroundService` class already implements the `IHostedService` interface,
and it has an `abstract` method that you need to override - `ExecuteAsync`.
When you are using the `BackgroundService` class, you only have to think about
the operation you want to implement.

Here's an example **background task** that runs **EF** migrations:

```csharp
public class RunEfMigrationsBackgroundTask : BackgroundService
{
    private readonly IServiceProvider _serviceProvider;

    public RunEfMigrationsBackgroundTask(IServiceProvider serviceProvider)
    {
        _serviceProvider = serviceProvider;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using IServiceScope scope = _serviceProvider.CreateScope();

        await using AppDbContext dbContext =
            scope.ServiceProvider.GetRequiredService<AppDbContext>();

        await dbContext.Database.MigrateAsync(stoppingToken);
    }
}
```

The **EF** `DbContext` is a **scoped** service, which we can't inject directly
inside of `RunEfMigrationsBackgroundTask`. We have to inject an instance of
`IServiceProvider` which we can use to create a custom service scope,
so that we can resolve the scoped `AppDbContext`.

I would _not recommend_ running the `RunEfMigrationsBackgroundTask` in production.
**EF** migrations can easily fail and you'll run into problems.
However, I think it's perfectly fine for local development.

## Periodic Background Tasks

Sometimes we want run a **background task** continuously, and have it
perform some operation on repeat. For example, we want consume messages
from a queue every ten seconds. How do we build this?

Here's an example `PeriodicBackgroundTask` to get you started:

```csharp
public class PeriodicBackgroundTask : BackgroundService
{
    private readonly TimeSpan _period = TimeSpan.FromSeconds(5);
    private readonly ILogger<PeriodicBackgroundTask> _logger;

    public PeriodicBackgroundTask(ILogger<PeriodicBackgroundTask> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using PeriodicTimer timer = new PeriodicTimer(_period);

        while (!stoppingToken.IsCancellationRequested &&
               await timer.WaitForNextTickAsync(stoppingToken))
        {
            _logger.LogInformation("Executing PeriodicBackgroundTask");
        }
    }
}
```

We're using a [PeriodicTimer](https://learn.microsoft.com/en-us/dotnet/api/system.threading.periodictimer?view=net-6.0)
to asynchronously wait for a given period, before executing our **background task**.

## What If You Need A More Robust Solution?

It should be obvious by now that `IHostedService` is useful when you need
simple **background tasks** that are running while your application is running.

What if you want to have a scheduled **background task** that runs at 2AM every day?

You can probably build something like this yourself, but there are existing solutions
that you should consider first.

Here are two popular solutions for running **background tasks** that I worked with before:

- [Quartz](https://www.quartz-scheduler.net/)
- [Hangfire](https://www.hangfire.io/)

I also have an example of [using Quartz for processing Outbox messages](https://youtu.be/XALvnX7MPeo)
on my YouTube channel that you can take a look at.

---



<Sponsorships>

Sick of staying up late firefighting deployments?
Reclaim your nights with a database that keeps your app up even when things go sideways.
[**Outsource hosting to the experts with RavenDB Cloud today**](https://cloud.ravendb.net/?utm_campaign=mnw_newsletter_13&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_cloud&utm_content=ad_link)
so you can sleep well tomorrow.

</Sponsorships>

In this week's newsletter, we're going to explore the new
`ExecuteUpdate` and `ExecuteDelete` methods that were released with **EF7**.

`ExecuteUpdate` allows us to write a query and run a bulk update
operation on the entities matching that query.

Similarly, `ExecuteDelete` allows us to write a query and delete
the entities matching that query.

We can significantly improve performance using the new methods in some scenarios, and I'm going to show you what those scenarios are.

Let's dive in.

## Updating And Deleting Entities Before EF Core 7

If you want to update a collection of entities before **EF7**,
you need to load the entities into memory using the `DatabaseContext`.

The **EF ChangeTracker** will then track any changes made to these entities.
When you are ready to commit the changes to the database,
you simply call the `SaveChanges` method.

Here's an example where we load a few notifications,
and we want to snooze them so they aren't sent:

```csharp
var notifications = dbContext
    .Notifications
    .Where(n => !n.Snoozed)
    .ToList();

foreach(var notification in notifications)
{
    notification.Snoozed = true;
}

dbContext.SaveChanges();
```

**EF7** will generate the following **SQL** statement to update the records in the database:

```sql
UPDATE [Notifications] n
SET n.[Snoozed] = TRUE
WHERE n.[Id] = @notificationId_1;

...

UPDATE [Notifications] n
SET n.[Snoozed] = TRUE
WHERE n.[Id] = @notificationId_N;
```

Notice that for every notification we end up with one **SQL UPDATE** statement.
This won't scale well as the number of notifications increases.

## Updating Entities With ExecuteUpdate

With **EF7**, we now have access to the new `ExecuteUdpate` method.
It also has an asynchronous version - `ExecuteUpdateAsync`.

How do you use it?

You need to write a query that will select the records you want to update,
and then call the `ExecuteUpdate` method on the resulting `IQueryable`.

Let's rewrite the previous example using the new approach:

```csharp
dbContext
    .Notifications
    .Where(n => !n.Snoozed)
    .ExecuteUpdate(s => s.SetProperty(
        n => n.Snoozed,
        n => true));
```

In the call to `ExecuteUpdate` we call the `SetProperty` method to specify
which properties we want to update, and what values we want to set.
The `SetProperty` method can be called multiple times, if you need to update more than one property.

In this case, **EF7** will generate the following **SQL** query:

```sql
UPDATE n
SET n.[Snoozed] = TRUE
FROM [Notifications] AS n
WHERE n.[Snoozed] = FALSE;
```

Notice that this time we only have one **SQL** query being sent to the database.
This is a major performance improvement. It can be as much as 10x faster
than the old version, from my testing.

## Deleting Entities With ExecuteDelete

Let's also see how we can do bulk deletes using the `ExecuteDelete` and `ExecuteDeleteAsync` methods.

Again, you have to write a query that will select the records you want to delete,
and then call the `ExecuteDelete` method on the resulting `IQueryable`.

If you want to delete all snoozed notifications:

```csharp
dbContext
    .Notifications
    .Where(n => n.Snoozed)
    .ExecuteDelete();
```

And **EF7** will generate the following **SQL** query:

```sql
DELETE FROM n
FROM [Notifications] AS n
WHERE n.[Snoozed] = TRUE;
```

I think this will be incredibly useful when you want to
delete records in the database based on a specific condition.

## Transactions, Change Tracking And Query Filters With Bulk Methods

You need to be aware how transactions and change tracking
work with the new bulk methods. `ExecuteUpdate` and `ExecuteDelete` will
immediately go to database, and run the **SQL** query.

**What does this mean for transactions?**

If you want to run a bulk method together with other updates
applied with `SaveChanges`, by default they won't run in the same transaction.
You need to open an explicit **transaction** using the `DatabaseContext` to keep everything consistent.

**What does this mean for change tracking?**

`ExecuteUpdate` and `ExecuteDelete` run directly on the database, without loading any entities into memory.
**EF7** will not track these entities in the `ChangeTracker`.

If you have any database interceptors defined, they won't execute
after calling one of the bulk update methods.
This also means that if you override `SaveChanges`
to add custom behavior, it won't be called.

**Do Query Filters still work?**

Yes, **query filters** will be **correctly applied** when calling `ExecuteUpdate` or `ExecuteDelete`.

## When Should You Use The New Bulk Methods?

I think this is an excellent new addition to **EF7**,
and it solves a real problem when you need to run
a typical **UPDATE** or **DELETE** query with a
**WHERE** statement applied.

Previously, you had to write raw **SQL** and execute it using something like **Dapper**.

I will likely use this approach when it applies to my projects.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source Visual Studio extension that lets you highlight
important objects and arbitrary texts to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter I want to show you how you can use
the powerful **options pattern** in **ASP.NET Core 7**.

The **options pattern** uses classes to provide **strongly typed settings**
in your application at runtime.

The values for the **options** instance can come from multiple sources.
The typical use case is to provide the settings from application configuration.

You can configure the **options pattern** in a few different ways in **ASP.NET Core**.
I want to discuss some of the approaches and their potential benefits.

Let's dive in.

## Creating The Options Class

I want to set the stage first, by creating the **options** class and
explaining what settings we want to bind to it.

We want to configure JWT Authentication for our application,
so we decided to create the `JwtOptions` class to hold that configuration:

```csharp
public class JwtOptions
{
    public string Issuer { get; init; }
    public string Audience { get; init; }
    public string SecretKey { get; init; }
}
```

And let's imagine that inside of our `appsettings.json` file
we have the following configuration values:

```json
"Jwt": {
    "Issuer": "Gatherly",
    "Audience": "Gatherly",
    "SecretKey": "dont-tell-anyone!"
}
```

Alright, that's looking good. Now I want to show you a few ways to bind
the values from JSON to our `JwtOptions` class.

## Setting Up Options Pattern Using IConfiguration

The most straightforward approach is to use the `IConfiguration` instance
that we can access while registering services.

We need to call the `IServiceCollection.Configure<TOptions>` method,
and specify the `JwtOptions` as the generic argument:

```csharp
builder.Services.Configure<JwtOptions>(
    builder.Configuration.GetSection("Jwt"));
```

It doesn't get simpler than this, does it?

The only downside is that we are limited to the configuration
values provided through application configuration.

This can be extended to include environment variables and user secrets also.

## Setting Up Options Pattern Using IConfigureOptions

If you want a more robust approach, I have you covered.
We're going to use the `IConfigureOptions` interface to define a class
to configure our **strongly typed options**.

There are two steps that we need to follow in this case:

- Create the `IConfigureOptions` implementation
- Call `IServiceCollection.ConfigureOptions<TOptions>` with our `IConfigureOptions`
  implementation as the generic argument

To start off, we will create the `JwtOptionsSetup` class:

```csharp
public class JwtOptionsSetup : IConfigureOptions<JwtOptions>
{
    private const string SectionName = "Jwt";
    private readonly IConfiguration _configuration;

    public JwtOptionsSetup(IConfiguration configuration)
    {
        _configuration = configuration;
    }

    public void Configure(JwtOptions options)
    {
        _configuration
            .GetSection(SectionName)
            .Bind(options);
    }
}
```

We wrote more code, to achieve the same thing. Was it worth it?

Perhaps, if you consider that we now have access to **dependency injection** in the `JwtOptionsSetup` class.
This means that we can resolve other services that we can use to get the configuration values.

We also need to tell the application to use the `JwtOptionsSetup` class:

```csharp
builder.Services.ConfigureOptions<JwtOptionsSetup>();
```

When we try to inject our `JwtOptions` somewhere, the `JwtOptionsSetup.Configure`
method will be called first the calculate the correct values.

## Injecting Options With IOptions

We've seen a few examples for how to configure the **options pattern** with the `JwtOptions` class.

But how do we actually use the **options pattern**?

Easy, you just need to inject `IOptions<JwtOptions>` from the constructor.

I'll just show the `JwtProvider` constructor here, for brevity.

```csharp
public JwtProvider(IOptions<JwtOptions> options)
{
    _options = options.Value;
}
```

The actual `JwtOptions` instance is available on the `IOptions<JwtOptions>.Value` property.

The `IOptions` instance that we injected here is configured
as a **Singleton** in dependency injection. This is very important to be aware of.

## What About IOptionsSnapshot and IOptionsMonitor?

If you want to use the latest configuration values every time
you inject an **options** class, then injecting `IOptions` won't work.

However, you can use the `IOptionsSnapshot` interface instead:

- It provides the latest configuration snapshot (cached per request)
- It is registered as a **Scoped** service
- It detects configuration changes after application start

You can also use the `IOptionsMonitor` which retrieves the current
option values at any time, and it's a **Singleton** service.

## Wrapping up

The **options pattern** gives us a way to use strongly typed configuration classes in our application.

We can configure the options class in a simple way with [`IConfiguration`](#setting-up-options-pattern-using-iconfiguration),
or we can create an [`IConfigureOptions`](#setting-up-options-pattern-using-iconfigureoptions)
implementation if we need something more powerful.

When it comes to using the **options pattern**, we have three approaches:

- [`IOptions`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptions-1?view=dotnet-plat-ext-7.0)
- [`IOptionsSnapshot`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptionssnapshot-1?view=dotnet-plat-ext-7.0)
- [`IOptionsMonitor`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptionsmonitor-1?view=dotnet-plat-ext-7.0)

Deciding which of them to use in your application depends on what kind of behavior you want.
If you don't need to support refreshing configuration values
after application start, [`IOptions`](#injecting-options-with-ioptions) is a perfect solution.

---



<Sponsorships>

Wish you could calculate prices of your 2.68MM online products 3,000X faster without a batch job?
By ditching their 12-JOIN SQL incantations, the “Amazon of medical supplies”
went from pricing items in 3,000 milliseconds to 1 millisecond - all from within the database.
[**Learn how RavenDB revolutionized the world's largest online medical supply store.**](https://ravendb.net/articles/3000x-faster-database-transactions-for-the-worlds-largest-medical-supplier?utm_campaign=mnw_newsletter_11&utm_medium=blog_ad&utm_source=milan_jovanovic_sponsorship&utm_term=variation_use_case&utm_content=ad_link)

</Sponsorships>

In this week's newsletter I want to highlight a few interesting things
that are now available with the release of **C# 11** and **.NET 7**.

In case you missed it, **.NET 7** was released November 8th.

There are many new features, and you can be sure I had a hard time choosing which ones to highlight.

Here's what we are going to cover:

- [Required members](#required-members)
- [Generic attributes](#generic-attributes)
- [Static abstract members in interfaces](#static-abstract-members-in-interfaces)
- [`file` keyword](#file-keyword)
- [LINQ Order and OrderDescending](#linq-order-and-orderdescending)

Let's see what the new features look like!

## Required Members

We can now define a class member as required by using the `required` keyword.
It can be applied to a _field_ or _property_ and it tells the compiler
these members must be initialized by all constructors or by the object initializer.

Why is this useful?

Before **C# 11**, the only way to enforce a property being set was through a constructor.
If you used an object initializer you could bypass the constructor and not initialize some properties.

Here's how you can say that a property is required:

```csharp
public class ContentCreator
{
    public required string Firstname { get; init; }
    public string? MiddleName { get; init; }
    public required string LastName { get; init; }
}
```

If you try to create a new `ContentCreator` instance without initializing
the `required` properties you get a compile error:

```csharp
var creator = new ContentCreator
{
    FirstName = "Milan" // Error: No LastName
}
```

## Generic Attributes

You can now declare a _generic_ class whose base class is `Attribute`.

Before **C# 11**, if you wanted to pass in a type as a parameter
to an `Attribute` you would need to pass it through the constructor:

```csharp
public class TypedAttribute : Attribute
{
    public TypedAttribute(Type t) => Param = t;

    public Type Param { get; }
}
```

And here's how you would use it with the `typeof` operator:

```csharp
[TypedAttribute(typeof(int))]
public int Method() => default;
```

Using the generic attributes feature, you can now define it like this:

```csharp
public class TypedAttribute<T> : Attribute { ... }
```

Now, we can specify the type parameter as a generic argument:

```csharp
[TypedAttribute<int>()]
public int Method() => default;
```

## Static Abstract Members in Interfaces

This is a very interesting feature that allows abstracting of static operations.
An example of this would be operators.

```csharp
public interface IMonoid<TSelf> where TSelf : IMonoid<TSelf>
{
    public static abstract TSelf operator +(TSelf a, TSelf b);

    public static abstract TSelf Zero { get; }
}
```

How can we use the `IMonoid` interface?

It may be confusing at first, since the members are virtual
and there is no instance to call the virtual members on.
The solution is to use generics and let the compiler infer the rest.

Here's a simple example:

```csharp
T AddAll<T>(params T[] elements) where T : IMonoid<T>
{
    T result = T.Zero;

    foreach (var element in elements)
    {
         result += element;
    }

    return result;
}
```

If you want to learn more, check out the docs on
[static abstract interface methods](https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/static-virtual-interface-members#static-abstract-interface-methods)
and [generic math](https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/static-virtual-interface-members#generic-math).

## File Keyword

With the new `file` keyword you can define a type whose scope and visibility
is restricted to the file in which it is declared.

```csharp
file class HiddenClass
{
}
```

This feature is practical when used inside of source generators, to avoid collisions when naming generated types.

But you may be able to find a use for it in your application.

## LINQ Order and OrderDescending

The new `Order` and `OrderDescending` methods allow us to sort an
`IEnumerable`, which simplifies the code for sorting.

Here's an example of ordering an array:

```csharp
var array = new[] { 19, 91, 21 };

var arrayAsc = array.Order();

var arrayDesc = array.OrderDescending();
```

I want to highlight that `IQueryable` also supports the new methods.

## Will You Upgrade to .NET 7?

**.NET 7** is not an LTS (Long Term Support) release,
and will be in support until May 2024,
with **.NET 8** releasing in November 2023.

Here are a few reasons why you should consider upgrading:

- Major performance improvements
- New features in **.NET 7**
- New features in **EF Core 7**
- Easier migration to **.NET 8**

I will be moving some of my new projects from **.NET 6** to **.NET 7**.

And I will also upgrade all of my YouTube content to **.NET 7**.

---



<Sponsorships>

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source
[**Visual Studio extension**](https://github.com/QuadSpinner/Highlighter) that lets you highlight
important objects and arbitrary texts to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter, we will take a look at five different ways to check if a collection **contains duplicates**.

I'm going to explain the idea behind each **algorithm**, discuss the **algorithm complexity** (Big O Notation), and at the end, we'll look at some **benchmark results**.

The five approaches for finding a duplicate will use the:

- [`foreach`](#check-for-duplicates-with-foreach-loop) loop
- LINQ [`Any`](#check-for-duplicates-with-linq-any) method
- LINQ [`All`](#check-for-duplicates-with-linq-all) method
- LINQ [`Distinct`](#check-for-duplicates-with-linq-distinct) method
- LINQ [`ToHashSet`](#check-for-duplicates-with-linq-tohashset) method

Let's see how we can implement each approach!

## Check For Duplicates With ForEach Loop

The first implementation will use the `foreach` loop and the `HashSet` data structure.

Here's the code for the `ContainsDuplicates` method:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   HashSet<T> set = new();

   foreach(var element in enumerable)
   {
      if (!set.Add(element))
      {
         return true;
      }
   }

   return false;
}
```

The idea is simple:

- Loop through the collection
- Add each element to the `HashSet`
- When `HashSet.Add` returns false we found a duplicate
- If we loop through the entire collection there are no duplicates

In terms of **algorithm complexity**, this would be **O(n)** or linear complexity.
This is because there's only one iteration through the collection.

Adding an element to a `HashSet` is a constant operation - **O(1)**.
So it doesn't affect the overall complexity.

## Check For Duplicates With LINQ Any

We'll combine the idea from the previous implementation of
using the `HashSet` and pair it with the LINQ `Any`
method to iterate over the collection.

Here's the implementation for the `ContainsDuplicates` method:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   HashSet<T> set = new();

   return enumerable.Any(element => !set.Add(element));
}
```

You can see this implementation is significantly shorter.
But it works the same as the one with the `foreach` loop.

If any element in the collection satisfies the specified expression,
`Any` will _short-circuit_ and return `true`.
Otherwise, it will iterate over the entire collection and return `false`.

We're still looking at linear complexity here, **O(n)**.

## Check For Duplicates With LINQ All

For our third implementation, we will use the opposite
of the LINQ `Any` method - the LINQ `All` method.

Here's the implementation with LINQ `All`:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   HashSet<T> set = new();

   return !enumerable.All(set.Add);
}
```

The idea here is a little different than in the previous implementation.

`All` will return `true` if all elements in a collection
satisfy the specified expression.

If at least one element doesn't satisfy the condition -
in our case when a **duplicate** is found - it will _short-circuit_ and return `false`.

This is still linear complexity, **O(n)**.

## Check For Duplicates With LINQ Distinct

So far, we've seen a few implementations using the `HashSet` data structure.
Now let's consider a different approach.

We can use the LINQ `Distinct` method to check for duplicates.

Here's the code for the `ContainsDuplicates` method:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   return enumerable.Distinct().Count() != enumerable.Count();
}
```

The idea is first find the `Distinct` elements and `Count` them,
and then compare that to the number of all elements.

If the number of distinct elements is not equal to
the number of all elements, we have a **duplicate** value.

In terms of **algorithm complexity**, this is still linear complexity.

But we have at least two iterations through the collection
or three in the worst-case scenario.

We have one iteration for `Distinct` and one more
iteration for the call to `Count` right after that.
The last call to `Count` can return in constant time,
if the collection is an `array` or `List`.

## Check For Duplicates With LINQ ToHashSet

For the last implementation we will use the LINQ `ToHashSet` method.

It takes a collection and creates a `HashSet` instance from it.

Here's what the `ContainsDuplicates` implementation looks like:

```csharp
public bool ContainsDuplicates<T>(IEnumerable<T> enumerable)
{
   return enumerable.ToHashSet().Count != enumerable.Count();
}
```

We compare the number of elements in the `HashSet` to the number of elements in the collection.

If they are different, we have a **duplicate** value.

This is also linear complexity, **O(n)**.

## Benchmark Results

Now that we've seen our implementations let's put them to the test.

I ran the benchmark for collections of varying sizes:

- 100
- 1,000
- 10,000

Each collection contains exactly one duplicate value located somewhere around the middle of the collection.

Here are the results:

<Image src="blogs/mnw_010/benchmark.png" width={1339} height={627} />

The approach using the `foreach` loop comes out as the clear winner in terms of performance.

However, I would lean towards using the implementations with LINQ `Any` or `All` because of their simplicity.

You can find the [source code for the benchmark](https://github.com/m-jovanovic/find-duplicates-benchmark)
on my GitHub. Feel free to submit a PR with a faster implementation if you can think of one.

---



<Sponsorships>

Delivered straight to your inbox every week,
[**Microns.io**](https://www.microns.io/) is a newsletter that helps you discover the best micro-startups for sale.
Join 3,500+ entrepreneurs and investors looking for their next micro-startup
acquisition opportunity and, along the way, learn about how great deals are done.

[**QuadSpinner Highlighter**](https://github.com/QuadSpinner/Highlighter) is an open-source Visual Studio extension that lets
you highlight important objects and arbitrary texts to help you navigate your code more easily.

</Sponsorships>

In this week's newsletter, I'll show you how you can remove repetitive conditions in **EF Core** database queries.

Which kinds of queries fit this description?

An example would be when you implement **soft-delete**, and have to check if a record was **soft-deleted** or not in every query.

Also, it's practical if you're working in a multi-tenant system and need to specify a `tenantId` on every query.

**EF Core** has a powerful feature that can help you remove repetitive conditions from your code.

I'm talking about [Query Filters](https://learn.microsoft.com/en-us/ef/core/querying/filters).

Let's see how we can implement it.

## How To Apply Query Filters

Before introducing **Query Filters**, we will see how the standard approach looks.
We have an `Orders` table that supports **soft-deleting**.
And we never want to return **soft-deleted** orders.

We'll start with an `Order` entity that has an `IsDeleted` property.

```csharp
public class Order
{
   public int Id { get; set; }
   public bool IsDeleted { get; set; }
}
```

And we have a business requirement that we can only query orders that are not deleted.

Here's what an **EF** query to get a single `Order` might look like:

```csharp
dbContext
   .Orders
   .Where(order => !order.IsDeleted)
   .Where(order => order.Id == orderId)
   .FirstOrDefault();
```

This works perfectly for what we need to do.

However, we need to remember to apply this condition every time we want to query the `Order` entity.

Now, let's see how we can define a **Query Filter** on the `Order` entity to
apply this check when querying the database.

Inside of the `OnModelCreating` method on the database context, we need to
call the `HasQueryFilter` method and specify the expression we want:

```csharp
modelBuilder
   .Entity<Order>()
   .HasQueryFilter(order => !order.IsDeleted);
```

Now we can omit the **soft-delete** check from the previous **LINQ** expression:

```csharp
dbContext
   .Orders
   .Where(order => order.Id == orderId)
   .FirstOrDefault();
```

And this is the **SQL** that **EF** will generate with the **Query Filter**:

```sql
SELECT o.*
FROM Orders o
WHERE o.IsDeleted = FALSE AND o.Id = @orderId
```

## Disabling Query Filters

You may run into a situation where you need to disable **Query Filters** for a specific query.
Luckily, there is an easy way to do this.

In your **LINQ** expression, you need to call the `IgnoreQueryFilters` method,
and all the **Query Filters** configured for this entity will be disabled:

```csharp
dbContext
   .Orders
   .IgnoreQueryFilters()
   .Where(order => order.Id == orderId)
   .FirstOrDefault();
```

Be careful when doing this, as you can easily introduce unwanted behavior in your application.

## Good Things To Know Before Using Query Filters

Here are a few more details that you should know about **Query Filters** before using them.
Hopefully, this will save you some trouble if you decide to use them in your application.

**Configuring multiple Query Filters**

Configuring multiple **Query Filters** on the same entity will only apply the last one.
If you need more than one condition, you can do that with the logical `AND` operator (&&).

**Ignoring specific Query Filters**

If you need to ignore a specific expression in a **Query Filter** and leave the rest in place,
unfortunately, you can't do that. Only one **Query Filter** is allowed per entity type.

One solution is calling `IgnoreQueryFilters`, which will remove the configured **Query Filter**
for that entity type. And then manually apply the condition that you need for that specific query.

---



<Sponsorships>

This week's issue is sponsored by [**QuadSpinner**](https://quadspinner.com/).

</Sponsorships>

In this week's newsletter, we'll see how we can work with **locking** in **.NET 6**.

We won't talk about how the lock is actually implemented at the operating system level.
Instead, I will focus on application-level **locking** mechanisms.

**Locking** allows us to control how many **threads** can access some piece of code.
Why would you want to do this?

Usually because you want to protect access to **expensive resources**,
and you need the **concurrency control** that locking enables.

We will use a simple `BankAccount` class with a `Deposit` method
to illustrate how to implement locking.

## The C# Lock Statement

The C# language supports locking with the [`lock` statement](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/lock).
You can use the `lock` statement to define a code block that only one thread can access.

The `lock` statement acquires a mutual-exclusion lock (mutex) for a given object,
executes the statement block, and releases the lock.

```csharp
lock(_lock)
{
   // Your code...
}
```

Here `_lock` is a reference type, usually an `object` instance.

Let's see how we can implement the `BankAccount` class using the `lock` statement:

```csharp
public class BankAccount
{
   private static readonly object _lock = new();
   private decimal _balance;

   public void Deposit(decimal amount)
   {
      lock(_lock)
      {
         _balance += amount;
      }
   }
}
```

The first thread to reach and execute the `lock` statement will be allowed to update
the `_balance`. Any other threads will block until the lock is released.

## Locking With Semaphore

The [`Semaphore`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.semaphore?view=net-6.0)
class is another option we can use to achieve the same effect.

We'll use the `Semaphore` constructor to set the `initialCount` to 0,
which means that the `Semaphore` is open at the start.
And we will also set the `maximumCount` to 1,
which means that only one thread is allowed to enter the `Semaphore`.

Let's see how we can implement the `BankAccount` class using the `Semaphore`:

```csharp
public class BankAccount
{
   private static readonly Semaphore _semaphore = new(
      initialCount: 0,
      maximumCount: 1);

   private decimal _balance;

   public void Deposit(decimal amount)
   {
      _semaphore.WaitOne();

      _balance += amount;

      _semaphore.Release();
   }
}
```

To enter the `Semaphore`, we have to call the `WaitOne` method.

If no thread was previously inside, our thread is allowed
to enter the `Semaphore` and update the balance.

After updating the balance, we call the `Release` method to
release the `Semaphore` for other threads that might be waiting.

## Asynchronous Locking With SemaphoreSlim

What if we wanted to call an asynchronous method in a locked context?

We can't use the `lock` statement as it doesn't support asynchronous calls.
Awaiting an asynchronous call inside a `lock` statement will cause a compilation error.

The `Semaphore` class can solve this problem.

But I want to show you another option that we have, [`SemaphoreSlim`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.semaphoreslim?view=net-6.0).
It's a lightweight alternative to the `Semaphore` class and has `async` methods.

Let's see how we can implement the `BankAccount` class using `SemaphoreSlim`:

```csharp
public class BankAccount
{
   private static readonly SemaphoreSlim _semaphore = new(
      initialCount: 0,
      maximumCount: 1);

   private decimal _balance;

   public async Task Deposit(decimal amount)
   {
      await _semaphore.WaitAsync();

      _balance += amount;

      _semaphore.Release();
   }
}
```

Notice that I updated the `Deposit` method to return a `Task`.

This time, we're calling `WaitAsync` to block the current
thread until it can enter the semaphore.

After updating the balance, we call the `Release` method
to release the `SemaphoreSlim` like in the previous example.

## Are There Other Options For Locking in .NET?

So far I mentioned three options to implement locking:

- [`lock` statement](#the-c-lock-statement)
- [`Semaphore`](#locking-with-semaphore)
- [`SemaphoreSlim`](#asynchronous-locking-with-semaphoreslim)

However, **.NET** has other classes for **concurrency control** that you can
explore like
[`Monitor`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.monitor?view=net-6.0),
[`Mutex`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.mutex?view=net-6.0),
[`ReaderWriterLock`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.readerwriterlock?view=net-6.0)
and many more.

I hope you enjoyed this brief introduction to a very complex topic.

---



<Sponsorships>

This week's issue is sponsored by [**QuadSpinner**](https://quadspinner.com/).

</Sponsorships>

Performance optimization is my favorite thing about software engineering.
Over the last 5 years, I've encountered various performance problems
that taught me different ways to overcome them.

About a month ago, I ran into an issue with an API endpoint that wasn't scaling well.

This endpoint is used to calculate a report for an e-commerce web application.
It needed to talk to multiple modules (services) to gather all the necessary data,
combine it and perform the calculations.

I made a [post about it on LinkedIn](https://www.linkedin.com/feed/update/urn:li:activity:6966700329111310336/)
that resonated with many people.

<Image src="blogs/mnw_007/linkedin_post.png" width={762} height={742} />

In this newsletter, I want to break down what I did to achieve a **15x performance improvement**.

## Focus On Bottlenecks First

The first thing I do when I'm solving a performance problem
is determine where the slowest piece of the code is.
Fixing this part of the code will usually give the most significant improvement.

Solving one bottleneck can also reveal where the next bottleneck is.\
This is a continual process.

In my situation, there were a few bottlenecks:

- Calling the database from a loop
- Calling an external service multiple times
- Executing a complex calculation multiple times with identical parameters

How can you measure performance?

A simple approach can be using `System.Timers.Timer` where you
manually log execution times between method calls.
Or you can use a performance profiler.

## Reduce The Number of Round Trips

A round trip between your application and a database
(or some other service) can last 5-10ms, or more.
If you have many round trips in your flow, it's going to add up quickly.

Here are a few things you can do reduce the number of round trips:

1. Don't call the database from a loop. This can usually be solved with a simple query like this:

```sql
   SELECT * FROM [TableName] WHERE Id IN (list_of_ids)
```

2. Use a query that returns multiple result sets from the database.
   One library that supports this is [Dapper](https://github.com/DapperLib/Dapper), with the `QueryMultiple` method.

3. If you need to make multiple calls to another service, try to convert that into one call.
   And in the service, aggregate the required data and return everything at once.

## Parallelize External Calls

I had a situation where I was awaiting multiple asynchronous calls from a few services.
These calls had no dependencies on each other, so I used a simple technique
to gain a significant performance improvement.

Let's say you're awaiting two tasks:

```csharp
var task1Result = await CallService1Async();

var task2Result = await CallService2Async();

// Use the results.
```

A simple way to parallelize these calls is using the `Task.WhenAll` method:

```csharp
var task1 = CallService1Async();

var task2 = CallService2Async();

await Task.WhenAll(task1, task2);

// Use the results.
task1.Result;
task2.Result;
```

Notice that I'm directly accessing the `Result` property on the tasks.
This can be **detrimental** if you're using it to block on an asynchronous call,
and can even lead to deadlocks.

However, in this situation it is perfectly safe to do,
because the two tasks will have completed after the call to `Task.WhenAll` completes.

Of course, whether or not these tasks will be executed in parallel
when calling `Task.WhenAll` depends on a few factors, which I won't cover here.

## Caching As a Last Resort

I try to leave caching for the end, after I have exhausted
all other possibilities to improve performance.
While I love to use caching in general, I'm aware
it can introduce some unwanted behavior when data is stale.

You have to consider how long you can safely cache the data,
and how you are going to clear the cache if the underlying data changes.

In simple applications, I use `IMemoryCache`
that is available in **ASP.NET Core** out of the box.
But you can also use an external cache like [Redis](https://redis.io/).

A good candidate for caching is data that is frequently accessed, but rarely modified.

## Closing Thoughts

I think that for most Web applications,
performance optimization can be boiled down to the following approaches:

- [Focus on bottlenecks first](#focus-on-bottlenecks-first)
- [Reduce the number of round trips](#reduce-the-number-of-round-trips)
- [Parallelize external calls](#parallelize-external-calls)
- [Caching](#caching-as-a-last-resort)

I didn't talk about database optimization and indexes here,
but this should also be on your mind if the database is your bottleneck.

---



<Sponsorships>

This week's issue is sponsored by [**QuadSpinner**](https://quadspinner.com/).

</Sponsorships>

Let's imagine we have an existing `Repository` implementation, and we want to introduce caching to reduce the load on the database.

How can we achieve this without changing the original `Repository` implementation?

**Decorator pattern** is a structural design pattern that allows you
to introduce new behavior to an existing class, without modifying the original class in any way.

I'll show you how you can implement this with the **ASP.NET Core DI** container.

## How To Implement The Decorator Pattern

We'll start with an existing `MemberRepository` implementation that implements the `IMemberRepository` interface.

It has only one method, which loads the `Member` from the database.

Here's what the implementation looks like:

```csharp
public interface IMemberRepository
{
    Member GetById(int id);
}

public class MemberRepository : IMemberRepository
{
    private readonly DatabaseContext _dbContext;

    public MemberRepository(DatabaseContext dbContext)
    {
        _dbContext = dbContext;
    }

    public Member GetById(int id)
    {
        return _dbContext
            .Set<Member>()
            .First(member => member.Id == id);
    }
}
```

We want to introduce caching to the `MemberRepository` implementation without modifying the existing class.

To achieve this, we can use the **Decorator pattern** and create a wrapper around our `MemberRepository` implementation.

We can create a `CachingMemberRepository` that will have a dependency on `IMemberRepository`.

```csharp
public class CachingMemberRepository : IMemberRepository
{
    private readonly IMemberRepository _repository;
    private readonly IMemoryCache _cache;

    public CachingMemberRepository(
        IMemberRepository repository,
        IMemoryCache cache)
    {
        _repository = repository;
        _cache = cache;
    }

    public Member GetById(int id)
    {
        string key = $"members-{id}";

        return _cache.GetOrCreate(
            key,
            entry => {
                entry.SetAbsouluteExpiration(
                    TimeSpan.FromMinutes(5));

                return _repository.GetById(id);
            });
    }
}
```

Now I'm going to show you the power of **ASP.NET Core DI**.

We will configure the `IMemberRepository` to resolve an instance of `CachingMemberRepository`,
while it will receive the `MemberRepository` instance as its dependency.

## Configuring The Decorator In ASP .NET Core DI

For the DI container to be able to resolve `IMemberRepository` as `CachingMemberRepository`,
we need to manually configure the service.

We can use the overload that exposes a service provider,
that we will use to resolve the services required to construct a `MemberRepository`.

Here's what the configuration would look like:

```csharp
services.AddScoped<IMemberRepository>(provider => {
    var context = provider.GetService<DatabaseContext>();
    var cache = provider.GetService<IMemoryCache>();

    return new CachingRepository(
         new MemberRepository(context),
         cache);
});
```

Now you can inject the `IMemberRepository`, and the DI will be able to resolve an instance of `CachingMemberRepository`.

## Configuring The Decorator With Scrutor

If the previous approach seems _cumbersome_ to you and like a lot of manual work - that's because it is.

However, there is a simpler way to achieve the same behavior.

We can use the **[Scrutor](https://github.com/khellang/Scrutor)** library to register the decorator:

```csharp
services.AddScoped<IMemberRepository, MemberRepository>();

services.Decorate<IMemberRepository, CachingMemberRepository>();
```

**[Scrutor](https://github.com/khellang/Scrutor)** exposes the `Decorate` method.
The call to `Decorate` will register the `CachingMemberRepository` while ensuring
that it receives the expected `MemberRepository` instance as its dependency.

I think this approach is much simpler, and it's what I use in my projects.

---



In this newsletter, we'll be covering three ways to create middleware in **ASP.NET Core** applications.

**Middleware** allows us to introduce additional logic before or after executing an HTTP request.

You are already using many of the built-in middleware available in the framework.

I'm going to show you three approaches to how you can define custom middleware:

- [With Request Delegates](#adding-middleware-with-request-delegates)
- [By Convention](#adding-middleware-by-convention)
- [Factory-Based](#adding-factory-based-middleware)

Let's go over each of them and see how we can implement them in code.

## Adding Middleware With Request Delegates

The first approach to defining a middleware is by writing a **Request Delegate**.

You can do that by calling the `Use` method on the `WebApplication` instance
and providing a lambda method with two arguments.
The first argument is the `HttpContext` and the second argument is
the actual next request delegate in the pipeline `RequestDelegate`.

Here's what this would look like:

```csharp
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.Use(async (context, next) =>
{
    // Add code before request.

    await next(context);

    // Add code after request.
});
```

By awaiting the `next` delegate, you are continuing the request pipeline execution.
You can _short-circuit_ the pipeline by not invoking the `next` delegate.

This overload of the `Use` method is the one suggested by **Microsoft**.

## Adding Middleware By Convention

The second approach requires us to create a class that will represent our middleware.
We have to follow the convention when creating this class so that we can use it as middleware in our application.

I'm first going to show you what this class looks like, and then explain what is the convention we are following here.

Here's how this class would look like:

```csharp
public class ConventionMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger _logger;

    public ConventionMiddleware(
        RequestDelegate next,
        ILogger logger)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        _logger.LogInformation("Before request");

        await next(context);

        _logger.LogInformation("After request");
    }
}
```

The convention we are following has a few rules:

- We need to inject a `RequestDelegate` in the constructor
- We need to define an `InvokeAsync` method with an `HttpContext` argument
- We need to invoke the `RequestDelegate` and pass it the `HttpContext` instance

There's one more thing that's required, and that is to tell our application to use this middleware.

We can do that by calling the `UseMiddleware` method:

```csharp
app.UseMiddleware<ConventionMiddleware>();
```

And with this, we have a functioning middleware.

## Adding Factory-Based Middleware

The third and last approach requires us to also create a class that will represent our middleware.

However, this time we're going to implement the `IMiddleware` interface.
This interface has only one method - `InvokeAsync`.

Here's what this class would like:

```csharp
public class FactoryMiddleware : IMiddleware
{
    private readonly ILogger _logger;

    public FactoryMiddleware(ILogger logger)
    {
        _logger = logger;
    }

    public async Task InvokeAsync(
        HttpContext context,
        RequestDelegate next)
    {
        _logger.LogInformation("Before request");

        await next(context);

        _logger.LogInformation("After request");
    }
}
```

The `FactoryMiddleware` class will be resolved at runtime from dependency injection.

Because of this, we need to register it as a service:

```csharp
builder.Services.AddTransient<FactoryMiddleware>();
```

And like the previous example, we need to tell our application to use our factory-based middleware:

```csharp
app.UseMiddleware<FactoryMiddleware>();
```

With this, we have a functioning middleware.

## A Word On Strong Typing

I'm a big fan of **strong typing** whenever possible.
Out of the three approaches I just showed you, the one using the
`IMiddleware` interface satisfies this constraint the most.
This is also my **preferred** way to implement **middleware**.

Since we're implementing an interface, it's very easy to create
a generic solution to never forget to register your middleware.

You can use reflection to scan for classes implementing
the `IMiddleware` interface and add them to dependency injection,
and also add them to the application by calling `UseMiddleware`.

---



**Clean Architecture** is a popular approach to structuring your application.

It's a layered architecture that splits the project into four layers:

- [Domain](#domain-layer)
- [Application](#application-layer)
- [Infrastructure](#infrastructure-layer)
- [Presentation](#presentation-layer)

Each of the layers is typically one project in your solution.

Here's a visual representation of the **Clean Architecture**:

<Image src="blogs/mnw_004/clean_architecture.png" width={1920} height={1080} />

How do we create this in our .NET solutions?

## Domain Layer

The **Domain layer** sits at the core of the **Clean Architecture**.
Here we define things like: entities, value objects, aggregates, domain events, exceptions, repository interfaces, etc.

Here is the folder structure I like to use:

```
📁 Domain
|__ 📁 DomainEvents
|__ 📁 Entities
|__ 📁 Exceptions
|__ 📁 Repositories
|__ 📁 Shared
|__ 📁 ValueObjects
```

You can introduce more things here if you think it's required.

One thing to note is that the **Domain layer** is not allowed to reference other projects in your solution.

## Application Layer

The **Application layer** sits right above the **Domain layer**.
It acts as an orchestrator for the **Domain layer**, containing the most important use cases in your application.

You can structure your use cases using services or using commands and queries.

I'm a big fan of the **CQRS** pattern, so I like to use the command and query approach.

Here is the folder structure I like to use:

```
📁 Application
|__ 📁 Abstractions
    |__ 📁 Data
    |__ 📁 Email
    |__ 📁 Messaging
|__ 📁 Behaviors
|__ 📁 Contracts
|__ 📁 Entity1
    |__ 📁 Commands
    |__ 📁 Events
    |__ 📁 Queries
|__ 📁 Entity2
    |__ 📁 Commands
    |__ 📁 Events
    |__ 📁 Queries
```

In the `Abstractions` folder, I define the interfaces required for the **Application layer**.
The implementations for these interfaces are in one of the upper layers.

For every entity in the **Domain layer**, I create one folder with the commands, queries, and events definitions.

## Infrastructure Layer

The **Infrastructure layer** contains implementations for external-facing services.

What would fall into this category?

- Databases - PostgreSQL, MongoDB
- Identity providers - Auth0, Keycloak
- Emails providers
- Storage services - AWS S3, Azure Blob Storage
- Message queues - Rabbit MQ

Here is the folder structure I like to use:

```
📁 Infrastructure
|__ 📁 BackgroundJobs
|__ 📁 Services
    |__ 📁 Email
    |__ 📁 Messaging
|__ 📁 Persistence
    |__ 📁 EntityConfigurations
    |__ 📁 Migrations
    |__ 📁 Repositories
    |__ #️⃣ ApplicationDbContext.cs
|__ 📁 ...
```

I place my `DbContext` implementation here if I'm using **EF Core**.

It's not uncommon to make the Persistence folder its project.
I frequently do this to have all database facing-code inside of one project.

## Presentation Layer

The **Presentation layer** is the entry point to our system.
Typically, you would implement this as a Web API project.

The most important part of the **Presentation layer** is the `Controllers`, which define the API endpoints in our system.

Here is the folder structure I like to use:

```
📁 Presentation
|__ 📁 Controllers
|__ 📁 Middlewares
|__ 📁 ViewModels
|__ 📁 ...
|__ #️⃣ Program.cs
```

Sometimes, I will move the **Presentation layer** away from the actual Web API project.
I do this to isolate the `Controllers` and enforce stricter constraints.
You don't have to do this if it is too complicated for you.

## Is This The Only Way?

You don't have to follow the folder structure I proposed to the T.
**Clean Architecture** is very flexible, and you can experiment with it and structure it the way you like.

Do you like more granularity? Create more specific projects.

Do you dislike a lot of projects? Separate concerns using folders.

I'm here to give you options to explore. But it's up to you to decide what's best.

---



I recently ran into an issue with **Entity Framework Core**.

The query I was running was constantly timing out.

I tried to scale up the application server, and it didn't help.

I tried to scale up the database server, and it didn't help.

So how did I solve the problem?

## What Was The Problem With This Query?

I'm working on an application in the e-commerce domain.
To be specific, it's an order management system for a kitchen cabinet manufacturer.

The table that I frequently query on is the `Orders` table.
The `Order` can have one or more `LineItems`.
A typical `Order` will contain 50 `LineItems`.
Also, `LineItems` have a table that contains the valid dimensions - `LineItemDimensions`.

This is the query I was trying to run:

```csharp
dbContext
    .Orders
    .Include(order => order.LineItems)
    .ThenInclude(lineItem => lineItem.Dimensions)
    .First(order => order.Id == orderId);
```

When EF Core converts this into SQL, this is what it will send to the database:

```sql
SELECT o.*, li.*, d.*
FROM Orders o
LEFT JOIN LineItems li ON li.OrderId = o.Id
LEFT JOIN LineItemDimensions d ON d.LineItemId = li.Id
WHERE o.Id = @orderId
ORDER BY o.Id, li.Id, d.Id;
```

In most cases, this query will execute just fine.

However, in my situation I was running into the problem of _Cartesian Explosion_.
This is mainly because of the join to the `LineItemDimensions` table.
And this is what's causing my query to fail, and time out.

So how did I solve this problem?

## Query Splitting To The Rescue

With the release of **EF Core 5.0** we got a new feature called **Query Splitting**.
This allows us to specify that a given LINQ query should be split into multiple `SQL` queries.

To use **Query Splitting**, all you need to do is call the `AsSplitQuery` method:

```csharp
dbContext
    .Orders
    .Include(order => order.LineItems)
    .ThenInclude(lineItem => lineItem.Dimensions)
    .AsSplitQuery()
    .First(order => order.Id == orderId);
```

In this case, EF Core will generate the following SQL queries:

```sql
SELECT o.*
FROM Orders o
WHERE o.Id = @orderId;

SELECT li.*
FROM LineItems li
JOIN Orders o ON li.OrderId = o.Id
WHERE o.Id = @orderId;

SELECT d.*
FROM LineItemDimensions d
JOIN LineItems li ON d.LineItemId = li.Id
JOIN Orders o ON li.OrderId = o.Id
WHERE o.Id = @orderId;
```

Notice that for each `Include` statement we have a separate `SQL` query.
The benefit here is that we are not duplicating data when fetching from the database,
as we were in the previous case.

## Turning On Query Splitting For All Queries

You can enable **Query Splitting** at the database context level.
When configuring your database context you need to call the `UseQuerySplittingBehavior` method:

```csharp
services.AddDbContext<ApplicationDbContext>(options =>
    options.UseSqlServer(
        "CONNECTION_STRING",
        o => o.UseQuerySplittingBehavior(
            QuerySplittingBehavior.SplitQuery)));
```

This will cause all queries that EF Core generates to be split queries.
To revert back to a single query, you need to call the `AsSingleQuery` method:

```csharp
dbContext
    .Orders
    .Include(o => o.LineItems)
    .ThenInclude(li => li.Dimensions)
    .AsSingleQuery()
    .First(o => o.Id == orderId);
```

## What You Should Know About Query Splitting

Although query splitting is an excellent addition to EF Core, there are a few things you need to be aware of.

There is no consistency guarantee for multiple SQL queries.
You may run into a problem if you have a concurrent update going through
at the same time when you query your data.
To mitigate this, you can wrap the queries inside of a transaction, but this will only introduce performance issues elsewhere.

Each query will require one network round trip. This can degrade performance if your latency to the database is high.

Now that you are armed with this knowledge, go and make your EF queries faster!

---



Today, I'm going to share some fascinating things you can do with records and anonymous types.
I will introduce you to the concept of non-destructive mutation.
And I will talk about when and why we might want to use this C# language feature.

## What Is a Record?

With **C# 9** we can use **records** that are a new reference type.
**C# 10** introduced record structs so that you can define records as value types.
Records are distinct from classes in that record types use value-based equality.

Let's see how we would define a `record`:

```csharp
public record Food(string Name, double Price);
```

This way of declaring a `record` is called a positional record.
The constructor we have defined here is called the **primary constructor**.

The `Name` and `Price` properties are init only properties.
This means they can only be set in the constructor or using a property initializer.

Since our properties are init only, is there any way to change their value?

## Non-Destructive Mutation Using The With Expression

We said we can't modify the properties of our `record`, because the properties are init only.
However, we can use the `with` expression (introduced in **C# 9**) to create a new instance
of our record with modified values.

Let's see how we would use the `with` expression:

```csharp
var banana = new Food("🍌", 1.95);

var bananaOnSale = banana with
{
    Price = 0.99
};
```

It's important to highlight two things here:

- The original banana instance remains unchanged
- The `with` expression creates a new record instance with only the `Price` property modified

I mentioned Anonymous Types in the title, so let me show you something interesting you can do with them.

## Anonymous Types And Non-Destructive Mutation

Did you know that you can use the `with` expression with anonymous types?

Just a reminder that the `with` expression is available from **C# 9** and later.

Let's create an anonymous type:

```csharp
var apple = new
{
    Name = "🍎",
    Price = 1.21
};
```

This is how we can modify it using the with expression:

```csharp
var orange = apple with
{
    Name = "🍊"
};
```

And again the same rules apply:

- The original apple instance remains unchanged
- The with expression creates a new anonymous type instance with only the Name property modified

I found this feature useful in LINQ method chains.

For example, loading an anonymous type from the database where some properties have a default value.
You can then use this feature to calculate the values for these properties in memory.

---



## Wishing You a Warm Welcome

First, I want to welcome you to the first edition of **Milan's .NET Weekly** newsletter.

I hope that this newsletter can become a positive force in the .NET community.
To bring many of us together so that we can all continue learning and improving.

With that out of the way, let's get into .NET!

## The Problem With Wide LINQ

Let's consider the following LINQ expression from a code style perspective.

I call this a wide LINQ expression because it stretches horizontally across the entire screen.

```csharp
dbContext.Animals.Where(animal => animal.HasBigEars)
    .OrderBy(animal => animal.IsDangerous).Select(
        animal => (animal.Id, animal.Name)).ToList();
```

- It is difficult to read.
- It is difficult to reason about.
- It is difficult to extend or maintain.

To improve this, I created a simple rule that you can follow:

> When writing LINQ, try to go tall, not wide.

## How to Write Tall LINQ

So how do we write tall LINQ expressions?

I'm going to rewrite the previous expression, to improve it.

Try to follow the _one dot per line rule_:

```csharp
dbContext
    .Animals
    .Where(animal => animal.HasBigEars)
    .OrderBy(animal => animal.IsDangerous)
    .Select(animal => (animal.Id, animal.Name))
    .ToList();
```

Is the new version easier to read? **Yes**, very much so.

It is easier to understand what each expression does,
and how it feeds into the next one in the chain.

If you are working in a team, try to propose this as a coding standard (if it isn't one already).\
You will see that over time this will make a noticeable difference.

---


Searched: "log/search"